Birkaç Seo Ayarı Robots.txt ve .Htaccess

S.a yabancı bir sitede aşşağıdaki ayarları buldum. Bunu birileri bize yorumlarsa iyi olur ben bazılarını kendi kafamdan ayarlıyıp kullandım.

Google Says

Make use of the robots.txt file on your web server. This file tells crawlers which directories can or cannot be crawled. Make sure it’s current for your site so that you don’t accidentally block the Googlebot crawler.


header.php meta seo trick

Place this in your wordpress themes header.php file, if the page is a single, page, or if its the home page then the robots will index and follow links on it. Otherwise search engines will not index the pages but will still follow the links.


<?php if(is_single() || is_page() || is_home()) { ?>
<meta name="googlebot" content="index,noarchive,follow,noodp" />
<meta name="robots" content="all,index,follow" />
<meta name="msnbot" content="all,index,follow" />
<?php } else { ?>
<meta name="googlebot" content="noindex,noarchive,follow,noodp" />
<meta name="robots" content="noindex,follow" />
<meta name="msnbot" content="noindex,follow" />
<?php }?>




seo robots.txt

See the Updated WordPress robots.txt file


User-agent: *
# disallow all files in these directories
Disallow: /cgi-bin/
Disallow: /z/j/
Disallow: /z/c/
Disallow: /stats/
Disallow: /dh_
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /contact/
Disallow: /tag/
Disallow: /wp-content/b
Disallow: /wp-content/p
Disallow: /wp-content/themes/askapache/4
Disallow: /wp-content/themes/askapache/c
Disallow: /wp-content/themes/askapache/d
Disallow: /wp-content/themes/askapache/f
Disallow: /wp-content/themes/askapache/h
Disallow: /wp-content/themes/askapache/in
Disallow: /wp-content/themes/askapache/p
Disallow: /wp-content/themes/askapache/s
Disallow: /trackback/
Disallow: /*?*
Disallow: */trackback/


User-agent: Googlebot
# disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.cgi$
Disallow: /*.wmv$
Disallow: /*.png$
Disallow: /*.gif$
Disallow: /*.jpg$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.php*
Disallow: */trackback*
Disallow: /*?*
Disallow: /z/
Disallow: /wp-*
Allow: /wp-content/uploads/


# allow google image bot to search all images
User-agent: Googlebot-Image
Allow: /*

# allow adsense bot on entire site
User-agent: Mediapartners-Google*
Disallow: /*?*
Allow: /z/
Allow: /about/
Allow: /contact/
Allow: /wp-content/
Allow: /tag/
Allow: /manual/*
Allow: /docs/*
Allow: /*.php$
Allow: /*.js$
Allow: /*.inc$
Allow: /*.css$
Allow: /*.gz$
Allow: /*.cgi$
Allow: /*.wmv$
Allow: /*.cgi$
Allow: /*.xhtml$
Allow: /*.php*
Allow: /*.gif$
Allow: /*.jpg$
Allow: /*.png$

# disallow archiving site
User-agent: ia_archiver
Disallow: /

# disable duggmirror
User-agent: duggmirror
Disallow: /


The Breakdown

disallow files in these directories


User-agent: *
Disallow: /cgi-bin/
Disallow: /z/j/
Disallow: /z/c/
Disallow: /stats/
Disallow: /dh_
Disallow: /about/
Disallow: /contact/
Disallow: /tag/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /contact
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/


disallow all files ending with these extensions


User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.cgi$
Disallow: /*.xhtml$


disallow all files with ? in url


Disallow: /*?*


disable duggmirror


User-agent: duggmirror
Disallow: /


disallow WayBack archiving site


User-agent: ia_archiver
Disallow: /


allow google image bot to search all images


User-agent: Googlebot-Image
Disallow:
Allow: /*


allow adsense bot on entire site


User-agent: Mediapartners-Google*
Disallow:
Allow: /*




Google User-agents


Googlebot

crawl pages from our web index and our news index
Googlebot-Mobile

crawls pages for our mobile index
Googlebot-Image

crawls pages for our image index
Mediapartners-Google

crawls pages to determine AdSense content. We only use this bot to crawl your site if you show AdSense ads on your site.
Adsbot-Google

crawls pages to measure AdWords landing page quality. We only use this bot if you use Google AdWords to advertise your site. Find out more about this bot and how to block it from portions of your site.

ia_archiver , alexa ile birlikte çalışan webarchiv botudur, engellerseniz, alexa sizi indexlemez

Teşekkürler. Sizin önerdiğiniz Robots.txt ve Sağlam .htaccess varmı?