Arkadaşlar WordPress için robots.txt aramaları yaparken internetten bulduklarımı bir araya getirip siz profesyonellerin eline döküyorum. Bu robots.txt dosyasına eklenmesi gerekenler, çıkartılması gerekenler hangileridir.

User-agent: *
Disallow: /wp-*
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/
Disallow: /wp-content/plugins
Disallow: /wp-content/themes
Disallow: /author
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /print
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: */print
Disallow: */respond
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.css$
Disallow: /*?*
Disallow: /*?
Disallow: /wp-activate.php
Disallow: /wp-app.php
Disallow: /wp-atom.php
Disallow: /wp-blog-header.php
Disallow: /wp-comments-post.php
Disallow: /wp-commentsrss2.php
Disallow: /wp-config.php
Disallow: /wp-cron.php
Disallow: /wp-feed.php
Disallow: /wp-links-opml.php
Disallow: /wp-load.php
Disallow: /wp-login.php
Disallow: /wp-mail.php
Disallow: /wp-pass.php
Disallow: /wp-rdf.php
Disallow: /wp-register.php
Disallow: /wp-rss.php
Disallow: /wp-rss2.php
Disallow: /wp-settings.php
Disallow: /wp-signup.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php


Allow: /wp-content/uploads/


User-agent: Mediapartners-Google
Allow: /

User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Image
Allow: /

User-agent: Googlebot-Mobile
Allow: /


Sitemap: http://www.siteadi.com.tr/sitemap.xml
Sitemap: http://www.siteadi.com.tr/sitemap.xml.gz