tarama bütçesini olumsuz etkiler

robots.txt dosyasına bazı kurallar ekleyerek engellemek gerekir. örnek olarak;

User-Agent: *
Allow: /wp-content/uploads/
Disallow: /wp-content/plugins/
Disallow: /wp-admin/
Disallow: /author/
Disallow:/?s*
Disallow: /*?sort*
Disallow: /*=DESC
Disallow: /*=ASC
Disallow: /*search&tag=*
Disallow: /*search&search=*
Disallow: /archives/
Disallow: /cgi-bin/
Disallow: /feed/
Disallow: /trackback/