Araştırıken şöyle;
http://stackoverflow.com/questions/4...ith-robots-txt

Birde şu şekilde;
RewriteEngine On

RewriteCond% {HTTP_USER_AGENT} (Googlebot | Bingbot | Baiduspider) [NC]
RewriteRule * -. [R = 403, L]

http://stackoverflow.com/questions/1...-with-htaccess

konular buldum denemekte fayda var.