RBMedya adlı üyeden alıntı:
mesajı görüntüle
Sitemi Semrush, Ahrefs Gibi Araçlardan Analizini nasıl kapatabilirim? BİLİN KİŞİ YOKMU???
24
●802
- 20-09-2024, 21:42:12ahrefs ya da semrush botları robots.txt kurallarına uyar fakat misal siz robots.txt ye sonradan eklerseniz bu engellemeyi direkt bu kurala uymaz çünkü kendi önbelleğinde izin verdiğinizi varsayarak işlemi yapar bir süre sonra tekrar kontrol ederse kurala uyar.
php ile gethostbyaddr() ve preg match vs. kullanarak engelleyebilirsiniz.
Tabi bu araçları sadece siteye sokmaya engellemeye yarar misal semrush yine arama sonuçlarına bakarak puanlama ya da diğer sitelerden alınan backlinkleri vs. görüp analizini yapar. - 21-09-2024, 18:55:35Nookta adlı üyeden alıntı: mesajı görüntüleByHazerfen adlı üyeden alıntı: mesajı görüntüle
halen kapatamadım koca r10'da halledecek yok mu bu işi ya .d - 22-09-2024, 01:05:00.htaccess kodları
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR] RewriteCond %{HTTP_USER_AGENT} ^Sogou\ web\ spider [OR] RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR] RewriteCond %{HTTP_USER_AGENT} ^PetalBot [OR] RewriteCond %{HTTP_USER_AGENT} ^Custo [OR] RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR] RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR] RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR] RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR] RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR] RewriteCond %{HTTP_USER_AGENT} ^HMView [OR] RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^Indy\ Library [OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR] RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR] RewriteCond %{HTTP_USER_AGENT} ^larbin [OR] RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR] RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR] RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR] RewriteCond %{HTTP_USER_AGENT} ^semalt [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR] RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR] RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR] RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR] RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR] RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR] RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR] RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR] RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR] RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget [OR] RewriteCond %{HTTP_USER_AGENT} ^Widow [OR] RewriteCond %{HTTP_USER_AGENT} ^SemrushBot [OR] RewriteCond %{HTTP_USER_AGENT} ^Baiduspider [OR] RewriteCond %{HTTP_USER_AGENT} ^BLEXBot [OR] RewriteCond %{HTTP_USER_AGENT} ^DotBot [OR] RewriteCond %{HTTP_USER_AGENT} ^Exabot [OR] RewriteCond %{HTTP_USER_AGENT} ^SeznamBot [OR] RewriteCond %{HTTP_USER_AGENT} ^aiHitBot [OR] RewriteCond %{HTTP_USER_AGENT} ^spbot [OR] RewriteCond %{HTTP_USER_AGENT} ^MJ12bot [OR] RewriteCond %{HTTP_USER_AGENT} ^oBot [OR] RewriteCond %{HTTP_USER_AGENT} ^DeuSu [OR] RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR] RewriteCond %{HTTP_USER_AGENT} ^MetaURI [OR] RewriteCond %{HTTP_USER_AGENT} ^FlipboardProxy [OR] RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR] RewriteCond %{HTTP_USER_AGENT} ^AhrefsBot [OR] RewriteCond %{HTTP_USER_AGENT} ^ZoominfoBot [OR] RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Zeus RewriteRule ^.* - [F,L]robots.txt kodları
User-agent: Chrome-Lighthouse Allow: / User-agent: Googlebot Allow: / User-agent: dotbot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: SemrushBot-SI Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: SemrushBot-CT Disallow: / User-agent: SemrushBot-BM Disallow: / User-agent: SplitSignalBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: Zoominfobot Disallow: / User-agent: EtaoSpider Disallow: / User-agent: Baiduspider Disallow: / User-agent: CazoodleBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: dotbot/1.0 Disallow: / User-agent: Gigabot Disallow: / User-agent: BLEXBot Disallow: / User-agent: PetalBot Disallow: /
bunları bi denermisin? - 22-09-2024, 01:23:351- Cloudflare WAF üzerinden ahrefs ve semrush ın agent ını engelleyebilirsiniz.
2- Ahrefs ve Semrush araçlarını ne kadar engellerseniz engelleyin sitenizi her türlü taramanın yolunu bulurlar. Çünkü mesela backlink aldığınız kaynağın sayfasını incelediğinde size backlink aldığınız için puan yazacak ve her türlü puanlandırmaya dahil edileceksiniz. Ancak semrush ve ahrefe kurumsal mail atarak sitenizin komple taranmaya kapatılmasını talep edebilirsiniz ki bunu da yapacaklarınız düşünmüyorum.
Google üzerinden gene analiz eder gibime geliyor