0
Kayıt Ol

Ai uyumlu robots.txt

2

●

156

  • 20-08-2026, 15:20:46
    #1
    etiyazilim
    etiyazilim
    Detaylı 1
    User-agent: *
    User-agent: OAI-SearchBot
    User-agent: Claude-SearchBot
    User-agent: PerplexityBot
    User-agent: meta-webindexer
    User-agent: Applebot
    User-agent: DuckAssistBot
    User-agent: YouBot
    User-agent: Bravebot
    User-agent: PetalBot
    User-agent: Andibot
    User-agent: PhindBot
    User-agent: LinerBot
    User-agent: iaskspider/2.0
    User-agent: Google-Extended
    User-agent: ClaudeBot
    User-agent: GPTBot
    User-agent: Applebot-Extended
    User-agent: meta-externalagent
    User-agent: CCBot
    User-agent: Bytespider
    User-agent: TikTokSpider
    User-agent: DeepSeekBot
    User-agent: PanguBot
    User-agent: ChatGLM-Spider
    User-agent: Amazonbot
    User-agent: FacebookBot
    User-agent: Google-CloudVertexBot
    User-agent: AI2Bot
    User-agent: Ai2Bot-Dolma
    User-agent: cohere-training-data-crawler
    User-agent: ICC-Crawler
    User-agent: SBIntuitionsBot
    User-agent: YandexAdditional
    User-agent: Timpibot
    User-agent: img2dataset
    User-agent: anthropic-ai
    User-agent: Claude-Web
    User-agent: ChatGPT-User
    User-agent: Claude-User
    User-agent: Perplexity-User
    User-agent: meta-externalfetcher
    User-agent: MistralAI-User
    User-agent: Google-GeminiNotebook
    User-agent: Google-NotebookLM
    User-agent: ia_archiver
    User-agent: archive.org_bot
    User-agent: redditbot
    User-agent: Pinterestbot
    User-agent: WhatsApp
    User-agent: TelegramBot
    User-agent: Discordbot
    User-agent: Slackbot-LinkExpanding
    User-agent: LinkedInBot
    User-agent: Twitterbot
    User-agent: facebookexternalhit
    User-agent: AhrefsBot
    User-agent: AhrefsSiteAudit
    User-agent: SemrushBot
    User-agent: SemrushBot-OCOB
    User-agent: SemrushBot-SWA
    User-agent: DataForSeoBot
    User-agent: MJ12bot
    User-agent: DotBot
    User-agent: rogerbot
    User-agent: Screaming Frog SEO Spider
    User-agent: Barkrowler
    User-agent: BLEXBot
    User-agent: serpstatbot
    User-agent: AdsBot-Google
    User-agent: AdsBot-Google-Mobile
    User-agent: Mediapartners-Google
    User-agent: OAI-AdsBot
    User-agent: adidxbot
    User-agent: meta-externalads
    User-agent: Google-Safety
    User-agent: APIs-Google
    User-agent: Google-Site-Verification
    User-agent: FeedFetcher-Google
    User-agent: GoogleProducer
    User-agent: Google-Pinpoint
    User-agent: GoogleMessages
    User-agent: Google-CWS
    User-agent: Googlebot
    User-agent: Googlebot-Image
    User-agent: Googlebot-Video
    User-agent: Googlebot-News
    User-agent: Storebot-Google
    User-agent: Google-InspectionTool
    User-agent: GoogleOther
    User-agent: GoogleOther-Image
    User-agent: GoogleOther-Video
    User-agent: bingbot
    User-agent: DuckDuckBot
    User-agent: YandexBot
    User-agent: Baiduspider
    User-agent: Sogou web spider
    User-agent: Yeti
    User-agent: SeznamBot
    User-agent: Qwantbot
    User-agent: Slurp
    User-agent: Diffbot
    User-agent: omgili
    User-agent: omgilibot
    User-agent: Webzio-Extended
    User-agent: Brightbot 1.0
    User-agent: ApifyBot
    User-agent: FirecrawlAgent
    User-agent: TavilyBot
    User-agent: ExaBot
    User-agent: ShapBot
    User-agent: Mozilla-Tabstack
    User-agent: Crawlspace
    User-agent: TerraCotta
    User-agent: bedrockbot
    User-agent: amazon-kendra
    User-agent: atlassian-bot
    User-agent: Cloudflare-AutoRAG
    User-agent: ImagesiftBot
    User-agent: Scrapy
    User-agent: ISSCyberRiskCrawler
    User-agent: VelenPublicWebCrawler
    User-agent: Claude-Code
    User-agent: Cursor
    User-agent: Devin
    User-agent: Google-Gemini-CLI
    User-agent: Trae
    User-agent: opencode
    User-agent: ChatGPT Agent
    User-agent: Operator
    User-agent: Google-Agent
    User-agent: GoogleAgent-Mariner
    User-agent: Manus-User
    User-agent: NovaAct
    User-agent: AmazonBuyForMe
    User-agent: TwinAgent
    User-agent: Gemini-Deep-Research
    User-agent: GoogleAgent-URLContext
    User-agent: Google-Read-Aloud
    User-agent: kagi-fetcher
    User-agent: Kimi-User
    User-agent: TongyiBot
    User-agent: cohere-ai
    User-agent: Amzn-User
    User-agent: YiyanBot
    Allow: /
    Disallow: /admin/
    Disallow: /cart/
    Disallow: /search/


    Content-Signal: ai-train=yes, search=yes, ai-input=yes

    Sitemap: https://www.siteniz.com/sitemap.xml


    Standart 2
    Allow: /
    Disallow: /admin/
    Disallow: /cart/
    Disallow: /search/

    Content-Signal: ai-train=yes, search=yes, ai-input=yes

    Sitemap: https://www.siteniz.com/sitemap.xml
  • 23-08-2026, 17:44:55
    #2
    SporYazari
    SporYazari
    Bunu kabul etmiyor hocam
    Content-Signal: ai-train=yes, search=yes, ai-input=yes
  • 23-08-2026, 20:47:03
    #3
    etiyazilim
    etiyazilim
    SporYazari adlı üyeden alıntı: mesajı görüntüle
    Bunu kabul etmiyor hocam
    Content-Signal: ai-train=yes, search=yes, ai-input=yes
    Googlebot ve GSC standardı dışındaki satırları doğrudan yok sayar (ignore eder) ve sitenin indekslenmesini engellemez.
    Content-Signal Amacı: Bu direktif, web sitelerinin yapay zeka botlarına içerik izinlerini bildirmesi amacıyla topluluk tarafından önerilen yeni bir taslaktır. Henüz resmi arama motoru standardı haline gelmediği için araçlar bunu "tanınmayan sözdizimi" olarak işaretler
    Uyarıyı GSC panelinde görmek istemiyorsanız; Header kısmına ekleyebilir ve halihazırda dosyanızda bulunan User-agent: GPTBot, User-agent: CCBot gibi botlara özel Allow veya Disallow kurallarını kullanmanız yeterli