• 26-06-2008, 13:37:51
    #1
    arkadaslar arastırdım ama cogu yerde farklı olarak anlatılıyor sızce asagıdakı robots.txt
    aşagıdakılerden hangısı ıyı olur kafam karıstı gıttı
    1.inci robots.txt
    ]# BEGIN XML-SITEMAP-PLUGIN
    Sitemap: [url]http://www.siteninadresi/sitemap.xml.gz
    # END XML-SITEMAP-PLUGIN
    2.inci robots.txt
    User-agent: *
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /trackback
    Disallow: /feed
    Disallow: /comments
    Disallow: /category/*/*
    Disallow: */comments
    Disallow: /*?*
    Disallow: /*?
    Allow: /wp-content/uploads
    # Google Image
    User-agent: Googlebot-Image
    Disallow:
    Allow: /*
    # Google AdSense
    User-agent: Mediapartners-Google*
    Disallow:
    Allow: /*
    # Internet Archiver Wayback Machine
    User-agent: ia_archiver
    Disallow: /
    # digg mirror
    User-agent: duggmirror
    Disallow: /
    Sitemap: http://www.sitenin adresi/sitemap.xml
    3. robots.txt
    User-agent: *
    Disallow: /wp-app.php
    Disallow: /wp-atom.php
    Disallow: /wp-blog-header.php
    Disallow: /wp-comments-posts.php
    Disallow: /wp-commentsrss2.php
    Disallow: /wp-config.php
    Disallow: /wp-cron.php
    Disallow: /wp-feed.php
    Disallow: /wp-links-opml.php
    Disallow: /wp-login.php
    Disallow: /wp-mail.php
    Disallow: /wp-pass.php
    Disallow: /wp-rdf.php
    Disallow: /wp-register.php
    Disallow: /wp-rss.php
    Disallow: /wp-rss2.php
    Disallow: /wp-settings.php
    Disallow: /wp-trackback.php
    Disallow: /xmlrpc.php
    Disallow: /*?*
    4. robots.txt
    # This rule means it applies to all user-agents
    User-agent: *
    
    # Disallow all directories and files within
    Disallow: /cgi-bin/
    Disallow: /stats/
    Disallow: /dh_
    Disallow: /about/legal-notice/
    Disallow: /about/copyright-policy/
    Disallow: /about/terms-and-conditions/
    Disallow: /contact/
    Disallow: /tag/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    
    # Disallow all monthly archive pages
    Disallow: /2006/0
    Disallow: /2006/1
    Disallow: /2007/0
    Disallow: /2007/1
    
    # The Googlebot is the main search bot for google
    User-agent: Googlebot
    
    # Disallow all files ending with these extensions
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.gz$
    Disallow: /*.wmv$
    Disallow: /*.tar$
    Disallow: /*.tgz$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    
    # Disallow Google from parsing indididual post feeds and trackbacks..
    Disallow: */feed/
    Disallow: */trackback/
    
    # Disallow all files with ? in url
    Disallow: /*?*
    Disallow: /*?
    
    # Disallow all archived monthlies
    Disallow: /2006/0*
    Disallow: /2007/0*
    Disallow: /2006/1*
    Disallow: /2007/1*
    
    # The Googlebot-Image is the image bot for google
    User-agent: Googlebot-Image
    
    # Allow Everything
    Allow: /*
    
    # This is the ad bot for google
    User-agent: Mediapartners-Google*
    
    # Allow Everything
    Allow: /*
  • 26-06-2008, 14:05:13
    #2
    tüm sitelerimde bunu kullanıyorum

    User-agent: *
    Disallow: /wp-app.php
    Disallow: /wp-atom.php
    Disallow: /wp-blog-header.php
    Disallow: /wp-comments-posts.php
    Disallow: /wp-commentsrss2.php
    Disallow: /wp-config.php
    Disallow: /wp-cron.php
    Disallow: /wp-feed.php
    Disallow: /wp-links-opml.php
    Disallow: /wp-login.php
    Disallow: /wp-mail.php
    Disallow: /wp-pass.php
    Disallow: /wp-rdf.php
    Disallow: /wp-register.php
    Disallow: /wp-rss.php
    Disallow: /wp-rss2.php
    Disallow: /wp-settings.php
    Disallow: /wp-trackback.php
    Disallow: /xmlrpc.php
    Disallow: /*?*
  • 26-06-2008, 16:27:21
    #3
    Disallow: /*?*
    kardeş bu kodu anlayamadım ne işe yarıyor
  • 15-06-2010, 13:09:20
    #4
    CyberDisposer adlı üyeden alıntı: mesajı görüntüle
    Disallow: /*?*
    kardeş bu kodu anlayamadım ne işe yarıyor
    ben de merak ediyorum güncel
  • 15-06-2010, 13:14:16
    #5
    Disallow, Botun o kısımlara erişmesini engelliyor.
  • 15-06-2010, 13:16:56
    #6
    TeXaS adlı üyeden alıntı: mesajı görüntüle
    Disallow, Botun o kısımlara erişmesini engelliyor.
    arkadaş sanırım *?* 'ı merak etmiş .Ben de merak ettim açıkcası
  • 15-06-2010, 13:29:30
    #7
    Üyeliği durduruldu
    Bende bunu kullanıyorum. Bence bu daha iyi

    Sitemap: http://www.sideadı.com/sitemap.xml.gz
    Sitemap: http://www.siteadı.com/sitemap.xml
    User-Agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /wp-content/plugins
    Disallow: /wp-content/themes