Bot ile veri çekilmesini engelleme ?

# This is a list of web-site copiers

User-agent: Domain Re-Animator Bot
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: EmailCollector
Disallow: /

User-agent: EmailSiphon
Disallow: /

User-agent: WebBandit
Disallow: /

User-agent: WebZIP
Disallow: /

User-agent: WebReaper
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: Web Downloader
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: Offline Explorer Pro
Disallow: /

User-agent: HTTrack Website Copier
Disallow: /

User-agent: Offline Commander
Disallow: /

User-agent: Leech
Disallow: /

User-agent: WebSnake
Disallow: /

User-agent: BlackWidow
Disallow: /

User-agent: HTTP Weazel
Disallow: /

# Other robots

User-agent: MJ12bot
User-agent: WeSEE
User-agent: GrapeshotCrawler
User-agent: AddThis.com
User-agent: YandexBot
User-agent: Mail.Ru bot
User-agent: ChangeDetection
User-agent: Exabot
User-agent: Genieo Web filter
User-agent: DotBot
User-agent: NaverBot
User-agent: proximic
User-agent: omgilibot
User-agent: magpie-crawler
User-agent: SEOENGBot
User-agent: aiHitBot
User-agent: AhrefsBot
User-agent: EasouSpider
User-agent: Woko
User-agent: sistrix
User-agent: Spinn3r
User-agent: FlipboardProxy
User-agent: BLEXBot
User-agent: sogou spider
User-agent: SeznamBot
User-agent: SearchmetricsBot
User-agent: ShopWiki
User-agent: CareerBot
User-agent: Wotbox
User-agent: BacklinkCrawler
User-agent: Baiduspider
User-agent: Vagabondo
User-agent: coccoc
User-agent: thumbshots-de-Bot
User-agent: Pinterest
User-agent: MetaJobBot
User-agent: seegnifybot
User-agent: NalezenCzBot
User-agent: spbot
User-agent: ichiro
User-agent: A6-Indexer
User-agent: uMBot
User-agent: OpenWebSpider
User-agent: yacybot
User-agent: Daumoa
User-agent: Aboundexbot
User-agent: rogerbot
User-agent: Netseer
User-agent: gocrawl
User-agent: MojeekBot
User-agent: trendictionbot
User-agent: ZumBot
User-agent: ShowyouBot
User-agent: IstellaBot
User-agent: idmarch
User-agent: linkdexbot
User-agent: bixocrawler
User-agent: Browsershots
User-agent: Kraken
User-agent: bitlybot
User-agent: Lipperhey Spider
User-agent: Infohelfer
User-agent: IntegromeDB
User-agent: Crawler4j
User-agent: Jyxobot
User-agent: LoadImpactPageAnalyzer
User-agent: 007AC9
User-agent: meanpathbot
User-agent: AntBot
User-agent: AskQuickly
User-agent: TurnitinBot
User-agent: BDCbot
User-agent: STINGbot
User-agent: psbot
User-agent: NetcraftSurveyAgent
User-agent: URLAppendBot
User-agent: Blekkobot
User-agent: SemrushBot
User-agent: WBSearchBot
User-agent: WebCorp
User-agent: BUbiNG
User-agent: SEOkicks-Robot
User-agent: Motoricerca-Robots.txt-Checker
User-agent: MetaGeneratorCrawler
User-agent: socialbm_bot
User-agent: CCBot
User-agent: HubSpot Connect
User-agent: iCjobs
User-agent: Company News Search engine
User-agent: x28-job-bot
User-agent: EveryoneSocialBot
User-agent: Seobility
User-agent: Ezooms
User-agent: Symfony Spider
User-agent: Iframely
User-agent: NextGenSearchBot
User-agent: KrOWLer
User-agent: netEstate Crawler
User-agent: Twingly Recon
User-agent: Robots_Tester
User-agent: FacebookExternalHit
User-agent: oBot
User-agent: Arachnophilia
User-agent: eCommerceBot
User-agent: Alexabot
User-agent: emefgebot
User-agent: UASlinkChecker
User-agent: Nuhk
User-agent: Panscient web crawler
User-agent: Najdi.si
User-agent: SecurityResearchBot
User-agent: CloudServerMarketSpider
User-agent: YYSpider
User-agent: UnisterBot
User-agent: 200PleaseBot
User-agent: ICC-Crawler
User-agent: Peeplo Screenshot Bot
User-agent: Steeler
User-agent: nekstbot
User-agent: 360Spider
User-agent: LoadTimeBot
User-agent: SpiderLing
User-agent: webinatorbot
User-agent: Cliqzbot
User-agent: Leikibot
User-agent: AboutUsBot
User-agent: TinEye
User-agent: musobot
User-agent: search.KumKie.com
User-agent: Nigma.ru
User-agent: CompSpyBot
User-agent: SeoCheckBot
User-agent: hawkReader
User-agent: PercolateCrawler
User-agent: Butterfly
User-agent: 008
User-agent: Plukkie
User-agent: WebThumbnail
User-agent: Falconsbot
User-agent: SSL-Crawler
User-agent: ThumbSniper
User-agent: Embedly
User-agent: linguatools
User-agent: backlink-check.de
User-agent: adressendeutschland.de
User-agent: XRL
User-agent: IdeelaborPlagiaat
User-agent: SiteCondor
User-agent: Web-Monitoring
User-agent: Vedma
User-agent: parsijoo
User-agent: GarlikCrawler
User-agent: FyberSpider
User-agent: classbot
User-agent: ZeerchBot
User-agent: Feedly
User-agent: WebCookies
User-agent: LinkedInBot
User-agent: TomTom places company search
User-agent: CloudFlare-AlwaysOnline
User-agent: Readability
User-agent: suggybot
User-agent: CatchBot
User-agent: Jabse.com Crawler
User-agent: woriobot
User-agent: ExB Language Crawler
User-agent: kulturarw
User-agent: BrainbruBot
User-agent: KomodiaBot
User-agent: Qualidator.com Bot
User-agent: IXEbot
User-agent: CMS Crawler
User-agent: immediatenet thumbnails
User-agent: Shareaholicbot
User-agent: YioopBot
User-agent: Qualidator.com SiteAnalyzer 1.0
User-agent: Qirina Hurdler
User-agent: BegunAdvertising
User-agent: LuminateBot
User-agent: linkdex.com
User-agent: Curious George
User-agent: Fetch-Guess
User-agent: SBSearch
User-agent: alexa site audit
User-agent: AraBot
User-agent: AMZNKAssocBot
User-agent: Speedy
User-agent: HostTracker
User-agent: CliqzBot
User-agent: findlinks
User-agent: CCResearchBot
User-agent: Semantifire
User-agent: LinkAider
User-agent: Zookabot
User-agent: ScreenerBot Crawler
User-agent: webmastercoffee
User-agent: PaperLiBot
User-agent: QuerySeekerSpider
User-agent: Crowsnest
User-agent: UnwindFetchor
User-agent: MetaURI API
User-agent: MiaDev
User-agent: AcoonBot
User-agent: firmilybot
User-agent: Sosospider
User-agent: OpenindexSpider
User-agent: MetaHeadersBot
User-agent: Strokebot
User-agent: GeliyooBot
User-agent: bot-pge.chlooe.com
User-agent: ownCloud Server Crawler
User-agent: CirrusExplorer
User-agent: ProCogSEOBot
User-agent: Dlvr.it/1.0
User-agent: Open Web Analytics Bot
User-agent: RyzeCrawler
User-agent: discoverybot
User-agent: crawler for netopian
User-agent: ADmantX Platform Semantic Analyzer
User-agent: R6 bot
User-agent: bl.uk_lddc_bot
User-agent: Linguee Bot
User-agent: SolomonoBot
User-agent: Grahambot
User-agent: Automattic Analytics Crawler
User-agent: YoudaoBot
User-agent: PiplBot
User-agent: FlightDeckReportsBot
User-agent: fastbot crawler
User-agent: 4seohuntBot
User-agent: Updownerbot
User-agent: JikeSpider
User-agent: NLNZ_IAHarvester2013
User-agent: wsAnalyzer
User-agent: YodaoBot
User-agent: Esribot
User-agent: Thumbshots.ru
User-agent: BlogPulse
User-agent: bot.wsowner.com
User-agent: wscheck.com
User-agent: Qseero
User-agent: drupact
User-agent: HuaweiSymantecSpider
User-agent: PagePeeker
User-agent: HomeTags
User-agent: facebookplatform
User-agent: Pixray-Seeker
User-agent: BDFetch
User-agent: MeMoNewsBot
User-agent: ProCogBot
User-agent: WillyBot
User-agent: peerindex
User-agent: Job Roboter Spider
User-agent: MLBot
User-agent: WebNL
User-agent: Peepowbot
User-agent: Semager
User-agent: MIA Bot
User-agent: heritrix
User-agent: Eurobot
User-agent: DripfeedBot
User-agent: Whoismindbot
User-agent: Bad-Neighborhood
User-agent: Hailoobot
User-agent: akula
User-agent: MetamojiCrawler
User-agent: Page2RSS
User-agent: EasyBib AutoCite
User-agent: NerdByNature.Bot
User-agent: EventGuruBot
User-agent: quickobot
User-agent: gonzo
User-agent: bnf.fr_bot
User-agent: UptimeRobot
User-agent: Influencebot
User-agent: MSRBOT
User-agent: KeywordDensityRobot
User-agent: Ronzoobot
User-agent: ScoutJet
User-agent: Twikle
User-agent: SWEBot
User-agent: RADaR-Bot
User-agent: DCPbot
User-agent: Castabot
User-agent: imbot
User-agent: EdisterBot
User-agent: WASALive-Bot
User-agent: Accelobot
User-agent: PostPost
User-agent: factbot
User-agent: Setoozbot
User-agent: biwec
User-agent: Search17Bot
User-agent: Lijit
User-agent: JUST-CRAWLER
User-agent: Apercite
User-agent: pmoz.info ODP link checker
User-agent: LemurWebCrawler
User-agent: Covario-IDS
User-agent: Holmes
User-agent: RankurBot
User-agent: envolk
User-agent: Ask Jeeves/Teoma
User-agent: LexxeBot
User-agent: StackRambler
User-agent: Abrave Spider
User-agent: EvriNid
User-agent: arachnode.net
User-agent: CamontSpider
User-agent: wikiwix-bot
User-agent: Nymesis
User-agent: trendictionbot
User-agent: Sitedomain-Bot
User-agent: SEODat
User-agent: SygolBot
User-agent: Snapbot
User-agent: OpenCalaisSemanticProxy
User-agent: ZookaBot
User-agent: CligooRobot
User-agent: cityreview
User-agent: nworm
User-agent: SBIder
User-agent: Dot TK - spider
User-agent: EuripBot
User-agent: ParchBot
User-agent: Peew
User-agent: YRSpider
User-agent: Urlfilebot (Urlbot)
User-agent: Gaisbot
User-agent: WatchMouse
User-agent: Tagoobot
User-agent: WebWatch/Robot_txtChecker
User-agent: urlfan-bot
User-agent: StatoolsBot
User-agent: page_verifier
User-agent: SSLBot
User-agent: SAI Crawler
User-agent: DomainDB
User-agent: LinkWalker
User-agent: WMCAI_robot
User-agent: voyager
User-agent: copyright sheriff
User-agent: Ocelli
User-agent: Twiceler
User-agent: amibot
User-agent: abby
User-agent: NetResearchServer
User-agent: VideoSurf_bot
User-agent: XML Sitemaps Generator
User-agent: BlinkaCrawler
User-agent: nodestackbot
User-agent: Pompos
User-agent: taptubot
User-agent: BabalooSpider
User-agent: Yaanb
User-agent: Girafabot
User-agent: livedoor ScreenShot
User-agent: eCairn-Grabber
User-agent: FauBot
User-agent: Toread-Crawler
User-agent: Setoozbot
User-agent: MetaURI
User-agent: L.webis
User-agent: Web-sniffer
User-agent: FairShare
User-agent: Ruky-Roboter
User-agent: ThumbShots-Bot
User-agent: BotOnParade
User-agent: Amagit.COM
User-agent: HatenaScreenshot
User-agent: HolmesBot
User-agent: dotSemantic
User-agent: Karneval-Bot
User-agent: HostTracker.com
User-agent: AportWorm
User-agent: XmarksFetch
User-agent: FeedFinder/bloggz.se
User-agent: CorpusCrawler
User-agent: Willow Internet Crawler
User-agent: OrgbyBot
User-agent: GingerCrawler
User-agent: pingdom.com_bot
User-agent: baypup
User-agent: Mp3Bot
User-agent: 192.comAgent
User-agent: Surphace Scout
User-agent: WikioFeedBot
User-agent: Szukacz
User-agent: DBLBot
User-agent: Thumbnail.CZ robot
User-agent: LinguaBot
User-agent: GurujiBot
User-agent: Charlotte
User-agent: 50.nu
User-agent: SanszBot
User-agent: moba-crawler
User-agent: HeartRails_Capture
User-agent: SurveyBot
User-agent: MnoGoSearch
User-agent: smart.apnoti.com Robot
User-agent: Topicbot
User-agent: JadynAveBot
User-agent: OsObot
User-agent: WebImages
User-agent: WinWebBot
User-agent: Scooter
User-agent: Scarlett
User-agent: GOFORITBOT
User-agent: DKIMRepBot
User-agent: Yanga
User-agent: DNS-Digger-Explorer
User-agent: YowedoBot
User-agent: botmobi
User-agent: Fooooo_Web_Video_Crawl
User-agent: UptimeDog
User-agent: ^Nail
User-agent: Metaspinner/0.01
User-agent: Touche
User-agent: RSSMicro.com RSS/Atom Feed Robot
User-agent: SniffRSS
User-agent: Kalooga
User-agent: FeedCatBot
User-agent: WebRankSpider
User-agent: Flatland Industries Web Spider
User-agent: DealGates Bot
User-agent: Link Valet Online
User-agent: Shelob
User-agent: Technoratibot
User-agent: Flocke bot
User-agent: FollowSite Bot
User-agent: Visbot
User-agent: LivelapBot
User-agent: semantic-visions.com crawler
User-agent: ScooperBot
User-agent: BuzzSumo
User-agent: Veooz
User-agent: MaxPointCrawler
User-agent: GrapeshotCrawler
User-agent: FreeWebMonitoring SiteChecker
User-agent: ToutiaoSpider
User-agent: trendictionbot
User-agent: NetSeer crawler
User-agent: TweetmemeBot
User-agent: SemrushBot
User-agent: Sogou web spider
User-agent: NewShareCounts.com
Disallow: /