Eski sayfalarım ne zaman silinir

şöyle birşey buldum bunlardan hangisini robots dosyasına eklememiz gerekiyor

Block or remove pages using a robots.txt file



You can use a robots.txt file to block Googlebot from crawling pages on your site.

For example, to block Googlebot from crawling all pages under a particular directory (for example, lemurs), you'd use the following robots.txt entry:

User-agent: Googlebot
Disallow: /lemurs
To block Googlebot from crawling all files of a specific file type (for example, .gif), you'd use the following robots.txt entry:

User-agent: Googlebot
Disallow: /*.gif$
To block Googlebot from crawling dynamically generated pages, you'd use this robots.txt entry:

User-agent: Googlebot
Disallow: /*?

Pages you block in this way may still be added to the Google index if other sites link to them. As a result, the URL of the page and, potentially, other publicly available information can appear in Google search results. However, no content from your pages will be crawled, indexed, or displayed.

To entirely prevent a page from being added to the Google index even if other sites link to it, use a noindex meta tag, and ensure that the page does not appear in robots.txt. When Googlebot crawls the page, it will recognize the noindex meta tag and drop the URL from the index.