Online robots.txt File Generation Tool

This online robots.txt generator provides various common setting options for webmasters to choose from, and can generate a robots file that complies with the robots protocol according to specific needs, which is a very practical SEO tool.


Default - All robots are:  
Crawl-delay:
Sitemap: (Leave blank for none)
   
General search robots: Google googlebot
  Baidu baiduspider
  MSN Search msnbot
  Yahoo yahoo-slurp
  Ask/Teoma teoma
  Cuil twiceler
  GigaBlast gigabot
  Scrub The Web scrubby
  DMOZ Checker robozilla
  Nutch nutch
  Alexa/Wayback ia_archiver
  Naver naverbot, yeti
   
Special search robots: Google Image googlebot-image
  Google Mobile googlebot-mobile
  Yahoo MM yahoo-mmcrawler
  MSN PicSearch psbot
  SingingFish asterias
  Yahoo Blogs yahoo-blogs/v3.9
   
Restrict directories: Paths are relative, but each path must include: " / "
 
 
 
 
 
   
   

Now, you create a blank text file, named: "robots.txt", then copy and paste the above content into the "robots.txt". Place the "robots.txt" in the root directory of your website, and make sure it can be accessed by visitors (such as search engines).