I am not technical. I am using WordPress as I have no intentions on trying to learn code. However, I was given a task how to create a robots.txt file at work. I stumbled upon this awesome tool that can self generate this for you which I would love to share. Today we are going to learn how to create a robots. txt file for your website in the matter of seconds. If you go to http://www.mcanerin.com/EN/search-engine/robots-txt.asp they will have everything ready for you to generate this file. This is what this will look like. Here is another good resource right here http://www.toolsiseek.com/robots-txt-generator-validator/ Next all you have to do is put in your website and fill out anything you either want the search engines to not index or index. It is pretty simple. Finally at the end you click on create robots .txt file and then you are all set. You can upload that to your site or you can manually put that into Google Webmaster Tools and Bing Webmaster Tools and then you are fine.