#robots.txt #09/05/05 #this file dictates which folders and files crawlers (like Google) may and may-not index #it must be named "robots.txt" and placed in the webRoot. ("/robots.txt") User-agent: * Disallow: /bin/ Disallow: /include/ Disallow: /process-callback.asp Disallow: /err404.asp Disallow: /err500.asp Disallow: /message.asp