# Second draft: Sept2008 Sitemap: http://www.savio.cc/sitemap.xml # Disallow all access User-agent: * Disallow: /archive/ Disallow: /cgi-bin/ Disallow: /contractor/ Disallow: /dealer-locator/ Disallow: /images/ Disallow: /notices/ Disallow: / # Then specifically allow Google's spider's User-Agent: Googlebot Disallow: /archive/ Disallow: /cgi-bin/ Disallow: /contractor/ Disallow: /dealer-locator/ Disallow: /images/ Disallow: /notices/ Allow: / User-agent: Googlebot-Image Disallow: /archive/ Disallow: /cgi-bin/ Disallow: /contractor/ Disallow: /dealer-locator/ Disallow: /images/ Disallow: /notices/ Allow: / # Example of allowing access to a subdirectory. # User-agent: Googlebot # Disallow: /folder1/ # Allow: /folder1/myfile.html # MSN amd Yahoo are allowed too... User-agent: MSNBot Disallow: /archive/ Disallow: /cgi-bin/ Disallow: /contractor/ Disallow: /dealer-locator/ Disallow: /images/ Disallow: /notices/ Allow: / User-agent: Slurp Disallow: /archive/ Disallow: /cgi-bin/ Disallow: /contractor/ Disallow: /dealer-locator/ Disallow: /images/ Disallow: /notices/ Allow: /