# # WWW Spider/Robot control file - leave as /robots.txt # This file was last altered on 05.06.02 by jw # # All spiders should avoid: # Script files # Test Area # Stats information # Custom error messages # Site administration # Stats logs # E-web includes User-agent: * Disallow: /cgi-bin/ Disallow: /test/ Disallow: /usage/ Disallow: /usage-old/ Disallow: /error/ Disallow: /admin/ Disallow: /logs/ Disallow: /eweb/