# Robots.txt # All robots will spider this domain User-agent: * Disallow: /disallowthisdirectory/ Disallow: /company/contact.htm Disallow: /company/formfailure.html Disallow: /company/formsuccess.html Disallow: /company/formmail.php