User-agent: * Disallow: / User-agent: Slurp Disallow: / # Allow State net's robot in User-agent: get-nh.track Allow: / User-agent: get-nh.text Allow: / # Allow DataGuidance robot in User-agent: Exbos Allow: / # Alow Google to crawl User-agent: Googlebot Allow: /