# Robots.txt file created for domain: my247.mobi, new.my247.com.au, my247.asia etc. # All robots will spider the domain User-agent: * Disallow: /ajax/ Disallow: /App_Code/ Disallow: /UserControls/ Disallow: /scripts/ Disallow: /styles/ # Except the Fast crawler(too resource hungry) User-agent: fast Disallow: /