# All robots will spider the domain User-agent: * Allow: / Sitemap: http://www.therealperu.co.uk/sitemap.gz # Disallow directory /bin/ User-agent: * Disallow: /bin Disallow: /aspnet_client Disallow: /_notes Disallow: /backups Disallow: /cgi-bin Disallow: /descs Disallow: /email Disallow: /includes Disallow: /usercontrols