User-agent: * # Directories Disallow: /404/ Disallow: /app/ Disallow: /cgi-bin/ Disallow: /downloader/ Disallow: /includes/ Disallow: /lib/ Disallow: /magento/ Disallow: /pkginfo/ Disallow: /report/ Disallow: /stats/ Disallow: /var/ Disallow: /wordpress/ ## Do not crawl search pages Disallow: /search/ Disallow: /search/* Disallow: /catalogsearch/advanced/* SITEMAP: https://www.ukbathroomstore.co.uk/sitemap.xml