# All robots will spider the domain User-agent: * Disallow: # Disallow directory /catalog/includes User-agent: * Disallow: /live/includes # Disallow directory /catalog/images/ User-agent: * Disallow: /live/images/ # Disallow directory /catalog/downloads/ User-agent: * Disallow: /live/downloads/