User-agent: * # directed to all robots Disallow: /puretecgen_data/ Disallow: /logs/ index: follow