User-Agent: * Request-rate: 1/120 # maximum rate is one page every 120 seconds Crawl-delay: 120