Crawl-Delay does have a big impact on the loadavg of the server, meaning: - (1) most requests are from crawlers and, - (2) most crawlers listen to Crawl-Delay. The prod server can handle the current loadavg just fine, let's let them up their game. Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com> |
||
|---|---|---|
| .. | ||
| fonts | ||
| img | ||
| autocomplete.js | ||
| dynamic-references.js | ||
| messages.json | ||
| robots.txt | ||
| script.js | ||
| style.css | ||