From ad558b04c89cda8ab1e4101f83325d8ebad4de34 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20Lebrun?= Date: Tue, 14 Jan 2025 10:49:51 +0100 Subject: [PATCH] static: robots.txt: change Crawl-Delay from 1s to 2s MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Recently crawlers have risen back up to high levels. Let's ask them nicely to calm down. Signed-off-by: Théo Lebrun --- static/robots.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/static/robots.txt b/static/robots.txt index 1cc6b81..73c5a68 100644 --- a/static/robots.txt +++ b/static/robots.txt @@ -1,3 +1,3 @@ User-Agent: * Allow: / -Crawl-Delay: 1 +Crawl-Delay: 2