portugalfuturista/elixir
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

static: robots.txt: change Crawl-Delay from 1s to 2s

Browse source 
Recently crawlers have risen back up to high levels.
Let's ask them nicely to calm down.

Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
This commit is contained in:
Théo Lebrun 2025-01-14 10:49:51 +01:00
parent ede73902cd
commit ad558b04c8
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
static/robots.txt
View file

@ -1,3 +1,3 @@
User-Agent: * User-Agent: *
Allow: / Allow: /
Crawl-Delay: 1 Crawl-Delay: 2