13 May
2023
13 May
'23
2:04 p.m.
I meant at the level of the firewall or web server configuration. Abusers are unlikely to obey robots.txt. Christian Zimmermann On Sat, May 13, 2023 at 4:20 AM Thomas Krichel <krichel@openlib.org> wrote:
Christian Zimmermann writes
Just deny that IP and be done with it.
I will try
https://collec.repec.org/robots.txt
instead. Service is up again.
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 21162nd day.