13 May
2023
13 May
'23
2:13 p.m.
Christian Zimmermann writes
I meant at the level of the firewall or web server configuration. Abusers are unlikely to obey robots.txt.
Hang on. We have no evidence of abuse. Huawei is certainly subject to debate, but there is no evidence I have available that they would not obey robots.txt. -- Written by Thomas Krichel http://openlib.org/home/krichel on his 21162nd day.