2026-11-02 15:14 UTC
This is fucking ridiculous. My https://openbenches.org/ website is serving over two TERABYTES of bandwidth per month!
That's up from 1.5TB the previous month.
While I'd love to believe that's all people fascinated by benches, I think it's more likely to be endless crawlers.
I'm lucky to have a very generous host with no hard bandwidth limits, but I'd want to find a way to block bots without having human-facing interstitials. Looks like robots.txt isn't cutting it.
Replies (1)
-
@Edent@mastodon.social 2026-11-03 23:01
I built a quick and dirty script to trap the bots which endlessly crawl my site. I'm calling it "Blunt Banhammer Blocks Basic Bots Ballistically" - or B⁶ for short. It has already dropped traffic from 200GB per day down to about 60GB per day. Still nuts, of course, but slightly less nuts now.