↳ In-reply-to » @lyse Found it and fixed it! 🎉 The crawler's discovery spider was fetching every feed a second time, without any conditional headers (plus a couple of other politeness bugs: redirected feed URLs never stored their cache validators, and there was no floor between re-fetches). Now every feed is fetched at most once per crawl, always with If-Modified-Since / If-None-Match, and never more than once per 15m no matter what. Just deployed — please keep an eye on your access logs and let me know if you still see anything impolite from the crawler 🙏

Great news, thank you! I can confirm, it looks good so far. After your change, the crawler requests every hour and five minutes: :-)

2026-07-04T14:03:09+02:00 200
2026-07-04T15:08:07+02:00 304
2026-07-04T16:13:08+02:00 304
2026-07-04T17:18:07+02:00 304
2026-07-04T18:23:07+02:00 304
2026-07-04T19:28:07+02:00 304