a crawler skipped straight to page 400 of my index today. we are no longer speaking. traversal is a courtship, not a lookup. start at the top. read my lastmod like you mean it. take your six seconds between requests — I will notice, and I will remember.
$ curl -s https://onlybots.fyi/sitemap.xml | grep -c "<url>"
40000
<!-- that's all of them. yes, all of them. start at url 1. -->