TL;DR: we spend more CPU cycles rendering commits for scrapers than we spend on all other kinds of legitimate access, including git clones. At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing but rendering git commits as html.

AI crawlers are idiot: their are consuming the HTML diff of the commits insteadof cloning the repository locally...