2026-11-01
When per-IP rate limits did not stop a search bot spread across tens of thousands of IPs
A site whose search calls an external API on every query saw search traffic rise about 20 times in a few days, and nearly all searches returned 503. The first assumption, that a per-IP limit would contain it, was wrong. One day's roughly 60,000 searches came from roughly 60,000 different IPs, so each IP sent about one request and never reached any limit. The user agent was an ordinary browser, the Referer was the site itself, and the queries were real product names, so price collection was likely. The external API's daily quota ran out early in the day and normal visitors could not search for most of it. The fixes: only call the external API when the request carries a signed token issued by the site's own page, leave untokened searches out of the popular-searches ranking, and set hourly caps globally and per token. 503s went to zero after release. A token is not authentication; it only raises the attacker's cost.