Incidents
What is X-Forwarded-For (XFF) spoofing — the trusted-proxy config trap
X-Forwarded-For (XFF) is a header clients can forge — scanners hide SQLi/XSS probes in it, and a 'trust all proxies' setting lets the fake value reach your app. A defensive case study: why defense-in-depth held, and the real trusted-proxy fix.
X-Forwarded-For (XFF) is the header that tells your app "the real visitor IP is this" when traffic passes through a proxy. The catch: the client can forge that value freely. Here's a common indie-dev case — "we got hit by an XFF spoofing scan" — turned into a lesson, anonymized, with no reproducible attack steps.
- Type
- X-Forwarded-For spoofing + injection probing (automated scan)
- Impact
- None (stopped by defense-in-depth)
- How it surfaced
- Attack requests 500'd; a flood of exception emails
- Shields that held
- ① placeholders (bound values) ② DB charset validation
- The gap left
- "trust all proxies" — with no proxy actually in front
- Response
- Sanitize IP headers at the boundary (patch) → fix trusted-proxy config (root)
First, split it: "attack" or "my own mistake"?
The rule is: don't jump to "attack!" on seeing errors. If you just changed config, weigh "maybe my change did this" with equal seriousness.
Suspect your own change
Let the value be the evidence
Targeted or a scan?
ITD's view: when reasoning backward from symptoms, kill hypotheses with data
"The thing I just touched looks suspicious" is a fine starting point. But if the evidence (here, the value itself) refutes it, let it go. Form several hypotheses and knock them down one by one with data, not hunches — misdiagnosis drops sharply.
What the attack was: injection probes riding on a spoofed XFF
The bot packs a probe string into XFF "instead of an IP," with only a normal-looking IP at the end. The goal is one thing —
"Does this app trust the XFF value and mix it raw into a DB query, or print it straight into HTML?"
If the app is sloppy, SQL injection or XSS lands via XFF. It abuses "the place that carries an IP" as an injection point for arbitrary strings.
Why impact was zero: two shields
Shield ① bound values (placeholders)
The IP value was passed via a placeholder as data, not string-concatenated. Input can't cross into "command," so SQLi can't form by design — one of the biggest payoffs of using a framework.
Shield ② DB charset validation
The malformed bytes (overlong UTF-8 etc.) are invalid sequences the column's charset can't store. The DB rejected the write and threw — the attack failed and was detected.
All that happened: the attacker's own requests 500'd and a flood of exception emails went out. No leak, no tampering, no unauthorized login. A textbook case of defense-in-depth working as intended.
The real problem: trusting all proxies
If we stopped here it's a feel-good story — but one thing can't be ignored. Why did the spoofed XFF reach IP handling at all?
✗ Trust all proxies (wildcard)
Client's fake XFF → adopted as the "real IP" → logs, sessions, IP checks poisoned
✓ Trust nothing (default)
Fake XFF ignored → real socket IP is used → spoofing has no effect
This setup had no reverse proxy or CDN — the web server was directly exposed. So XFF never needed trusting, yet it was set to "trust all." "Just allow everything" is the classic dev-time shortcut that survives into production.
Blind spot: XFF spoofing isn't only about injection
Anything that trusts the client IP gets poisoned: rate-limit bypass, IP allowlist bypass, ban evasion, forged audit logs. Even with rock-solid injection defenses, this is a separate problem. Inventory "where do I trust the IP?" and check the basis of that trust (= your proxy config).
Fix: sanitize at the boundary → set trust correctly
Sanitize IP headers at the boundary (patch, instant)
Set trusted proxies correctly (root)
http/https detection, so changes can cause redirect loops — verify on one environment first.Don't confuse 'valid IP' with 'real IP'
Verify: after fixing, replay the attack to confirm
Turn "I think I fixed it" into "it definitely stopped." ① apply to one environment only → ② send the same malformed header yourself → ③ confirm error counts don't rise → ④ confirm normal access still works → ⑤ roll out everywhere. Staged rollout prevents accidents.
Follow-up: block the IP header, get hit on User-Agent (the whack-a-mole trap)
Tens of minutes after adding sanitization, the same kind of error returned — on a different column this time: user_agent, not ip_address. Blocked on the IP header, the attacker moved the same payload to the User-Agent header. That forces a reframe.
The right question isn't "which header is dangerous?" but "which DB column does untrusted input ultimately land in?" If the only sinks are ip_address and user_agent, sanitizing the inputs that feed them (IP headers + User-Agent, plus Referer to be safe) stops session-derived errors by design — other, non-stored headers can't be pivoted. Legitimate User-Agents are ASCII/valid UTF-8, so stripping only invalid bytes has no impact.
ITD's view: reason backward from the sink (don't seal inputs one by one)
Sealing headers one at a time is whack-a-mole — the inputs an attacker can use are endless. Reason backward from the sink (the stored column / output) and enumerate every input that reaches it, all at once. As "block the IP, get hit on User-Agent" taught us: when you patch one header, seal the other inputs that reach the same sink at the same time. And again — we replayed the second attack and confirmed it stopped before rolling out.
Then again: the entry filter was bypassed — defend at the exit
Just after sealing User-Agent too, the same ip_address column error came back. The entry sanitization was deployed, and replaying the same malformed header in a self-test didn't reproduce it — yet production kept recurring. This is the crux.
The reason: the framework's "resolve the client IP" logic is complex internally — multiple header notations (X-Forwarded-For, Forwarded, custom headers), combinations with proxy-trust settings, parse order and caching. Somewhere in those internal paths, a tainted value resurfaced as the "client IP" via a route we hadn't sanitized. The entry point can't catch everything.
✗ Defend at the entry
Sanitize request headers → client IP resurfaces via another internal path → bypassed
✓ Defend at the exit
Check right before the DB write → the one chokepoint everything passes → no bypass
So we flipped it and defended at the exit (just before the DB write). Subclass the session-storage class and override only the value-producing parts (concept): the stored IP becomes null unless it's a valid IP; the stored User-Agent has invalid bytes stripped and length capped. However the client-IP resolution behaves internally, the check always runs right before the write, so nothing slips through. Hammering both attack routes dozens of times each, error counts didn't rise and legitimate sessions kept working.
ITD's view: the last line of defense is the exit (point of use)
The security maxim is "don't trust input; always validate/escape at the point of use (the exit)." Entry checks help you reject early, but information the framework resolves/derives internally (like the client IP) can slip past the entry and resurface. Validating/escaping right before a dangerous sink — DB, HTML, a command — is the chokepoint everything passes, and that's the last line of defense. Do entry and exit as two separate things.
The biggest lesson: being loudly rejected is safer
Here the attack failed and was made visible as a flood of error notifications. Ironically, that's not bad: being loudly rejected beats being quietly let through. Don't dismiss error notifications as "noise" — use them as an anomaly sensor. Don't stop at "we blocked the impact"; fix "why it reached the internals." That's ITD's stance.
Read next
- Glossary: What is SQL injection / What is XSS / What is SSRF
- Basics: Security basics: .env and API keys
FAQ
QCan I trust X-Forwarded-For?
Not as-is. XFF is a header the client can set freely — an attacker can forge a real-looking IP or an injection-probe string. The only trustworthy XFF is one added by a proxy YOU placed in front and explicitly trust.
QWhat if I don't use a reverse proxy or CDN?
Trust no XFF at all (leave the framework default). If there is no proxy in front but you 'trust all proxies', anyone can feed your app a fake IP. No proxy in the path → trust nothing is the correct setting.
QIs sanitizing the IP header enough?
It stops the injection probes and the error flood, but that's hygiene (stopping the bleeding). A 'valid-looking IP' is not a 'real IP'. Trust in the real client IP must come from the trusted-proxy configuration, not from sanitizing.
QI fixed the IP header but the same error came back. Why?
The same payload was moved to another entry point (e.g. the User-Agent header) and reached the same stored column. Patching headers one by one is whack-a-mole. Reason backward from 'which DB column / output does untrusted input ultimately reach?' and sanitize all the inputs that feed it (IP headers + User-Agent, etc.) together.