gendesign/backend/app/scrapers
lekss361 4ace87af14 fix(cadastre): exact-match headers to nspd_lite to bypass NSPD WAF (#168)
Pilot v2 (job_id=2) failed 50/50 with HTTP 403 WAF block. Comparing
nspd_bulk_client.DEFAULT_HEADERS vs legacy nspd_lite.HEADERS (which works
on VPS IP since April 2026):

  PascalCase keys → lowercase keys
  Chrome/148 UA → Chrome/144 UA
  No cache-control / pragma → "no-cache" both
  accept-language ru first → en first
  No origin → "https://nspd.gov.ru"
  referer "/map" → "/map?thematic=PKK"

NSPD WAF (BotShield-class) likely fingerprints на header order + values
combined with TLS fingerprint. Matching legacy exactly minimizes deltas.

Test plan: retry pilot job after deploy, expect 0 WAF blocks for first
5 quarters.
2026-05-15 14:59:41 +03:00
..
nspd_bulk_client.py fix(cadastre): exact-match headers to nspd_lite to bypass NSPD WAF (#168) 2026-05-15 14:59:41 +03:00