DOM.РФ накатил WAF между 2026-05-17 и 2026-05-24 на /сервисы/api/object/*/*.
Без session cookies (___dmpkit___, domain_sid) — все запросы → 403 + WAF
challenge HTML. Cookies выдаются после visit на /сервисы/каталог-новостроек/.
Дополнительно DOM.РФ переименовал 6 endpoints (photos, 5 × doc_*) — старые
URLs возвращают тот же WAF catch-all.
Run 26 (2026-05-24): 15,340 fail requests (1534 obj × 10 endpoints).
Extras snapshots не обновлялись с 2026-05-17.
- stealth.py: BrowserSession.warm_up() — visit catalog listing, кэшит WAF cookies
- domrf_kn.py: warm_up() перед Phase A/B/C (покрывает все extras запросы)
- domrf_catalog_object.py: warm_up() перед SSR HTML fetch batch
PR #322 used placeholder URLs:
- /api/object/{id}/documents -> 404 (Playwright showed: 5 real endpoints:
document/rpd / developer/report / project/documentation /
documentation/other / document/permits). Replaced with 5 parallel
fetches + merge into single upsert_documents call.
- /api/object/{id}/checks -> 404. 6 checkboxes likely inline in bulk
kn/object payload; endpoint /checks does not exist. Commented out
_fetch_obj_checks_safe with TODO for separate investigation PR.
- BrowserSession Semaphore(3) -> Semaphore(6) — double concurrency
for intra-object gather.
GlitchTip: kn_scrape_failures run #19 showed 40 failures each for
both placeholder endpoints across first 30 objects of 1532.
