gendesign/backend/tests/scrapers
lekss361 b7889d113b
feat(cadastre): NSPDBulkClient + schemas for bulk ingest (#168 PR2/5) (#170)
Add async NSPD client (nspd_bulk_client.py) with 3 new methods:
- search_by_quarter: REST /api/geoportal/v2/search/geoportal — snapshot 20 per cat + meta totals
- wms_feature_info: WMS GetFeatureInfo для grid-walk discovery
- list_objects_in_building: tab-group-data (Q3 deferred — код ready)

Rate limit: asyncio.Semaphore(3) + 0.05s jitter.
429 → exponential backoff 2→4→8s, max 3 retries.
403 (WAF) → raise immediately, no retry.

Pydantic schemas (nspd_bulk.py): NSPDBulkFeature, QuarterSnapshot, ObjectsListing
with extra='allow' to preserve unknown fields → raw_props JSONB downstream.

15 unit tests + 4 e2e (real NSPD, skipped in CI by default).

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 12:50:41 +03:00
..
__init__.py feat(cadastre): NSPDBulkClient + schemas for bulk ingest (#168 PR2/5) (#170) 2026-05-15 12:50:41 +03:00
test_nspd_bulk_client.py feat(cadastre): NSPDBulkClient + schemas for bulk ingest (#168 PR2/5) (#170) 2026-05-15 12:50:41 +03:00