gendesign/backend/app/services
lekss361 be143880d0 feat(scrapers): NSPD client foundation (#94) — search_by_cad + WMS + layers
Foundation для G1 #28 ПЗЗ, G2 #29 ВРИ, G3 #30 ЗОУИТ, E1 #51 cad_parcels,
I3 #44 engineering, и поддержки on-demand #93.

Backend (`app/services/scrapers/nspd_client.py`):
- `NSPDClient` с 4 core методами:
  - `search_by_cad(cad, thematic_id)` — REST /api/geoportal/v2/search
    возвращает GeoJSON + ВРИ + land_category + cost_value за 1 запрос
  - `get_feature_info(layer_id, lon, lat, buffer_m=100)` — WMS GetFeatureInfo
    на точке: какие feature'ы layer'а покрывают (lon,lat)
  - `get_features_in_bbox(layer_id, bbox_3857)` — bulk через WMS workaround
    (WFS GetCapabilities → 404, поэтому большой bbox + центральная I/J точка)
  - `list_layers(theme_id)` — каталог слоёв в теме (PKK=1, ARN=665)
- Typed responses: NSPDFeature, NSPDSearchResult, NSPDLayer (frozen dataclasses)
- LAYERS catalog: 32 layer-id с семантическими именами (territorial_zones=875838,
  zouit_engineering=37578, risk_flooding=872205, etc) — TIER 1-6 per #94
- Coordinate helpers: lonlat_to_3857(), bbox_around_point_m()
- Reuse: HEADERS + SSL ctx + fetch_geoportal из existing nspd_lite (WAF-compatible
  urllib trick). Rate limit через rate_ms.
- WAF/Rate-limit: raises NspdLiteWafError на 403/429 — caller backoff.

Tests (`backend/tests/test_nspd_client.py`): 14 unit tests, no network.
- LAYERS catalog sanity (territorial_zones=875838 закрывает G1)
- Coordinate transforms (zero, ЕКБ center, bbox)
- NSPDFeature.from_raw parsing (full / missing fields)
- NSPDSearchResult helpers (empty/first)
- _walk_layer_tree (flat / nested / empty)
- search_by_cad через monkeypatch fetch_geoportal

Также включён micro-fix из PR #95 review (cosmetic):
- parcels.py inline comment "до 25s" → "до 15s" (matched _INLINE_FETCH_WAIT_S)

Scope NOT в этом PR (отдельные follow-up issues, sequential rule):
- Schema migrations (cad_parcels_geom +columns, nspd_territorial_zones,
  nspd_zouit, nspd_red_lines, nspd_risk_zones tables) → отдельный PR
- Celery tasks (sync_territorial_zones_bbox, sync_zouit_*, sync_risk_*) →
  отдельный PR использует client
- Admin endpoints (trigger / status) → отдельный PR
- Замена on-demand fetch на nspd_client → PR after schema ready

Closes part of #94 (foundation only — sub-issues for schema/tasks).
2026-05-12 09:34:42 +03:00
..
exporters init 2026-04-25 13:45:19 +03:00
generative deploy.yml: sync compose+Caddyfile from git, reload caddy after up 2026-04-26 13:31:26 +03:00
photos fix ui lag 2026-04-27 20:26:55 +03:00
scrapers feat(scrapers): NSPD client foundation (#94) — search_by_cad + WMS + layers 2026-05-12 09:34:42 +03:00
site_finder feat(site-finder): auto-fetch cadastre geometry on-demand (#93) (#95) 2026-05-12 09:02:17 +03:00
__init__.py init 2026-04-25 13:45:19 +03:00
analytics_queries.py feat(analytics): recommend_mix v3.1-v3.4 - noise + 2D competitors + 24m cap + success-driven 2026-05-11 22:19:41 +03:00
analytics_refresh.py fix(scrape): SQLAlchemy text() парсил ':00:' regex как named-param 2026-04-30 23:16:11 +03:00
job_settings.py feat(jobs): centralized job_settings API + DB-driven beat schedule 2026-05-11 15:38:13 +03:00
objective_etl.py fix(worker): port objective_etl from psycopg2 → psycopg v3 2026-05-11 12:07:57 +03:00
objective_sync_config.py feat(objective): full sync pipeline + dynamic admin config 2026-05-10 19:54:15 +03:00