feat(nspd): harvest_quarter Celery task + beat + admin endpoint (#94 pt.4/4 part A)
Sprint 1.1 item #3 из #94 part 2. Замыкает harvest cycle: Celery task
оборачивает NSPDClient.search_by_quarter (PR #109) + UPSERT в
nspd_quarter_dumps (PR #110). После — analyze_parcel будет читать dumps
вместо on-demand NSPD HTTP в request-цикле (Sprint 1.1 #4, следующий PR).
Backend:
- New `backend/app/workers/tasks/nspd_sync.py`:
- `harvest_quarter(quarter_cad, region_code=66, include_zouit=True,
include_risks=False)` — single-quarter harvest. autoretry_for=
(NspdLiteWafError,) с retry_backoff=True, max_retries=3,
soft_time_limit=120s. Generic exception → upsert error row с
harvest_error=str(exc), counts=0 (НЕ re-raise).
- `harvest_stale_quarters(region_code, max_age_days=90, batch_size=50)` —
beat fanout: SELECT cad_quarters_geom EXCEPT fresh dumps → apply_async per cad.
- UPSERT использует ST_Multi(ST_Transform(ST_SetSRID(ST_GeomFromGeoJSON,3857),4326))
для quarter_geom (schema MultiPolygon strict per database-expert contract).
- `celery_app.py`:
- Add `nspd_sync` в `include`
- Beat entry `nspd-harvest-stale-quarters` Mon 04:00 МСК, batch_size=50,
max_age_days=90
- `worker_ready` sanity check `SELECT 1 FROM nspd_quarter_dumps LIMIT 0`
(non-fatal critical log per Bug_Worker_Ready_EarlyReturn_Fixed pattern)
- `admin_scrape.py`:
- `HarvestQuarterRequest` Pydantic с pattern `^\d+:\d+:\d+$` (quarter only)
- `POST /api/v1/admin/scrape/nspd/harvest-quarter` manual trigger,
requires X-Admin-Token, returns task_id.
Tests (`test_nspd_sync.py`): 14 mock-based (no real NSPD/DB):
- happy path: search_by_quarter mock → verify SQL params/structure
- empty quarter → row with quarter_geom=NULL, total=0, no harvest_error
- WAF error → autoretry kicks in (raise re-propagated)
- Generic error → upsert error row
- Fanout: 50 stale → 50 apply_async calls
- Layer names preservation (all 5 zouit + 11 risks if enabled)
- Geometry SRID handling
Code review (code-reviewer pre-push): APPROVE, 0 blocking, 5 minor deferred:
- inline datetime import in error branch (style)
- layers_fetched array literal via string concat (current layer names safe)
- harvest_stale_quarters error rows always re-tried (intended, undocumented)
- test call_args.kwargs fragile (works for current callsite)
- beat collision with refresh-ekb-districts-medians every ~2 years (negligible)
Part of #94. Next: Sprint 1.1 #4 — analyze_parcel reads dump cache.
9b2a289e83