lekss361
|
6fce81e10c
|
feat(db): add nspd_quarter_dumps table for PKK harvest persistence (#94 pt.3/4) (#110)
Sprint 1.1 item #2 из плана #94 part 2. Persistent storage для QuarterDump
snapshots (PR #109). Foundation для Celery harvest_quarter task (next PR)
и analyze_parcel integration (после).
Schema (data/sql/88_nspd_quarter_dumps.sql):
- Table `nspd_quarter_dumps`: 17 columns matching QuarterDump dataclass.
- PK quarter_cad (natural key, one row per quarter)
- quarter_geom (MultiPolygon, 4326) + bbox_3857 (Polygon, 3857) — dual-index
стратегия: bbox для scheduler queries без transform, geom для joins с
PZZ/district в 4326
- 7 per-layer count columns + total_features (computed denorm для быстрых
статистических queries без распаковки JSON)
- features_json JSONB — array of ~117 NSPDFeature objects (~120KB per row,
TOAST handles transparently). Geometry в raw EPSG:3857 — caller transforms.
- layers_fetched TEXT[] mirrors QuarterDump.layers_fetched tuple
- harvest_duration_ms (nullable — NULL = "not measured"), harvest_error TEXT
- region_code SMALLINT NOT NULL (no DEFAULT — explicit per insert)
- 6 indexes: PK, GIST(bbox_3857), GIST(quarter_geom), B-tree(fetched_at_utc DESC),
B-tree(region_code), partial B-tree(fetched_at_utc WHERE harvest_error IS NULL)
- View v_quarter_dumps_freshness — admin monitoring (AGE + is_failed flag)
Конвенции:
- BEGIN/COMMIT atomic
- IF NOT EXISTS на каждом CREATE — idempotent
- COMMENT ON TABLE + 7 COMMENT ON COLUMN + comments на каждом index
- psycopg v3 compatible (DDL only, no %s)
Применено к prod в рамках database-expert verification (DDL only, idempotent).
Vault entry: code/schemas/Schema_Nspd_Quarter_Dumps.md (upsert template +
EXPLAIN plans + cross-refs).
Code review (code-reviewer pre-push): APPROVE, 2 minor non-blocking:
- MINOR-1 quarter_geom MultiPolygon vs Polygon — resolved в Celery PR via
ST_Multi(ST_Transform(...)) на ingest contract
- MINOR-2 layers_fetched default '{}' — documented в comment как
partial-failure path для error rows
Next: Celery harvest_quarter task + beat schedule.
Part of #94 Sprint 1.1.
Co-authored-by: lekss361 <claudestars@proton.me>
|
2026-05-12 18:22:50 +03:00 |
|