Commit graph

4 commits

Author SHA1 Message Date
lekss361
cd0d80b626 fix: добавить admin_etl.py + rebase на актуальный main
Worker предыдущей задачи (PR #437) ошибочно пропустил admin_etl.py — 2 endpoints
оставались под AdminTokenAuth. Также исправлены 2 оставшихся AdminTokenAuth
параметра в admin_scrape.py (ekburg-permits, kn-catalog-objects) у которых import
уже был удалён — NameError при запросе. Rebase на main HEAD 0e29464.

Total: 6 файлов, 47 endpoints без AdminTokenAuth.
2026-05-23 13:32:12 +03:00
lekss361
2694e3180b feat(etl): fuzzy matcher v2 — pruned threshold 0.85→0.80 for objective mapping coverage
- Add AUTO_ACCEPT_THRESHOLD_V2 = 0.80 constant to objective_backfill.py
- Add min_threshold param to find_match_candidates() (default REVIEW_THRESHOLD=0.6)
- Add match_method param to auto_apply_matches() (default 'fuzzy_trgm')
- Add ?v2=true query param to POST /api/v1/admin/etl/objective-backfill
  - v2 mode: threshold=0.80, method='fuzzy_v2', search from 0.80 (not 0.60)
- Return type widened to dict[str,object] to include threshold_used + match_method_used

DB audit: 1068 unmapped EKB objs; v1 adds ~133 rows, v2 adds ~47 more.
Expected coverage: 8.5% → ~20% after sequential v1+v2 runs on prod.
Run: POST /objective-backfill (v1) then POST /objective-backfill?v2=true (v2).
Part of task #44 Part A, epic #271.
2026-05-17 16:28:20 +03:00
lekss361
db1ff5b1fc feat(nspd): denorm nspd_parcels + nspd_buildings tables (#94 PR3 of 4)
- data/sql/99_nspd_entities_denorm.sql: 2 tables (cad_num PK) + 6 indexes
  (quarter_cad, GIST geom, partial WHERE purpose ILIKE '%многокв%')
- nspd_denorm.py: denorm_parcel_feature / denorm_building_feature / denorm_dump
  - SAVEPOINT per row via with db.begin_nested()
  - ST_Transform(ST_SetSRID(ST_GeomFromGeoJSON(:geom), 3857), 4326)
  - json.dumps(geom, ensure_ascii=False) for psycopg bind
  - All CAST(:x AS type) — no :type
- nspd_sync.py: inline denorm after _upsert_dump in harvest_quarter (non-fatal)
- nspd_denorm_backfill.py: Celery one-shot task (per-quarter commit)
- admin_etl.py: POST /api/v1/admin/etl/nspd-denorm-backfill (AdminTokenAuth)
- 19 tests: coerce helpers, denorm parcel/building, aggregate counts

Foundation for downstream features (МКД neighbors lookup, parcel attrs).

Part of #94
2026-05-16 20:20:03 +03:00
dbf31cc911 feat(etl): objective_complex_mapping backfill via pg_trgm (#203) (#214)
All checks were successful
Deploy / changes (push) Successful in 4s
Deploy / build-backend (push) Successful in 1m38s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m36s
Deploy / deploy (push) Successful in 38s
2026-05-16 12:47:09 +00:00