data: backfill objective_complex_mapping (114→~2000 EKB ЖК) для повышения mv_layout_velocity coverage #203

New issue

Closed

opened 2026-05-16 10:21:40 +00:00 by lekss361 · 1 comment

lekss361 commented

2026-05-16 10:21:40 +00:00

Owner

Source: Coverage audit 2026-05-16. mv_layout_velocity (#113 PR B) показывает coverage 17-33% на test cad'ах — это design limitation.

Числа

DOM.РФ EKB объектов:                    3 855
Объектов с objective_complex_mapping:    114  ← только 3%
mv_layout_velocity rows:                459 (114 obj × ~4 buckets)

mv_layout_velocity берёт velocity только из Objective через objective_complex_mapping. Если ЖК нет в mapping → нет velocity → low coverage % в /best-layouts response.

Что делать (ETL backfill)

Phase 1: Auto-match по name+dev (fuzzy)

backend/app/services/etl/objective_backfill.py (NEW):

LEFT JOIN domrf_kn_objects ↔ objective_corpus_room_month
Fuzzy match по comm_name (Levenshtein / pg_trgm similarity)
Match по dev_inn если есть
Score >= threshold (e.g. 0.85) → auto-insert в objective_complex_mapping
Result: target ~50-70% coverage (1500-2700 ЖК mapped) automatically

Phase 2: Manual review queue (опц)

Score 0.6-0.85 → admin review queue (admin UI):

Suggested matches per DOM.РФ obj
Approve/reject/edit
Goal: 80%+ coverage

Phase 3: Celery beat refresh

После backfill:

mv_layout_velocity REFRESH CONCURRENTLY (helper layout_velocity_refresh.py уже готов)
Schedule daily refresh при changes в objective_corpus_room_month

Acceptance

objective_complex_mapping row count: 114 → ≥1500 (вместо 3%, целевое 40%+)
mv_layout_velocity rows: 459 → ≥6000 (4-bucket × 1500 obj)
/best-layouts coverage % на test cad >50% (вместо 17%)
Backfill idempotent (re-run не дублирует)
Manual review queue UI (Phase 2 — опц)

Effort

Phase 1 auto-match: ~1.5-2 дня (ETL + fuzzy matching + tests)
Phase 2 review queue: ~1 день (опц)
Phase 3 celery beat: 0.5 дня

Owner

backend-engineer (ETL + matching algorithm) + database-expert (если нужны index'ы на domrf_kn_objects(comm_name) для pg_trgm)

Links

Triggered by: best-layouts low coverage audit 2026-05-16
Built on: PR #194 (mv_layout_velocity), objective_corpus_room_month (existing)
Related: #113 (Layout Analysis Phase 2.1)

**Source:** Coverage audit 2026-05-16. `mv_layout_velocity` (#113 PR B) показывает coverage 17-33% на test cad'ах — это design limitation. ## Числа ``` DOM.РФ EKB объектов: 3 855 Объектов с objective_complex_mapping: 114 ← только 3% mv_layout_velocity rows: 459 (114 obj × ~4 buckets) ``` `mv_layout_velocity` берёт velocity **только из Objective** через `objective_complex_mapping`. Если ЖК нет в mapping → нет velocity → low coverage % в `/best-layouts` response. ## Что делать (ETL backfill) ### Phase 1: Auto-match по name+dev (fuzzy) `backend/app/services/etl/objective_backfill.py` (NEW): - LEFT JOIN `domrf_kn_objects` ↔ `objective_corpus_room_month` - Fuzzy match по `comm_name` (Levenshtein / pg_trgm similarity) - Match по `dev_inn` если есть - Score >= threshold (e.g. 0.85) → auto-insert в `objective_complex_mapping` - Result: target ~50-70% coverage (1500-2700 ЖК mapped) automatically ### Phase 2: Manual review queue (опц) Score 0.6-0.85 → admin review queue (admin UI): - Suggested matches per DOM.РФ obj - Approve/reject/edit - Goal: 80%+ coverage ### Phase 3: Celery beat refresh После backfill: - `mv_layout_velocity` REFRESH CONCURRENTLY (helper `layout_velocity_refresh.py` уже готов) - Schedule daily refresh при changes в `objective_corpus_room_month` ## Acceptance - [ ] `objective_complex_mapping` row count: 114 → ≥1500 (вместо 3%, целевое 40%+) - [ ] `mv_layout_velocity` rows: 459 → ≥6000 (4-bucket × 1500 obj) - [ ] `/best-layouts` coverage % на test cad >50% (вместо 17%) - [ ] Backfill idempotent (re-run не дублирует) - [ ] Manual review queue UI (Phase 2 — опц) ## Effort - Phase 1 auto-match: ~1.5-2 дня (ETL + fuzzy matching + tests) - Phase 2 review queue: ~1 день (опц) - Phase 3 celery beat: 0.5 дня ## Owner `backend-engineer` (ETL + matching algorithm) + `database-expert` (если нужны index'ы на `domrf_kn_objects(comm_name)` для pg_trgm) ## Links - Triggered by: best-layouts low coverage audit 2026-05-16 - Built on: PR #194 (mv_layout_velocity), `objective_corpus_room_month` (existing) - Related: #113 (Layout Analysis Phase 2.1)