PR #335 использовал `with db.begin_nested():` SAVEPOINT для per-row UPDATE,
но outer transaction никогда не commit'ился. После цикла Celery task
закрывал session (`db.close()` в finally) → autobegin'd outer tx
rollback → все UPDATE'ы теряются.
Симптом: task завершается с processed=3 succeeded=3, в Glitchtip нет ошибок,
но в БД ни одного catalog_scraped_at != NULL.
Fix: добавлен db.commit() после цикла + rollback на ошибку commit'а.
PR #322 used placeholder URLs:
- /api/object/{id}/documents -> 404 (Playwright showed: 5 real endpoints:
document/rpd / developer/report / project/documentation /
documentation/other / document/permits). Replaced with 5 parallel
fetches + merge into single upsert_documents call.
- /api/object/{id}/checks -> 404. 6 checkboxes likely inline in bulk
kn/object payload; endpoint /checks does not exist. Commented out
_fetch_obj_checks_safe with TODO for separate investigation PR.
- BrowserSession Semaphore(3) -> Semaphore(6) — double concurrency
for intra-object gather.
GlitchTip: kn_scrape_failures run #19 showed 40 failures each for
both placeholder endpoints across first 30 objects of 1532.
Add _extract_obj_class_from_ai() with regex patterns for Комфорт/Бизнес/Премиум/Элит/Стандарт.
_norm_object() uses it as fallback; objClass from API still takes priority if ever returned.
Add backfill SQL 105 to retroactively populate existing rows from raw JSON payloads.
Coverage: ~1806/4548 rows will be filled; remainder NULL is expected (Нежилое/no AI desc).
Replace unweighted AVG(deals_total_avg_area_m2) and AVG(deals_total_avg_price_thousand_rub_per_m2)
with SUM(x * count) / NULLIF(SUM(count), 0) pattern in _INLINE_VELOCITY_SQL.
Months with zero deals no longer dilute the weighted mean 2-4x.
P0 follow-up to PR #290 (mv fix, issue #21).
When Objective mapping coverage falls below 50% of competitors in the radius,
fall back to rosreestr_deals JOIN on the parcel's cadastral quarter. Audit shows
237/237 EKB quarters (100%) have rosreestr data for the last 12 months, compared
to <20% Objective coverage before bulk mapping.
- velocity.py: add _compute_rosreestr_fallback(), _OBJECTIVE_COVERAGE_MIN_RATIO
constant, velocity_source field on VelocityResult (objective/rosreestr_fallback/none)
- parcels.py: extract cad_quarter from cad_num, pass to compute_velocity
- site-finder.ts: add velocity_source field to Velocity interface
- VelocityBlock.tsx: badge "Источник: квартальные сделки" when rosreestr_fallback
Epic #271 item #17
- Add AUTO_ACCEPT_THRESHOLD_V2 = 0.80 constant to objective_backfill.py
- Add min_threshold param to find_match_candidates() (default REVIEW_THRESHOLD=0.6)
- Add match_method param to auto_apply_matches() (default 'fuzzy_trgm')
- Add ?v2=true query param to POST /api/v1/admin/etl/objective-backfill
- v2 mode: threshold=0.80, method='fuzzy_v2', search from 0.80 (not 0.60)
- Return type widened to dict[str,object] to include threshold_used + match_method_used
DB audit: 1068 unmapped EKB objs; v1 adds ~133 rows, v2 adds ~47 more.
Expected coverage: 8.5% → ~20% after sequential v1+v2 runs on prod.
Run: POST /objective-backfill (v1) then POST /objective-backfill?v2=true (v2).
Part of task #44 Part A, epic #271.
Replace INNER JOIN on objective_complex_mapping with LEFT JOIN approach:
competitors without mapping now return velocity=0 with
velocity_data_available=False instead of being silently dropped.
UI: VelocityBlock shows 'нет данных velocity' badge when flag is False
and hides the gauge (meaningless zero). TS type updated (additive optional
field, backward compat).
Audit (EKB, region_cd=66): mapped=129, unmapped=1387, total=1516.
Closes#271 item #10
Round 1 (commit bcd7dc8) был broken: на 2-bucket входах surplus уходил в free
полностью без учёта capacity → free превышал cap → следующая итерация
clamp'ировала его и наоборот. Infinite oscillation в FastAPI handler.
Round 2 fix per review BLOCK (#282 comment):
- Surplus распределяется пропорционально available capacity (cap - v),
не текущему v. Free никогда не вылетит выше cap.
- free = строго < cap (не <=) — иначе деление на 0 capacity.
- Hard guard `for _ in range(N+1)` — гарантированно завершается.
- Pathological (surplus > total_capacity): возвращаем оригинальный pct_map
+ cap_skipped=True (sum=100 invariant сохранён).
- Hamilton round вынесен в _hamilton_round() helper.
Tests:
- 2-bucket cases (90/10, 70/30, 99/1) expected cap_skipped=True
- test_cap_iteration_count_bounded — все pathological завершаются < 100ms
- All 13 cases verified standalone (3 fast-path + 7 reproduced + 3 pathological)
_build_recommendation зеркалило рынок через чистый Hamilton apportionment.
Добавлен _cap_and_redistribute: bucket >35% clamp к 35%, surplus
пропорционально перераспределяется в остальные bucket'ы (Hamilton финальный
pass гарантирует sum=100). Pathological case (все bucket'ы >35%) — warning
в лог, cap пропускается. Frontend: warning banner при maxPct>60 в
RecommendationCard (Tailwind amber utilities, без inline hex).
DOM.РФ API classifies euro-format flats (26-50м²) as rooms=2.
_SUPPLY_BATCH_SQL now maps:
rooms=2 + area<35 → 'euro-1'
rooms=2 + area<50 → 'euro-2'
before the generic rooms IN (1,2,3) branch.
Audit: 4 140 units rooms=2 + area<35 in domrf_kn_flats were inflating
the supply count for genuine 2-room apartments (median 57м²).
euro buckets have no velocity counterpart in objective_corpus_room_month
so they are filtered out of top_layouts by min_velocity.
Also extends RoomBucket Literal and room_bucket_from_flat() in
layout_signature.py with the new euro-1/euro-2 values.
Closes#271 item 8
Resolve conflict in best_layouts.py: keep SF-01 inline velocity
(deals_window numerator) + add SF-05 sold_pct clamp at 100% and
is_oversold flag from main.
