feat(tradein-estimator): cohort filter + Tier 0 for analog selection quality #519
No reviewers
Labels
No labels
admin
analytics
auth
automation
bug
business
chore
ci
compliance
data
data-moat
docs
duplicate
dx
enhancement
Fable 5 ревью
feedback/max
generative
GG-форсайт
needs-discussion
needs-human
observability
pause-bots
performance
priority/p0
priority/p1
priority/p2
priority/p3
scope/backend
scope/db
scope/devops
scope/frontend
scope/qa
scrapers
security
site-finder
stage/1
stage/2
status/blocked
status/done
status/needs-analysis
status/needs-fix
status/qa
status/ready
status/review
status/wip
tech-debt
tradein
ux
week ревью 1
wontfix
вторичка
ИРД
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: lekss361/gendesign#519
Loading…
Add table
Reference in a new issue
No description provided.
Delete branch "feat/tradein-estimator-cohort-filter"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Production audit (estimate
d4ec4610-92a7-4985-b857-1dc6753435fc, 2026-05-24) показал смешивание разных эпох застройки в выборке аналогов: для target Малышева 84 (1978, кирпич, 9-эт) в top-10 попали 31-этажные новостройки и центральные сталинки.relevance_scoreдаёт слишком слабый вес году постройки (year_mismatch/12) — distance-в-км перевешивает. Median расплывается, CV доходит до 32%.Fix — когортный hard-filter с graceful fallback
Добавлен Tier 0 в начало каскада fallback:
target_yearзадан5 когорт массовой застройки РФ:
khrushchevbrezhnevlate_soviet2000smodern_target_cohort_range(year_built)возвращает(year_min, year_max)илиNone. None → Tier 0 skipped (graceful — поведение как до этого PR).SQL filter в _fetch_analogs
CAST(:x AS integer)обязателен — psycopg3 prepared statement не выводит тип $N вIS NULLpredicate (известный trap, см. PR #518).year_built IS NULLпропускается — не штрафуем листинги с неполными данными (типично для Avito anonymous).What didn't change
relevance_scoreweights в SQL — НЕ трогали (это была Опция 1, отдельный потенциальный PR).Diff
1 файл, +83/-9. Тесты pytest local не прогнал в worker'е (venv проблема в worktree, pre-existing), AST parse OK.
Test plan
modern(2011-2100) — должны попадать только новостройки.Deep Code Review — verdict
Summary
1d4d585· base mainStrengths
IS NULLpredicate (CAST(:cohort_year_min AS integer) IS NULL) — matches PR #518 fix and explicit comment cites it. No psycopg v3 AmbiguousParameter trap.target_year is Noneor out-of-range (1900/2050) →cohort_range=None→ Tier 0 skipped → behaviour identical to pre-PR. Fully backward-compatible default (cohort_year_min/max=None).OR year_built IS NULL) — avoids dropping Avito anonymous-address listings with incomplete metadata. Correct trade-off.MEDIUM (non-blocking, worth follow-up)
1. Cohort filter only applies on Tier W path of
_fetch_analogscall.When
cohort_rangeis set and_fetch_analogs(...cohort_year_min=...)runs, the SQL clause lives only inside the Tier W SELECT (line 1285-1290 of head). Tier S (line 1097,address ILIKE prefix) and Tier H (line 1175-1180,year_built BETWEEN target-15 AND target+15) execute BEFORE the cohort filter ever evaluates — they have early-return atreturn _stratify_candidates(...), ..., "S"|"H"(lines 1122, 1214).Concrete leak:
target_year=1978(brezhnev cohort = 1970–1989). Tier H allowsyear_built ∈ [1963, 1993]. A 1965 khrushchev or 1992 late_soviet building within 1km, ≥5 same-rooms+area listings → Tier H wins → cohort filter is bypassed →analog_tier='H'returned. PR description claims "Tier 0: 1km + ±15% area + cohort match" but in reality cohort match is enforced only on Tier W.For the target audit case (Малышева 84, 1978, 9-эт brezhnev) impact is small — year±15 = 1963-1993 leaks ≤7 years across cohort boundaries vs current ±50-year noise from 31-эт новостроек. Still an improvement. But the PR over-states intent.
Suggested follow-up (separate PR): pass cohort to Tier H too:
2. Double-execute on Tier 0 fallback.
When
len(listings_tier0) < MIN_ANALOGS_TIER_0(5), Tier A re-calls_fetch_analogswithout cohort. Inside, Tier S/H run identically to the first call (cohort filter doesn't touch them). So Tier S/H SQL roundtrips fire 2× whenever Tier 0 falls back. ~2 extra DB queries per estimate on the fallback path. Sub-second, but unnecessary. Optimisation deferred — out of scope here.3. No tier-0 observability in API response.
The
analog_tierreturned from Tier 0 call is whatever internal tier resolved (S/H/W).tier_note(line 634-643) doesn't distinguish "matched via cohort" from "matched without cohort". User-facingexplanationwon't reflect whether cohort filter actually ran. If you want to A/B verify the audit fix (Малышева 84) — there's no signal in the response. Consider emittingcohort_applied: boolinconfidence_explanationor adding a structured field. Out of scope here.MINOR
4. No tests added. PR body acknowledges venv issue in worker.
_target_cohort_rangeis the cleanest place to add a 6-case parametrized unit test (None / 1900 / 1969 / 1970 / 1988 → brezhnev / 1990 → late_soviet / 2050 → None). Smoke test plan in PR body is reasonable. Not a blocker because logic is small and graceful-fallback design contains blast radius.5.
late_sovietcohort effectively redundant (1985-1999 overlaps brezhnev 1970-1989). First-match picks brezhnev for 1985-1989 → late_soviet covers only 1990-1999. Comment says "overlap intentional" — confirm semantics. TODO worth: rename toearly_90sor shift tolate_soviet=1990–1999to remove confusion.6. Vault decision not extended.
Decision_TradeIn_DataQuality_8PR_Roadmapends at PR 8. This is effectively a PR 9 added from a second audit (UUID d4ec4610). Main session should append PR 9 entry to the decision doc post-merge. Out of scope for this PR.Cross-file impact
estimate_quality(line 528) is the only entry point invoking the newcohort_rangeorchestration.tradein-mvp/backend/app/api/v1/trade_in.py→ callsestimate_quality→ no schema change inAggregatedEstimate.Vault cross-check
Pre-flight
--no-verify/--force/--amendindicatorsRisk / blast radius
target_yearnot provided or out-of-cohort-range. Tier A fallback preserves current results.Verdict
Approve and merge. Notes (1) and (3) above are worth tracking as follow-up tasks but don't block this PR — the primary audit issue (Малышева 84 returning 31-эт novostroyki) gets meaningfully addressed by the Tier W cohort hard-filter, and the architecture leaves room for tightening Tier H later.
Deep Code Review — verdict
Summary
1d4d585· base mainStrengths
IS NULLpredicate (CAST(:cohort_year_min AS integer) IS NULL) — matches PR #518 fix and explicit comment cites it. No psycopg v3 AmbiguousParameter trap.target_year is Noneor out-of-range (1900/2050) →cohort_range=None→ Tier 0 skipped → behaviour identical to pre-PR. Fully backward-compatible default (cohort_year_min/max=None).OR year_built IS NULL) — avoids dropping Avito anonymous-address listings with incomplete metadata. Correct trade-off.MEDIUM (non-blocking, worth follow-up)
1. Cohort filter only applies on Tier W path of
_fetch_analogscall.When
cohort_rangeis set and_fetch_analogs(...cohort_year_min=...)runs, the SQL clause lives only inside the Tier W SELECT (head line 1285-1290). Tier S (line 1097,address ILIKE prefix) and Tier H (line 1175-1180,year_built BETWEEN target-15 AND target+15) execute BEFORE the cohort filter ever evaluates — they have early-return at lines 1122 / 1214.Concrete leak:
target_year=1978(brezhnev cohort = 1970–1989). Tier H allowsyear_built ∈ [1963, 1993]. A 1965 khrushchev or 1992 late_soviet building within 1km with ≥5 same-rooms+area listings → Tier H wins → cohort filter is bypassed →analog_tier='H'returned. PR description claimsTier 0: 1km + ±15% area + cohort matchbut in reality cohort match is enforced only on Tier W.For the target audit case (Малышева 84, 1978, 9-эт brezhnev) impact is small — year±15 = 1963-1993 leaks ≤7 years across cohort boundaries vs current ±50-year noise from 31-эт novostroyki. Still an improvement. But the PR over-states intent.
Suggested follow-up (separate PR): pass cohort to Tier H too:
2. Double-execute on Tier 0 fallback.
When
len(listings_tier0) < MIN_ANALOGS_TIER_0(5), Tier A re-calls_fetch_analogswithout cohort. Inside, Tier S/H run identically to the first call (cohort filter doesn't touch them). So Tier S/H SQL roundtrips fire 2× whenever Tier 0 falls back. ~2 extra DB queries per estimate on the fallback path. Sub-second, but unnecessary. Optimisation deferred — out of scope here.3. No tier-0 observability in API response.
The
analog_tierreturned from Tier 0 call is whatever internal tier resolved (S/H/W).tier_note(line 634-643) doesn't distinguishmatched via cohortfrommatched without cohort. User-facingexplanationwon't reflect whether cohort filter actually ran. If you want to A/B verify the audit fix (Малышева 84) — there's no signal in the response. Consider emittingcohort_applied: boolinconfidence_explanationor adding a structured field. Out of scope here.MINOR
4. No tests added. PR body acknowledges venv issue in worker.
_target_cohort_rangeis the cleanest place to add a 6-case parametrized unit test (None / 1900 / 1969 / 1970 / 1988 → brezhnev / 1990 → late_soviet / 2050 → None). Smoke test plan in PR body is reasonable. Not a blocker because logic is small and graceful-fallback design contains blast radius.5.
late_sovietcohort effectively redundant (1985-1999 overlaps brezhnev 1970-1989). First-match picks brezhnev for 1985-1989 → late_soviet covers only 1990-1999. Comment saysoverlap intentional— confirm semantics. TODO worth: rename toearly_90sor shift tolate_soviet=1990–1999to remove confusion.6. Vault decision not extended.
Decision_TradeIn_DataQuality_8PR_Roadmapends at PR 8. This is effectively a PR 9 added from a second audit (UUID d4ec4610). Main session should append PR 9 entry to the decision doc post-merge. Out of scope for this PR.Cross-file impact
estimate_quality(line 528) is the only entry point invoking the newcohort_rangeorchestration.tradein-mvp/backend/app/api/v1/trade_in.py→ callsestimate_quality→ no schema change inAggregatedEstimate.Vault cross-check
Decision_TradeIn_DataQuality_8PR_Roadmap— original PR 3 washouse-match tiered S→H→W(already merged in PR #507). PR #519 builds on that as a new tier (Tier 0 = cohort hard-filter wrapping Tier W). Direction consistent with audit intent (year mismatch dominates relevance score).cast-in-is-not-null-predicatesmemory rule — respected (CAST present, comment cites PR #518).Pre-flight
--no-verify/--force/--amendindicatorsRisk / blast radius
target_yearnot provided or out-of-cohort-range. Tier A fallback preserves current results.Verdict
Approve and merge. Notes (1) and (3) above are worth tracking as follow-up tasks but don't block this PR — the primary audit issue (Малышева 84 returning 31-эт novostroyki) gets meaningfully addressed by the Tier W cohort hard-filter, and the architecture leaves room for tightening Tier H later.