The tier_aware_ratio_enabled path (#928) binned SOLD deals by sold-ppm² against
ASKING-derived percentile bounds, then divided within-tier medians. That is a
ratio-of-truncated-medians ARTIFACT: a Monte-Carlo with a CONSTANT true
sold/asking=0.84 reproduced the prod tier values (0.94/0.99/0.96) and the skewed
deal split (57/27/15%) exactly — the "premium sells closer to asking" gradient is
100% spurious. Flipping the flag ON made prod WORSE (overall MAPE 14.6→17.2,
эконом bias +5.8→+18.1). It shipped dark (default False) but is a latent footgun.
A valid within-price-tier sold/asking is uncomputable from asking-less ДКП deals;
the per-rooms blend + the shipped hedonic (year+area) are the correct conditioning.
So the dead path is removed entirely, not just disabled.
- estimator._get_asking_sold_ratio: drop the tier branch (bounds read, t33/t66,
asking_to_sold_ratios_tiered read, empirical-Bayes shrink) + the _legacy/tier
cache-key split; keep only the legacy per-rooms → global -1 lookup. Cache key
collapses to rooms bucket. anchor_ppm2 param retained for call-site compat (now
unused).
- config: remove tier_aware_ratio_enabled + tier_ratio_shrink_k settings.
- tasks/asking_to_sold_ratio: drop the tiered refresh (DELETE/re-derive bounds +
tier rows, tiered counters); the daily task no longer touches the dead tier
tables. Legacy asking_to_sold_ratios DELETE+re-derive intact.
- tests: delete the tier-ratio unit test file (legacy path covered by
test_estimator_expected_sold.py Layer 1); fix the daily-refresh fake-db
(8→3 execute calls, 3 counter keys); drop the two segment-guard tests that
asserted the removed tiered SQL; clean a dead per_rooms_tier basis in fixtures.
Tier tables asking_to_sold_ratios_tiered / asking_to_sold_tier_bounds + migration
098 are left in place (inert once nothing reads/writes them); a DROP migration is
a separate, optional follow-up.
Regression gate stays byte-identical (flag was False in prod → active path
unchanged). Refs #2002
#1786 merged before the test fix landed → main red on
test_geocode_uses_cadastral_before_yandex. The new geoportal house-match
tier runs first in geocode(); this legacy cadastral test must mock it to
None so it exercises the cadastral forward path (else the MagicMock db row
leaks through the new tier as lat=1.0).
The legacy resolve_cian_zhk_url hit /zhk/<id>/ which now 404s, leaving the
318 geo-matched cian houses (house_sources.ext_id set, cian_zhk_url NULL)
unfetchable -> the newbuilding-enrichment backfill matched 0 houses.
Add resolve_cian_zhk_url_via_search(nb_id): fetch the cat.php newbuilding SERP
and extract the canonical zhk-<slug>.cian.ru url, ANCHORED on the
<h1 data-name="Title"> header anchor (NOT naive first-match — the SERP carries
promo/recommendation zhk-* links before the title that would otherwise resolve
the wrong ЖК and silently corrupt enrichment). Validated against the real
2.81MB prod SERP + an adversarial poisoned-recommendation test.
Wire into newbuilding_enrich_backfill: broaden selection to "has ext_id OR
cian_zhk_url", resolve+persist the url under a SAVEPOINT before enriching,
rate-limited + resumable + idempotent. Keep the old resolver (deprecated).
Bounded prod proof (5 houses, direct): 5/5 urls resolved+persisted, 3/5 fully
enriched (+18 price_dynamics, +3 reliability); the 2 misses were direct-mode
anti-bot on the 2nd fetch. Full 318-run gated on the cian mobile proxy
(mproxy.site) being restored. code-reviewer APPROVE (SQL/idempotency) +
resolver hardened against wrong-ЖК. 19 tests green.
Refs #972.
Replaces tradein.cad_buildings snapshot with live postgres_fdw foreign table reading gendesign.v_tradein_cad_buildings. Fixes /trade-in/api/v1/geocode/reverse 500 (Nominatim ban) and address_not_geocoded for cadastre addresses (e.g. Хохрякова 81).
Security (deep-review fixes):
- 100_tradein_fdw_role.sql: passwordless CREATE ROLE; password set by deploy.yml ALTER ROLE bootstrap reading GENDESIGN_FDW_PASSWORD from backend/.env.runtime (via psql :'pw' var → format %L — injection-safe).
- core/fdw.py: regex whitelist [A-Za-z0-9_-]{32,256} on password, ValueError without echoing value, try/rollback on commit.
- 060_postgres_fdw_extension.sql: connect_timeout='3' on FOREIGN SERVER + ALTER ADD/SET fallback.
- geocoder.py: _cadastral_forward_sync / _cadastral_reverse_sync wrapped in asyncio.to_thread.
- 100_*.sql: REVOKE ALL ON ALL TABLES/SEQUENCES/FUNCTIONS IN SCHEMA public; only GRANT SELECT on v_tradein_cad_buildings.
- pg_user_mappings query handles PUBLIC mapping (usename IS NULL).
Tests: 3 SQL-injection guards on ensure_fdw_user_mapping + rewritten cadastral suite.
