_STATUS_BADGE_CLS_RE was too broad: `status|badge|tag|chip|label` matched
generic UI elements (e.g. <span class="label">, <span class="chip">) that
are unrelated to the sale-status badge, risking picking the wrong block.
Narrowed to `(?<![a-z])status(?![a-z])` — requires the literal token
"status" as a hyphen-delimited component of the CSS class (matches
`status-badge`, `flat-status-tag`, `object-status` but not bare `label`,
`tag`, `chip`, `badge`).
Adds regression test: page with generic label/tag/chip/badge elements
containing "В продаже" must NOT activate Level-1; only the real
`status-badge` block ("Продана") should be returned → status=sold.
Replace the system-curl SOCKS5 subprocess transport with the shared
BrowserFetcher (camoufox). camoufox's real-browser fingerprint is not
tarpitted by yandex and does not burn the mobile-proxy IP; raw system-curl
got the proxy IP flagged.
One BrowserFetcher is opened per scraper session in __aenter__ and reused
across all page fetches. _http_get now fetches via the browser, extracts the
gate-API JSON from camoufox's HTML <pre> wrapper, and returns it as the body
of a _CurlResponse (status 200 on success, 0 on fetch/extraction failure) so
all existing callers and the tarpit retry/rotate logic keep working unchanged.
URL building, _parse_gate_json, pagination, combos and field mapping are
untouched. Removed the dead curl subprocess constants/method.
sat_factor = 1 + ((sold_pct-50)/100)*0.30 was computed in 09_macro_and_trend.py,
written to district_economics.sat_factor, and fetched in server.py and 10_score_v2.py
— but never multiplied into any score. The live market sub-score uses a separate
sat_score = min(100, sold_pct*100/70) directly, so sat_factor was dead code that
would double-count absorption if ever wired in.
- 09_macro_and_trend.py: remove sat_factor computation, ALTER TABLE column, UPDATE
binding, and debug print column
- 10_score_v2.py: remove sat_factor from SELECT and unpacking
- server.py: remove sat_factor variable assignment and from macro_factors response
- static/index.html: remove sat_factor documentation row
- data/sql/162_drop_district_economics_sat_factor.sql: DROP COLUMN IF EXISTS
- Add window_months > 0 guard to vel_by_room dict comprehension (mirrors _monthly_rate)
- Correct outer comment in recommendation.py: honest-zero for known-zero buckets,
fallback only when velocity_by_room=None or bucket absent from _FORECAST_TO_METRIC_BUCKETS
- Add comment in as_dict() noting velocity_by_room is intentionally not serialized
(internal pipeline attr consumed directly by recommendation.py)
Introduce INFORMATIONAL_WARNING_CODES frozenset; verdict assembly now only
downgrades to «С ограничениями» when verdict-relevant warnings exist.
NO_ENGINEERING_NEARBY still appears in warnings[] for display but a clean
residential parcel (Ж-zone, no ЗОУИТ) correctly receives verdict_label=«Можно».
`_build_confidence` was calling `compute_report_confidence` without
`deal_count_months`, so the «за N мес» suffix never appeared in
production. Add `_deal_count_months(market_metrics)` extractor
(reads `window_months`, same key as `_history_months`) and pass it.
Also fix pre-existing UP038 violations (isinstance tuple → X | Y).
Add `velocity_by_room: dict[str, float] | None` to `MarketMetrics` — per-bucket
unit velocity (ед./мес) derived from the existing `sold_by_room` ROLLUP data that
`_query_sales_window` already returns. No new SQL required.
Thread per-bucket velocity through `_demand_only_overlay` via the new
`_FORECAST_TO_METRIC_BUCKETS` constant that maps each forecast bucket to its
market_metrics room-bucket keys. "80+ м²" sums "4" + "5+" keys. Fallback to
aggregate `unit_velocity` when `velocity_by_room` is None (thin-data path).
Previously `base_pace` was identical for all 5 room-buckets, so §9.4 norm and §9.2
base_pace cancelled out in pace/max_pace and ranking was driven purely by §9.5
macro_coef (segment steepness proxy). Now §9.2 reflects real per-bucket observed
demand from objective_lots.contract_date data.
Callers of `compute_market_metrics` that don't use `velocity_by_room` are unaffected
(the new field is additive to the frozen dataclass). All existing callers verified —
none construct `MarketMetrics` directly except the one production site.
#1511: replace equal-weight per-row median with volume-weighted median
(sorted by price, cumulative sold_volume_m2 weight, stop at 50th percentile).
Each corpus×month row now counts proportionally to its deal volume instead of
contributing equal weight regardless of how many flats were sold.
#1512: pin latest_stock to the single most-recent month in the window (last3[0])
instead of per-corpus ROW_NUMBER latest. Stale stock from inactive corpuses no
longer inflates the MTS numerator; stock and sold_volume_m2 denominator now
refer to the same consistent period.
Also clean pre-existing ruff E401/E701/E702/E722 violations (no logic change).
097_index_hygiene.sql de-partialized 5 listings indexes on the false premise
that is_active=100% true. 104_index_hygiene_geom_dedup.sql acknowledged the
premise is stale (49% active on prod, 2026-06-13) but only cleaned up the
redundant geom duplicate, leaving 3 composite indexes full.
Restore WHERE is_active=true on the composite hot-path indexes:
- listings_active_filter_idx (rooms, price_rub, area_m2, scraped_at DESC)
- listings_house_price_idx (house_id_fk, price_rub)
- listings_rooms_area_idx (rooms, area_m2)
Excluded: listings_geom_active_idx (dropped by 104, listings_geom_idx covers it),
listings_scraped_desc_idx (not in issue #1398 suggested fix scope).
CONCURRENTLY not used — migration runner wraps in BEGIN/COMMIT (see 117 note).
Add SCORE_THRESHOLDS constant (mirror of backend/app/api/v1/parcels.py) to server.py
and filter out weighted scores below SCORE_THRESHOLDS["хорошо"] (25.0) before appending
to the suggestions list, honouring the contract stated in the endpoint docstring.
Replace absolute min/max with robust percentiles so a single outlier ДКП
deal cannot shift the soft-bound corridor boundaries. Small samples fall
back gracefully via linear interpolation (n=3: P10≈index 0.2, P90≈index 2.8).
Both metrics were querying prinzip_deals without any date filter,
returning all-time figures while the surrounding stats (leads_window,
converted_window, conv_pct_window) were scoped to the last N months.
Now both subqueries restrict to deals linked to leads in window_leads
(via deal_id IN (...)), making all «за период» figures consistent.
- Rename top_poi → items in poi-score.json to match PoiScoreResponse TS type
(mock was cast as PoiScoreResponse but had wrong field name → items undefined
at runtime in MOCK_POI_SCORE mode → PoiList2Gis crashed at [...items].sort)
- Recompute all score_contribution values using backend formula
(weight / _MAX_STRAIGHT_SCORE * 100, _MAX_STRAIGHT_SCORE=0.315) and
poi_weighted_score=19.9 (was 72, which was inconsistent with the new normalization)
- Add assert result.poi_weighted_score == 0.0 to test_routing_decay_empty_db
to match the straight-line empty-db assertion
- Remove stale comment in PoiList2Gis.tsx saying normalization needs fixing in
site-finder-api.ts (already done backend-side in this PR)
_GEO_WEIGHT_UNKNOWN was 0.1, which equals exp(−6.9/3)≈0.10 (weight of a
confirmed-far project at ~6.9 km). Projects beyond that distance got a
weight *below* 0.1, meaning unknown-coordinate projects outweighed
confirmed-far ones — an inversion of the documented intent.
Lowered to 0.05 (≈ exp(−3) = exp(−9 km / scale)), restoring the correct
hierarchy: confirmed-close > confirmed-far > unknown. Updated TestGeoWeight
(hardcoded 0.1 expectation) and TestCannibalizationTrueMode (overlap ×
floor comment/value) accordingly. Added two new assertions in TestGeoWeight
that enforce the hierarchy monotonically and verify unknown < exp(−6.9/3).
_INN_RE now requires an explicit ИНН/inn keyword anchor (case-insensitive) within
~20 chars before a 10/12-digit block. Bare digit sequences without the keyword are
no longer candidates — eliminates false-positives on large monetary amounts such as
1 200 000 000 (which coincidentally passes the ФНС checksum). Checksum validation
is kept as a second gate to avoid redacting e.g. «ИНН 1234567890» with bad digits.
_inn_repl updated to use match.group(1) (digit-only capture group) instead of the
full match that now also includes the keyword prefix.
7 new regression tests in test_redaction.py: bare large numbers not redacted,
keyword-cued real INNs (10/12 digit) still redacted, bad-checksum + keyword left
intact, latin «inn:» accepted.
Replace whole-HTML re.search for status with a 3-level block-scoped strategy:
1. CSS badge classes (status/badge/tag/chip/label) — highest precision.
2. Proximity to block labelled «Статус» via _find_text_near.
3. Full blocks scan where sold/reserved keywords always beat free —
preventing «в продаже» from nav/similar-flats sections from
misclassifying sold flats as free.
Add _STATUS_KW_RE and _classify_status_kw at module level with full
morphological coverage: продан/продана/продано, реализован[аоы]?,
забронирован[аоы]?, свободн[аоы]?.
Add 31 tests in test_domrf_catalog_parse.py covering all three
extraction levels plus regression for fem. word-form «Квартира продана».
_YEAR_PERIODS ('год') → month=1 collided with Q1 ('I квартал') → month=1
for the same region/year: both produced obs_date=YYYY-01-01, so the second
series in the XML silently overwrote the first in the in-memory by_key dict.
Fix: add _emiss_period_granularity() and extend the dedup key from
(region, obs_date) to (region, obs_date, granularity). 'year' and 'quarter'
are now distinct slots, so both rows survive the parser and reach the upsert.
Test: test_yearly_and_q1_both_survive_dedup verifies len==2 with both
values present; test_period_granularity covers all classification branches.
