gendesign/tradein-mvp/backend/app/tasks
bot-backend 772ae116b5 fix(tradein/deactivate): validate cap_mult, calibrate avito, document scope gap
Round-2 review (MAJOR) left three items open:

1. cap_mult was threaded through as a jsonb default_params parameter but never
   validated, reproducing the exact ttl_days<=0 hole the earlier guard closed.
   Verified live: cap_mult=0 -> effective_ttl=0 -> whole active pool of the
   source would deactivate; cap_mult=0.5 pushes the ceiling BELOW the operator-
   configured ttl_days. Added `if cap_mult < 1: raise ValueError` next to the
   ttl_days guard (same fail-fast contract, before any SQL). Non-numeric values
   (e.g. a stringly-typed "6" from a typo in default_params) already fail safe
   via TypeError on the comparison, caught by the same except-block -> mark_failed.
   Covered with 5 new tests (zero/negative/<1/non-numeric/mark_failed routing).

2. The mechanical part of cap_mult (parameter + wiring) was merged but never
   calibrated for avito on prod -- no migration shipped, so prod default_params
   for deactivate_stale_avito still lacked "cap_mult" and ran with the module
   default (CAP_MULT=2, ceiling=20d), which is BELOW avito's own p99 revisit gap
   (42.1d) and below the observed prod peak (floor=52, three runs 08-10..08-12).
   Added data/sql/264_deactivate_stale_avito_cap_mult.sql (idempotent, same
   pattern as 219) setting cap_mult=6 for deactivate_stale_avito only (ceiling
   60d, matching the order of magnitude already used for cian/yandex). cian/
   yandex/domklik keep the CAP_MULT=2 default -- their p99 gaps (26.6/43.0/3.1)
   sit comfortably under their default ceilings (60/60/28), no override needed.
   Pinned the calibration with a dedicated test
   (test_avito_prod_floor_is_capped_by_calibrated_cap_mult) instead of leaving
   the avito slice skipped in the false-kill coverage test.

3. Confirmed (SSH read-only, prod counts): active rows aged >60d that this PR
   cannot touch regardless of cap_mult -- cian/novostroyki 9483, cian/NULL
   211, yandex/NULL 523 (0 inside the jobs' actual scope: cian/vtorichka,
   yandex/vtorichka). deactivate_stale_cian/_yandex are scoped to
   segments=['vtorichka'] by a deliberate, documented DECISION (blanket TTL on
   novostroyki risks killing live inventory cian/yandex don't fully sweep).
   Widening that scope is a separate, riskier investigation and is out of
   scope here -- documented the gap directly in the module docstring next to
   the existing DECISION so it isn't lost.

Verification (SSH read-only against prod, 2026-08-15): recomputed the exact
per-source formula the next scheduled run will use. In-scope next-run
deactivation is currently 0 for all four sources -- the active pool has
already self-corrected to be consistent with each source's own recent
effective TTL (yesterday's yandex run used effective=54, so no active row is
older than that yet). This matches the round-2 reviewer's own conclusion: the
cap is a preventative guardrail, not a retroactive cleanup, and isn't expected
to fire on the exact day it's calibrated. It is not idle, though -- live
recompute of yandex/vtorichka's raw (uncapped) floor right now is 78.2d,
already above its 60d ceiling; the trailing 6-day counters show the identical
loop (floor=75, deactivated=0, three days straight) already recurred twice
without this cap in place. The mechanism will bind the moment the pool ages
past the ceiling, which is exactly the recurrence it exists to stop.

Tests: 106 passed (test_deactivate_stale_ttl_cap.py,
test_deactivate_stale_revisit_floor.py, test_deactivate_stale_health_gate.py,
test_deactivate_stale_listings.py, test_migrations_manifest.py). ruff clean.
scripts/check-migration-lock-timeout.py: pass (UPDATE-only migration, no
blocking DDL, no SET LOCAL needed).
2026-08-15 19:58:19 +03:00
..
__init__.py feat(tradein): search matview + indexes (Phase 3.1) (#469) 2026-05-23 14:01:18 +00:00
asking_to_sold_ratio.py fix(tradein): area-бакеты в asking→sold — и расчёт, и применение (#2620) (#2648) 2026-08-05 08:11:04 +00:00
avito_detail_backfill.py feat(tradein): вся Свердловская область — 40 городов в city-sweep (#2879) 2026-08-13 19:08:44 +00:00
backfill_house_coords_from_listings.py fix(tradein/houses): вернуть координаты объявлений в дом, когда объявления согласны (#2771) (#2780) 2026-08-07 09:08:06 +00:00
backfill_listings_coords_geoportal.py fix(tradein/geocode): прошить city_hint в deals-скрипт + развести счётчики гейта (#2603) 2026-08-05 17:36:07 +05:00
cadastral_geo_match.py fix(tradein/matching): честность тиров сопоставления домов (#2674) 2026-08-06 04:37:12 +05:00
cian_history_backfill.py fix(tradein/cian): обогащение ЖК падало не на разметке, а на сожжённом узле (#2767) (#2798) 2026-08-09 17:38:40 +00:00
deactivate_stale_avito.py fix(tradein/deactivate): validate cap_mult, calibrate avito, document scope gap 2026-08-15 19:58:19 +03:00
deal_city_price_bands_refresh.py feat(tradein/db): авто-refresh ценовых бэндов по городам + порог для малых городов (#2576) 2026-07-31 16:33:20 +03:00
deals_freshness_monitor.py fix(tradein): сигналы о сбоях наконец становятся событиями, а протухание кук предупреждает заранее (#2674) 2026-08-06 02:29:11 +05:00
domclick_detail_backfill.py docs(tradein): снять устаревшее обоснование обрыва прогона Домклика (#2854) (#2859) 2026-08-13 07:33:59 +00:00
domrf_kapremont_load.py feat(tradein): fill houses.year_built/material_walls from ДОМ.РФ КР1.1/1.2 + zhkh_year, propagate to listings (#2013) (#2476) 2026-07-12 13:12:07 +00:00
ekb_geoportal_ingest.py feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310) 2026-07-04 01:19:34 +03:00
gar_flats_load.py feat(tradein): канонический матч ГАР↔houses (0%→41% yield) + tradein_canon_addr + --match-only (#2063) 2026-06-28 15:50:07 +00:00
geocode_missing.py fix(tradein/geocode): гейт мусорного города вынести в общий хелпер и прошить в admin-путь (#2603) 2026-08-05 20:20:41 +05:00
listing_source_snapshot.py fix(tradein): убрать невыводимые события, развести «снято» и «протухло» (#2674) 2026-08-06 03:01:05 +05:00
newbuilding_enrich_backfill.py fix(tradein/scrapers): убрать оставшиеся обходы пула прокси (#2830) (#2833) 2026-08-12 10:53:36 +00:00
osm_poi_ekb_refresh.py fix(tradein/location): заменить сломанный коэффициент локации на калиброванный индекс (#2531) 2026-07-26 21:48:15 +00:00
purge_expired_trade_in_data.py fix(tradein/payments): миграция 234 → 240 — номер снова занят на main 2026-08-07 15:31:01 +03:00
refresh_search_matview.py feat(tradein): premium_houses MV + premium_building flag (#2002) 2026-06-27 16:40:39 +00:00
rosreestr_quarter_poll.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
sber_freshness_monitor.py fix(tradein/sber): сторож мерит отставание загрузки, а не календарь (#2846) (#2849) 2026-08-12 19:36:20 +00:00
sber_index_pull.py feat(tradein): СберИндекс monthly city-level price index pull (#887, data-layer) (#890) 2026-05-31 12:44:56 +00:00
yandex_address_backfill.py feat(tradein): schedule yandex address backfill task (#855) (#866) 2026-05-31 07:31:10 +00:00
yandex_detail_backfill.py fix(tradein/yandex): «непригодных» 3535 не было — адресуем их по offerId (#2838) 2026-08-12 15:04:53 +00:00
yandex_newbuilding_sweep.py refactor(tradein/tasks): yandex_newbuilding_sweep на kit + config= (#2397 Part D5) 2026-07-04 15:18:40 +03:00
zhkh_flats_load.py fix(tradein): cadastre ЖКХ-fallback + COALESCE merge-key + median-floors guard 2026-06-29 01:32:48 +03:00