gendesign/tradein-mvp/backend/app/tasks
bot-backend eb564cfc01 feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310)
Migrates legacy app.services.scrapers.* imports to scraper_kit equivalents for
house_imv_backfill.py, avito_detail_backfill.py, cian_history_backfill.py,
ekb_geoportal_ingest.py, and yandex_detail_backfill.py, proving parity via
tests/support/parity.assert_parity per the epic's gate (#2304).

newbuilding_enrich_backfill.py and yandex_newbuilding_sweep.py are left fully
on legacy imports: both call into scraper_kit.providers.{cian,yandex}.newbuilding,
which construct BrowserFetcher(source=...) without the now-mandatory endpoint=
kwarg (issue #2322, verified still open against the actual provider source, not
just issue status) -- no caller-side fix is possible, matching Group A's (#2305)
precedent for the same bug in admin.py.

Config-gated kit function footguns found and fixed (Group B #2306 pattern):
  - avito fetch_detail's backconnect-on-403 retry silently drops when config=
    is omitted -- now passes config=RealScraperConfig() explicitly, with a
    regression test proving the gate.
  - kit AvitoScraper's constructor now requires ScraperConfig positionally --
    wired via RealScraperConfig(), which _rotate_ip() reads for
    avito_proxy_rotate_url.
  - BrowserFetcher(source=...) call sites (house_imv_backfill, avito/cian
    detail backfills) now pass the mandatory endpoint=settings.browser_http_endpoint.

New footgun discovered (NOT #2322, flagged for follow-up): kit's
build_warmed_session() builds its curl_cffi session via _build_detail_session()
with no config parameter at all, unlike fetch_detail -- migrating it would
silently drop the sticky MGTS-proxy egress on avito_detail_backfill's
warm-batch path (the prod default). Left build_warmed_session/
_AVITO_WARM_SEARCH_URL on legacy imports, documented inline.

Refs #2310
2026-07-04 01:19:34 +03:00
..
__init__.py feat(tradein): search matview + indexes (Phase 3.1) (#469) 2026-05-23 14:01:18 +00:00
asking_to_sold_ratio.py fix(tradein/estimator): remove dead tier-aware-ratio path — truncation artifact + footgun (#2002) 2026-06-27 17:34:54 +00:00
avito_detail_backfill.py feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310) 2026-07-04 01:19:34 +03:00
backfill_listings_coords_geoportal.py feat(tradein): бэкфилл координат листингов из ekb_geoportal_buildings (street+house match) 2026-06-20 17:08:46 +03:00
cadastral_geo_match.py fix(cadastral): single-statement match UPDATE — OFFSET chunk skipped rows 2026-06-18 10:28:19 +03:00
cian_history_backfill.py feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310) 2026-07-04 01:19:34 +03:00
deactivate_stale_avito.py chore(tradein): полное выключение источника n1 — миграция 165 + вычистка backend/ops (#2204) 2026-07-03 10:08:56 +03:00
deals_freshness_monitor.py feat(tradein/monitor): алерт на staleness данных deals по max(deal_date) (#2212) (#2226) 2026-07-02 20:50:37 +00:00
ekb_geoportal_ingest.py feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310) 2026-07-04 01:19:34 +03:00
gar_flats_load.py feat(tradein): канонический матч ГАР↔houses (0%→41% yield) + tradein_canon_addr + --match-only (#2063) 2026-06-28 15:50:07 +00:00
geocode_missing.py fix(geocode): set geocode_tried_at on successful geocode UPDATE 2026-06-16 10:07:51 +03:00
listing_source_snapshot.py feat(tradein): per-source listing price history — schema + daily snapshot (#570) (#646) 2026-05-29 12:18:04 +00:00
newbuilding_enrich_backfill.py feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310) 2026-07-04 01:19:34 +03:00
osm_poi_ekb_refresh.py feat(tradein): location-coef MVP через FDW-мост к OSM POI Птицы (#2045) 2026-07-03 23:44:55 +03:00
refresh_search_matview.py feat(tradein): premium_houses MV + premium_building flag (#2002) 2026-06-27 16:40:39 +00:00
rosreestr_quarter_poll.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
sber_index_pull.py feat(tradein): СберИндекс monthly city-level price index pull (#887, data-layer) (#890) 2026-05-31 12:44:56 +00:00
yandex_address_backfill.py feat(tradein): schedule yandex address backfill task (#855) (#866) 2026-05-31 07:31:10 +00:00
yandex_detail_backfill.py feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310) 2026-07-04 01:19:34 +03:00
yandex_newbuilding_sweep.py feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310) 2026-07-04 01:19:34 +03:00
zhkh_flats_load.py fix(tradein): cadastre ЖКХ-fallback + COALESCE merge-key + median-floors guard 2026-06-29 01:32:48 +03:00