gendesign/tradein-mvp/backend/tests/scrapers
bot-backend 7049717782
All checks were successful
CI Trade-In / changes (pull_request) Successful in 11s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 11s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m56s
test(tradein/scraper-kit): golden-parity + config-footgun proof for avito_imv (#2334)
Group E1 of the scraper_kit migration epic (#2277 -> #2308 -> #2334): prepares the
kit side of app/services/scrapers/avito_imv.py -> scraper_kit.providers.avito.imv
WITHOUT switching any real call site (estimator.py is #2337, a separate final step;
house_imv_backfill.py is deliberately deferred too, see report).

- Golden-parity: full evaluate_via_imv() async flow (geocode A/B + evaluate C) proven
  byte-identical between legacy and kit on REAL live-captured Avito API responses
  (tests/fixtures/avito_imv_geo_position.json + avito_imv_getdata.json), routed through
  the browser_fetcher transport to stay offline/deterministic. Previously this function
  had zero parity coverage (only _parse_price was covered, via admin.py Group A).
  Added cheap pure-function parity for compute_imv_cache_key/_parse_geo_position/
  _parse_placement_history/_parse_suggestions too.
- Config-footgun regression: kit evaluate_via_imv only reads config.scraper_proxy_url
  when config= is explicitly passed; called bare it silently builds a proxy-less
  curl_cffi session even with a real proxy configured. Two tests lock this down
  (without config= -> proxies=None, with config=RealScraperConfig() -> proxies wired) —
  manually verified by breaking scraper_kit/providers/avito/imv.py's config threading,
  watching the "with config" test fail on the exact proxies assertion, then reverting.

No production import switched in this commit — see PR/issue report for the full
caller audit and the house_imv_backfill.py deferral rationale.
2026-07-04 01:49:21 +03:00
..
__init__.py fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487) 2026-05-23 20:08:40 +00:00
test_admin_avito_kit_parity.py feat(tradein/scraper-kit): migrate admin.py debug routes + domclick ingest to kit (#2305) 2026-07-04 00:27:01 +03:00
test_admin_cian_kit_parity.py feat(tradein/scraper-kit): migrate admin.py debug routes + domclick ingest to kit (#2305) 2026-07-04 00:27:01 +03:00
test_admin_domclick_ingest_kit_parity.py feat(tradein/scraper-kit): migrate admin.py debug routes + domclick ingest to kit (#2305) 2026-07-04 00:27:01 +03:00
test_admin_yandex_kit_parity.py feat(tradein/scraper-kit): migrate admin.py debug routes + domclick ingest to kit (#2305) 2026-07-04 00:27:01 +03:00
test_avito_403_backconnect_reconnect.py fix(avito): gate 403 reconnect-retry on proxy presence only, not rotate_url absence 2026-06-18 21:31:20 +03:00
test_avito_429_bucket_skip.py fix(avito): tolerate 429 per-bucket instead of banning whole sweep 2026-06-18 23:03:28 +03:00
test_avito_anti_bot.py fix(scrapers): avito pipeline — новостройки house graceful (без housePage), AvitoBlockedError per-item не крашит прогон (#1820) 2026-06-20 09:53:46 +03:00
test_avito_detail_403_reconnect.py fix(avito): reconnect-retry on detail 403/firewall under backconnect proxy 2026-06-18 22:12:03 +03:00
test_avito_detail_429_reconnect.py fix(avito): reconnect-retry on detail 429 under backconnect proxy 2026-06-18 23:29:47 +03:00
test_avito_detail_browser_only.py fix(scrapers): avito_detail_backfill — снапшот ЕКБ+active, detail browser-only без curl-fallback на бан (#1814) 2026-06-19 21:58:35 +03:00
test_avito_detail_kit_parity.py fix(tradein/tests): harden parity harness — bool-type mismatch detection + divergence-catch proof (#2304) 2026-07-03 23:45:58 +03:00
test_avito_detail_not_found.py fix(tradein/avito): reap dead 404 listings in detail backfill (don't trip block breaker) 2026-06-28 14:05:31 +03:00
test_avito_detail_soft_block.py fix(tradein/avito): detect HTTP-200 soft-block → rotate + coords-priority backfill ordering (#1950, #1967) 2026-06-28 08:06:12 +00:00
test_avito_detail_warmup.py fix(tradein/avito-detail): живой прогрев сессии + батч-на-сессию — снимает 403 на curl-backfill (#1551) 2026-06-28 22:34:38 +00:00
test_avito_exhaustive.py fix(tradein): avito _extract_total_count returns 0 on no-results page (stop empty-bracket split waste + fallback-listing scrape) 2026-06-18 20:01:41 +03:00
test_avito_imv_browser_transport.py feat(tradein-imv): route IMV backfill через BrowserFetcher.fetch_json за dormant-флагом (#915 Stage 3) (#1916) 2026-06-26 09:54:28 +00:00
test_avito_imv_kit_parity.py test(tradein/scraper-kit): golden-parity + config-footgun proof for avito_imv (#2334) 2026-07-04 01:49:21 +03:00
test_avito_incremental_pagination.py feat(avito): incremental SERP pagination with date early-stop (param-gated, default off) 2026-06-18 22:01:46 +03:00
test_avito_sweep_retry.py fix(tradein): avito sweep page-level retry — transient page error no longer aborts run 2026-06-18 14:29:01 +03:00
test_avito_unix_date_tz_consistency.py fix(scrapers/avito): unify _unix_to_date timezone across serp/houses/imv (#2129) 2026-07-02 14:03:11 +00:00
test_browser_fetcher.py feat(tradein-browser): additive POST /fetch-json + BrowserFetcher.fetch_json (#915 Stage 2) (#1915) 2026-06-26 09:30:35 +00:00
test_cian_exhaustive.py feat(scrapers): cian SERP на BrowserFetcher (camoufox), снят code-side rotate (#1806) 2026-06-19 20:41:44 +03:00
test_domclick.py feat(tradein/domclick): enrich Layer A SERP parser — description, repair-inference, minhash, seller 2026-06-27 16:35:25 +03:00
test_domclick_detail.py feat(tradein/domclick): local-runner + ingest tooling → main (split from parked Layer-B) 2026-06-28 10:26:52 +03:00
test_domclick_detail_kit_parity.py feat(tradein/scraper-kit): port domclick_detail + ekb_geoportal_client to kit (#2307) 2026-07-04 00:05:07 +03:00
test_ekb_geoportal_client_kit_parity.py feat(tradein/scraper-kit): port domclick_detail + ekb_geoportal_client to kit (#2307) 2026-07-04 00:05:07 +03:00
test_price_brackets.py perf(tradein): shared EKB seed price-brackets across avito/cian/yandex (city-dict ready for multi-city) 2026-06-18 17:55:55 +03:00