ea24289f47
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m36s
Group A of the scraper_kit migration epic (#2277). Switches admin.py's manual "run parser" debug endpoints and scripts/ingest_domclick_jsonl.py from direct app.services.scrapers.* imports to their scraper_kit.providers.* equivalents, using the DI adapters (RealScraperConfig/RealMatcherAdapter/RealProxyProvider, app.services.scraper_settings.get_scraper_delay) already established by app.scheduler_main._run_kit_scheduler. Migrated: /scrape (AvitoScraper/CianScraper/YandexRealtyScraper + save_listings), scrape_avito_house, scrape_avito_detail, scrape_avito_imv, scrape_yandex_detail, scrape_yandex_valuation, scrape_cian_detail, cian_auto_login's BrowserFetcher. scripts/ingest_domclick_jsonl.py: ScrapedLot/save_listings + (now that #2307/ Group D ported a kit equivalent while this was in flight) DomClickDetailEnrichment/ save_detail_enrichment. Deliberately NOT migrated (documented in admin.py): scrape_yandex_newbuilding / scrape_cian_newbuilding — their kit equivalents (providers/{yandex,cian}/newbuilding.py) construct an internal BrowserFetcher(...) without the mandatory `endpoint` kwarg, so any call crashes/silently-fails regardless of caller-side DI. Bug lives in scraper_kit provider code, out of scope here (only consuming, not touching provider logic) — flagged as follow-up. Parity proven via tests/support/parity.py (assert_parity) against the exact functions each debug route now calls, on offline fixtures — no live network/DB. Refs #2305 |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_admin_avito_kit_parity.py | ||
| test_admin_cian_kit_parity.py | ||
| test_admin_domclick_ingest_kit_parity.py | ||
| test_admin_yandex_kit_parity.py | ||
| test_avito_403_backconnect_reconnect.py | ||
| test_avito_429_bucket_skip.py | ||
| test_avito_anti_bot.py | ||
| test_avito_detail_403_reconnect.py | ||
| test_avito_detail_429_reconnect.py | ||
| test_avito_detail_browser_only.py | ||
| test_avito_detail_kit_parity.py | ||
| test_avito_detail_not_found.py | ||
| test_avito_detail_soft_block.py | ||
| test_avito_detail_warmup.py | ||
| test_avito_exhaustive.py | ||
| test_avito_imv_browser_transport.py | ||
| test_avito_incremental_pagination.py | ||
| test_avito_sweep_retry.py | ||
| test_avito_unix_date_tz_consistency.py | ||
| test_browser_fetcher.py | ||
| test_cian_exhaustive.py | ||
| test_domclick.py | ||
| test_domclick_detail.py | ||
| test_domclick_detail_kit_parity.py | ||
| test_ekb_geoportal_client_kit_parity.py | ||
| test_price_brackets.py | ||