bot-backend 7800f6014a refactor(tradein/scraper-kit): yandex providers on shared _base.py (#2363) ... F4c-yandex (epic #2277 Group F, parent #2352): migrate the SERP scraper's BrowserFetcher construction onto build_browser_fetcher() from the Foundation module (#2358, providers/_base.py) instead of local ad-hoc construction. - yandex/serp.py::__aenter__ now calls build_browser_fetcher(config, source=, proxy_provider=, fetch_timeout_s=) instead of constructing BrowserFetcher(...) directly. Preserves the existing fetch_timeout_s=30 (see caveat below), endpoint/proxy_provider/use_pool resolution byte-for-byte. The retry/tarpit logic in fetch_around() (rotate_ip + sleep on status==0/JSON-error) is untouched, per the Foundation module's documented exclusion. Left unchanged (deliberate, not an oversight): - yandex/serp.py::_rotate_ip's curl_cffi AsyncSession(timeout=30) — this hits the mobile-proxy provider's own changeip API, not Yandex; it intentionally carries no impersonate/headers/proxy (same as the mirrored avito/serp.py rotate-ip session). Not listed among the _base.py docstring's documented extraction sites -- consistent exclusion, not a new one. - yandex/detail.py -- no BrowserFetcher/curl_cffi construction exists in this file (uses BaseScraper's httpx-based _http_get). Nothing to migrate. - yandex/newbuilding.py -- both BrowserFetcher(...) call sites take an OPTIONAL `config: ScraperConfig | None`, a genuinely different contract from build_browser_fetcher's mandatory `config`. Migrating would require making config mandatory (breaking admin.py:1511, yandex_newbuilding_sweep.py:353, and several tests that call without config -- out of scope for a call-site swap) and breaks test_scraper_kit_newbuilding_endpoint.py's "without config, endpoint=None" regression case (its SimpleNamespace mock lacks use_proxy_pool_browser, and the mock config path can never satisfy a mandatory-config helper). Left untouched; the class-bug #2322/#2330 this file already fixed (config now threaded, just not via the shared helper) remains fixed. Also updates test_kit_serp_proxy_pool.py's 3 yandex-specific tests: they monkeypatched the module-local `yandex_serp.BrowserFetcher` name, which the migrated call path no longer references directly (it now goes through scraper_kit.providers._base.build_browser_fetcher -> _base's own BrowserFetcher import). Patch target moved to scraper_kit.providers._base.BrowserFetcher; every assertion is unchanged -- this adapts the mock to the new (intentional) call path, it does not weaken what the test verifies. Cian's tests in the same file are untouched (cian/serp.py is not migrated yet, separate F4b issue).		2026-07-04 09:38:43 +03:00
..
src/scraper_kit	refactor(tradein/scraper-kit): yandex providers on shared _base.py (#2363)	2026-07-04 09:38:43 +03:00
pyproject.toml	feat(scraper-kit): copy avito providers with protocol injection, strangler (#2133 avito)	2026-07-02 15:58:00 +00:00