gendesign/tradein-mvp/packages/scraper-kit/src/scraper_kit
bot-backend debe55b8a0
All checks were successful
CI Trade-In / changes (pull_request) Successful in 10s
CI / changes (pull_request) Successful in 10s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m42s
refactor(tradein/scraper-kit): domclick providers on shared _base.py (#2364)
Group F4d (epic #2277, parent #2352). Migrated the ad-hoc BrowserFetcher
construction in providers/domclick/serp.py::fetch_city to the shared
build_browser_fetcher(config, source) helper from Foundation (#2358).

Scope actually touched:
- serp.py: replaced the local BrowserFetcher(source="domclick", endpoint=...)
  construction with build_browser_fetcher(self._config, "domclick").
- detail.py: NO changes needed. Grepped for BrowserFetcher(/AsyncSession(/
  curl_cffi and found none - fetch_detail() receives an already-constructed
  browser_fetcher from the caller (orchestrator owns the session lifecycle,
  per its own module docstring), so there was nothing to migrate here.

Caveat verified (same class as avito, per issue instructions): the old
call site omitted use_pool, so it implicitly defaulted to False via the
BrowserFetcher class default. build_browser_fetcher(config, source) reads
use_pool=config.use_proxy_pool_browser instead - a real difference in the
value passed in. However this call site does not pass proxy_provider
(stays the default None), and BrowserFetcher._pool_proxy computes
`use_pool and proxy_provider is not None` before ever using the pool - so
with proxy_provider=None the effective behavior is identical regardless
of what use_pool evaluates to (confirmed by reading browser_fetcher.py
directly, matching the build_browser_fetcher docstring's own note that
domclick is one of the providers not yet wired to the proxy pool, #2160
P4). No observable behavior change. Documented inline at the call site.

Tests: domclick-specific suite (91 tests: golden parity, sweep, admin
ingest parity, detail + detail kit parity) all green, plus cross-provider
test_scraper_kit_pipeline_parity2.py (8 tests). Full backend suite:
3278 passed, 6 skipped, 1 pre-existing flake (test_search_cache_hit,
#2208) - no new failures.
2026-07-04 09:18:04 +03:00
..
orchestration fix(tradein/scraper-kit): проброс config= в pipeline.py::fetch_newbuilding call 2026-07-04 03:12:08 +03:00
pricing refactor(scraper-kit): unify price bisection engine across providers + dedup (#2134) 2026-07-02 16:41:04 +00:00
providers refactor(tradein/scraper-kit): domclick providers on shared _base.py (#2364) 2026-07-04 09:18:04 +03:00
__init__.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00
avito_exceptions.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00
base.py fix(tradein/scrapers): bump scraped_at при ре-подтверждении живым + бэкфилл (#2206) (#2224) 2026-07-02 20:30:01 +00:00
browser_fetcher.py feat(proxy-pool): P4 — browser dynamic pool-proxy (acquire→/fetch→relaunch) + orchestration wiring, ship-dark (#2164) 2026-07-02 19:20:29 +00:00
cian_state_parser.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00
contracts.py feat(proxy-pool): P4 — browser dynamic pool-proxy (acquire→/fetch→relaunch) + orchestration wiring, ship-dark (#2164) 2026-07-02 19:20:29 +00:00
domclick_exceptions.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00
geo.py feat(scraper-kit): copy avito providers with protocol injection, strangler (#2133 avito) 2026-07-02 15:58:00 +00:00
house_type_normalizer.py feat(scraper-kit): copy cian providers with protocol injection, strangler (#2133 cian) 2026-07-02 16:11:31 +00:00
price_brackets.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00
py.typed chore(scraper-kit): scaffold internal package + uv workspace, build-safe (#2128) 2026-07-02 14:06:07 +00:00
repair_state_normalizer.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00
snapshot_writer.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00
yandex_helpers.py feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132) 2026-07-02 15:43:15 +00:00