Commit graph

3 commits

Author SHA1 Message Date
bot-backend
39826058ea fix(tradein/scraper-kit): wire browser proxy-pool into yandex/cian SERP paths (#2160)
P4 (#2164) подключил раздачу прокси из пула (scrape_proxies) в camoufox-браузер
только для avito-путей. Yandex/Cian SERP-скраперы создавали BrowserFetcher сами в
__aenter__ без пула → весь их трафик шёл через env-прокси браузера (перегружен).

Прокидываем proxy_provider + use_pool=config.use_proxy_pool_browser в BrowserFetcher
внутри YandexRealtyScraper / CianScraper, а сам proxy_provider — через
run_yandex_city_sweep / run_yandex_full_load / run_cian_city_sweep / run_cian_full_load
и kit-scheduler (_job_yandex_city_sweep). Ship-dark сохранён: proxy_provider=None ИЛИ
флаг off → BrowserFetcher без пула = env-прокси, поведение байт-в-байт как раньше.

newbuilding.fetch_newbuilding НЕ трогали — config/proxy_provider до неё не доходят
(сигнатура без них, callers не передают), нужен отдельный рефактор.
2026-07-03 09:23:05 +03:00
ca473a0012 feat(proxy-pool): P4 — browser dynamic pool-proxy (acquire→/fetch→relaunch) + orchestration wiring, ship-dark (#2164)
All checks were successful
Deploy Trade-In / build-browser (push) Successful in 2m13s
Deploy Trade-In / build-backend (push) Successful in 1m18s
Deploy Trade-In / deploy (push) Successful in 2m29s
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
2026-07-02 19:20:29 +00:00
509cecfa6a feat(scraper-kit): copy scheduler with registry-dispatch (25+ trigger_* → dict), strangler (#2136)
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 1m21s
Deploy Trade-In / deploy (push) Successful in 59s
2026-07-02 17:44:03 +00:00