Commit graph

4 commits

Author SHA1 Message Date
bot-backend
39826058ea fix(tradein/scraper-kit): wire browser proxy-pool into yandex/cian SERP paths (#2160)
P4 (#2164) подключил раздачу прокси из пула (scrape_proxies) в camoufox-браузер
только для avito-путей. Yandex/Cian SERP-скраперы создавали BrowserFetcher сами в
__aenter__ без пула → весь их трафик шёл через env-прокси браузера (перегружен).

Прокидываем proxy_provider + use_pool=config.use_proxy_pool_browser в BrowserFetcher
внутри YandexRealtyScraper / CianScraper, а сам proxy_provider — через
run_yandex_city_sweep / run_yandex_full_load / run_cian_city_sweep / run_cian_full_load
и kit-scheduler (_job_yandex_city_sweep). Ship-dark сохранён: proxy_provider=None ИЛИ
флаг off → BrowserFetcher без пула = env-прокси, поведение байт-в-байт как раньше.

newbuilding.fetch_newbuilding НЕ трогали — config/proxy_provider до неё не доходят
(сигнатура без них, callers не передают), нужен отдельный рефактор.
2026-07-03 09:23:05 +03:00
0937da4260 feat(proxy-pool): P3 — kit curl-paths acquire/release from pool behind USE_PROXY_POOL_CURL, ship-dark (#2163)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / test (push) Successful in 1m26s
Deploy Trade-In / build-backend (push) Successful in 1m18s
Deploy Trade-In / deploy (push) Successful in 1m22s
2026-07-02 18:56:36 +00:00
62fa1f732a refactor(scraper-kit): unify price bisection engine across providers + dedup (#2134)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 14s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Has been cancelled
Deploy Trade-In / test (push) Has been cancelled
2026-07-02 16:41:04 +00:00
19c0be5d5a feat(scraper-kit): copy cian providers with protocol injection, strangler (#2133 cian)
Some checks failed
Deploy Trade-In / test (push) Failing after 1m30s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
2026-07-02 16:11:31 +00:00