feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) #1735

Merged
lekss361 merged 1 commit from feat/browser-per-source-proxy-pool into main 2026-06-18 06:06:58 +00:00

1 commit

Author SHA1 Message Date
b9eb478c8f feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Route each scraper source (avito/cian/yandex/domclick) to its own camoufox
browser+proxy so they no longer wedge each other through a single global egress.

Feature-flagged (FEATURE_BROWSER_POOL_ENABLED, default OFF): with the flag off the
/fetch and /login paths are byte-for-byte the existing single-browser behavior.
When on, /fetch routes by body["source"] to a per-proxy browser via BROWSER_PROXY_MAP
(BROWSER_PROXY_AVITO/CIAN/YANDEX/DOMCLICK with legacy fallbacks), each guarded by its
own lazy-launched lock. /login stays single-browser in Phase 1.

BrowserFetcher gains a source arg (default avito) and sends it in the /fetch body;
all scraper callsites pass their source. No docker-compose/.env.runtime changes
(Phase 2, owner-gated).
2026-06-18 09:01:50 +03:00