gendesign/tradein-mvp/backend/tests/scrapers
bot-backend 5fb367295a feat(browser): split camoufox into tradein-browser container, connect-mode (#905)
Phase 0.5 — revise #884's in-backend camoufox into a dedicated browser service.
camoufox.server.launch_server exposes a Playwright WS endpoint (fingerprint/geoip/
proxy configured server-side); the backend connects remotely via playwright, so it
no longer ships Firefox.

- NEW tradein-mvp/browser/{server.py,Dockerfile}: camoufox WS server on :3000;
  Firefox fetch under app-user + ENV HOME (carries the #899 cache-path fix)
- BrowserFetcher rewritten launch-in-process → playwright.firefox.connect(ws_endpoint);
  recycle (close+reconnect every N) + crash-recovery (TargetClosed → reconnect + 1 retry)
- backend Dockerfile slimmed: drop 5 Firefox libs + camoufox fetch layer
- pyproject: remove camoufox[geoip] from backend, promote playwright to main deps
- docker-compose.prod.yml: add browser service (mem_limit 2.5g moves here), remove
  mem_limit from backend, backend depends_on browser (service_started)
- config: browser_ws_endpoint default ws://tradein-browser:3000/playwright
- tests/scrapers/test_browser_fetcher.py rewritten for connect-mode (6 pass)

Dormant: scraper_fetch_mode stays curl_cffi → no prod behavior change.
launch_server signature verified (**kwargs→launch_options, port passes through).

Refs #905, #883, #884, #899
2026-05-31 17:24:35 +03:00
..
__init__.py fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487) 2026-05-23 20:08:40 +00:00
test_avito_anti_bot.py fix(tradein): sweep shared-session (SERP+detail) через mobile proxy (#623) (#805) 2026-05-30 18:07:37 +00:00
test_browser_fetcher.py feat(browser): split camoufox into tradein-browser container, connect-mode (#905) 2026-05-31 17:24:35 +03:00