gendesign/tradein-mvp/backend/app/services
bot-backend 6044a1fe3c fix(browser): local camoufox + HTTP wrapper, drop broken playwright WS server (#905)
The playwright WS-server path (camoufox.server.launch_server) is incompatible
with playwright >=1.45: camoufox 0.4.11's launchServer.js requires
playwright/driver/package/lib/browserServerImpl.js, which no longer exists in
any playwright 1.45-1.60 → tradein-browser crash-looped with MODULE_NOT_FOUND
('Server process terminated unexpectedly'). Verified by running the built image.

Pivot (no playwright protocol between containers):
- browser/server.py: aiohttp service holding a local AsyncCamoufox (headless,
  os/locale/geoip/humanize, proxy from SCRAPER_PROXY_URL). POST /fetch {url} →
  new_page/goto/content/close → {html}; GET /health. Page-recycle every N +
  crash-recovery (relaunch + 1 retry), serialized via asyncio.Lock.
- browser/Dockerfile: +aiohttp (camoufox fetch-as-root/#908 block untouched).
- backend BrowserFetcher: playwright.connect → httpx POST to the browser service
  (one retry on transport/HTTP error). All playwright imports dropped.
- config: browser_ws_endpoint → browser_http_endpoint (http://tradein-browser:3000).

Verified by a REAL local image build + run: image builds, container stays Up
(Restarts=0, no crash-loop), GET /health=200, POST /fetch launches camoufox and
drives Firefox (only NS_ERROR_UNKNOWN_HOST = no DNS egress in the local build env,
proving the full HTTP→camoufox→Firefox chain; real fetch validates on prod with
network+proxy). 7 unit tests + ruff clean.

Dormant: scraper_fetch_mode stays curl_cffi. Refs #905, #883, #884
2026-05-31 18:25:00 +03:00
..
exporters fix(pdf): brand-spoof + median=0 empty-state + SSRF + дисклеймер + sources (Refs #772) (#789) 2026-05-30 16:55:56 +00:00
matching refactor(tradein): drop dead schema — sale_type_text column + audit_log (#731) (#893) 2026-05-31 13:13:43 +00:00
scrapers fix(browser): local camoufox + HTTP wrapper, drop broken playwright WS server (#905) 2026-05-31 18:25:00 +03:00
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
account_quota.py fix(tradein): atomic conditional quota increment — close TOCTOU (#747) (#749) 2026-05-30 15:52:34 +00:00
brand.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
cache.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00
cian_price_history.py feat(tradein): admin backfill endpoints — cian price-history (T8a) + yandex address (T10) + house IMV (T7) (#856) 2026-05-31 06:37:10 +00:00
cian_session.py feat(scrapers): route cian sessions through mobile proxy (Refs #806) (#818) 2026-05-30 19:10:36 +00:00
dadata.py feat(tradein): DaData /suggest tier в /suggest endpoint (PR Q2) 2026-05-27 19:08:58 +05:00
estimator.py refactor(tradein): unify kadastr_num -> cadastral_number (#732) 2026-05-31 15:41:52 +03:00
geocoder.py feat(tradein): DaData /suggest tier в /suggest endpoint (PR Q2) 2026-05-27 19:08:58 +05:00
house_imv_backfill.py feat(tradein): avito city-sweep — IMV-фаза (один прогон = SERP+detail+houses+IMV) (#861) 2026-05-31 07:09:42 +00:00
house_metadata.py feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) 2026-05-24 08:57:30 +00:00
image_sanitizer.py fix(tradein-photos): sanitize uploaded images via Pillow re-encode (#511) 2026-05-24 11:32:04 +00:00
rosreestr_poll.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
sber_index.py feat(tradein): add EKB/MSK REF_AREA codes to sber_price_index pull (#794) (#900) 2026-05-31 13:56:17 +00:00
scheduler.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
scrape_pipeline.py feat(tradein): per-anchor watchdog timeout in city sweeps (#880) (#881) 2026-05-31 10:40:41 +00:00
scrape_runs.py feat(observability): sentry alert on N consecutive scraper failures (Refs #827) (#833) 2026-05-30 20:10:29 +00:00
scraper_settings.py fix(tradein): domklik source name, scraper delay, DataDome block detect (#796 fixup) 2026-05-31 17:21:30 +03:00
search_query.py refactor(tradein): unify kadastr_num -> cadastral_number (#732) 2026-05-31 15:41:52 +03:00
yandex_address_backfill.py fix(tradein): yandex address-enrich — city-agnostic title regex (спутники ЕКБ) (#875) 2026-05-31 09:08:29 +00:00