lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/tradein-mvp/backend/app/services
History
bot-backend 5fb367295a feat(browser): split camoufox into tradein-browser container, connect-mode (#905) 
Phase 0.5 — revise #884's in-backend camoufox into a dedicated browser service.
camoufox.server.launch_server exposes a Playwright WS endpoint (fingerprint/geoip/
proxy configured server-side); the backend connects remotely via playwright, so it
no longer ships Firefox.

- NEW tradein-mvp/browser/{server.py,Dockerfile}: camoufox WS server on :3000;
  Firefox fetch under app-user + ENV HOME (carries the #899 cache-path fix)
- BrowserFetcher rewritten launch-in-process → playwright.firefox.connect(ws_endpoint);
  recycle (close+reconnect every N) + crash-recovery (TargetClosed → reconnect + 1 retry)
- backend Dockerfile slimmed: drop 5 Firefox libs + camoufox fetch layer
- pyproject: remove camoufox[geoip] from backend, promote playwright to main deps
- docker-compose.prod.yml: add browser service (mem_limit 2.5g moves here), remove
  mem_limit from backend, backend depends_on browser (service_started)
- config: browser_ws_endpoint default ws://tradein-browser:3000/playwright
- tests/scrapers/test_browser_fetcher.py rewritten for connect-mode (6 pass)

Dormant: scraper_fetch_mode stays curl_cffi → no prod behavior change.
launch_server signature verified (**kwargs→launch_options, port passes through).

Refs #905, #883, #884, #899
2026-05-31 17:24:35 +03:00
..
exporters fix(pdf): brand-spoof + median=0 empty-state + SSRF + дисклеймер + sources (Refs #772) (#789) 2026-05-30 16:55:56 +00:00
matching refactor(tradein): drop dead schema — sale_type_text column + audit_log (#731) (#893) 2026-05-31 13:13:43 +00:00
scrapers feat(browser): split camoufox into tradein-browser container, connect-mode (#905) 2026-05-31 17:24:35 +03:00
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
account_quota.py fix(tradein): atomic conditional quota increment — close TOCTOU (#747) (#749) 2026-05-30 15:52:34 +00:00
brand.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
cache.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00
cian_price_history.py feat(tradein): admin backfill endpoints — cian price-history (T8a) + yandex address (T10) + house IMV (T7) (#856) 2026-05-31 06:37:10 +00:00
cian_session.py feat(scrapers): route cian sessions through mobile proxy (Refs #806) (#818) 2026-05-30 19:10:36 +00:00
dadata.py feat(tradein): DaData /suggest tier в /suggest endpoint (PR Q2) 2026-05-27 19:08:58 +05:00
estimator.py refactor(tradein): unify kadastr_num -> cadastral_number (#732) 2026-05-31 15:41:52 +03:00
geocoder.py feat(tradein): DaData /suggest tier в /suggest endpoint (PR Q2) 2026-05-27 19:08:58 +05:00
house_imv_backfill.py feat(tradein): avito city-sweep — IMV-фаза (один прогон = SERP+detail+houses+IMV) (#861) 2026-05-31 07:09:42 +00:00
house_metadata.py feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) 2026-05-24 08:57:30 +00:00
image_sanitizer.py fix(tradein-photos): sanitize uploaded images via Pillow re-encode (#511) 2026-05-24 11:32:04 +00:00
rosreestr_poll.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
sber_index.py feat(tradein): add EKB/MSK REF_AREA codes to sber_price_index pull (#794) (#900) 2026-05-31 13:56:17 +00:00
scheduler.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
scrape_pipeline.py feat(tradein): per-anchor watchdog timeout in city sweeps (#880) (#881) 2026-05-31 10:40:41 +00:00
scrape_runs.py feat(observability): sentry alert on N consecutive scraper failures (Refs #827) (#833) 2026-05-30 20:10:29 +00:00
scraper_settings.py feat(tradein): global scraper delay setting (applies across all scrapers) (#485) 2026-05-23 15:43:28 +00:00
search_query.py refactor(tradein): unify kadastr_num -> cadastral_number (#732) 2026-05-31 15:41:52 +03:00
yandex_address_backfill.py fix(tradein): yandex address-enrich — city-agnostic title regex (спутники ЕКБ) (#875) 2026-05-31 09:08:29 +00:00