lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions

fix(browser): read AVITO_PROXY_URL for camoufox proxy (#905)

Browse source 
Prod smoke: tradein-browser fetched avito but got the ~400KB soft-block stub
(preloadedState present but priceDetailed=0, firewall=0) — because camoufox ran
with NO proxy (datacenter IP). Root cause: server.py read SCRAPER_PROXY_URL, but
the proxy in backend/.env.runtime is named AVITO_PROXY_URL (mobileproxy, #623) —
the same var curl_cffi-avito uses. So the var was empty in the browser container.

Fix: read AVITO_PROXY_URL first, SCRAPER_PROXY_URL as a generic fallback. One-line
env-name swap; the value is the working mobileproxy egress already on the host.

After deploy, re-run tradein-mvp/scripts/browser-smoke.sh — expect real data
(priceDetailed>0, ~1.4MB, firewall 0). IP rotation (AVITO_PROXY_ROTATE_URL) stays
backend-side and keeps working (same proxy endpoint, changing egress IP).

Refs #905, #883, #623
This commit is contained in:
bot-backend 2026-05-31 18:49:45 +03:00
parent 71874310aa
commit 57acec215f
1 changed files with 9 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
tradein-mvp/browser/server.py
View file

@ -18,7 +18,8 @@ WS-сервер) несовместим с современными версия
BROWSER_RECYCLE_PAGES — страниц в одном сеансе браузера до перезапуска (default: 15)
BROWSER_NAV_TIMEOUT_MS — таймаут page.goto в мс (default: 60000)
BROWSER_WAIT_MS — ожидание после DOMContentLoaded в мс (default: 2500)
SCRAPER_PROXY_URL — прокси URL формата ``http://user:pass@host:port`` (опционально)
AVITO_PROXY_URL — прокси ``http://user:pass@host:port`` (mobileproxy, #623);
SCRAPER_PROXY_URL — generic-fallback. Без него — soft-block.
Эндпоинт /fetch доступен из Docker-сети как ``http://tradein-browser:3000/fetch``.
"""
@ -39,7 +40,13 @@ BROWSER_PORT: int = int(os.environ.get("BROWSER_PORT", "3000"))
BROWSER_RECYCLE_PAGES: int = int(os.environ.get("BROWSER_RECYCLE_PAGES", "15"))
BROWSER_NAV_TIMEOUT_MS: int = int(os.environ.get("BROWSER_NAV_TIMEOUT_MS", "60000"))
BROWSER_WAIT_MS: int = int(os.environ.get("BROWSER_WAIT_MS", "2500"))
SCRAPER_PROXY_URL: str | None = os.environ.get("SCRAPER_PROXY_URL")
# Прокси: основной источник — AVITO_PROXY_URL (mobileproxy, #623, лежит в
# backend/.env.runtime, тот же что у curl_cffi-avito). SCRAPER_PROXY_URL —
# опциональный generic-fallback на будущее (cian/yandex). Без прокси camoufox
# фетчит с датацентр-IP → avito отдаёт soft-block заглушку.
SCRAPER_PROXY_URL: str | None = os.environ.get("AVITO_PROXY_URL") or os.environ.get(
"SCRAPER_PROXY_URL"
)
def _parse_proxy(proxy_url: str | None) -> dict[str, str] | None: