feat(tradein-browser): additive POST /fetch-json + BrowserFetcher.fetch_json (#915 Stage 2) #1915

Merged

bot-backend merged 1 commit from feat/915-stage2-fetch-json into main

2026-06-26 09:30:36 +00:00

bot-backend commented

2026-06-26 09:26:56 +00:00

Collaborator

Summary

Add additive POST /fetch-json to the camoufox sidecar + BrowserFetcher.fetch_json() client method.
Lets backend run an in-page fetch() from the warm camoufox page (same-origin, real fingerprint + context cookies + per-provider proxy) and get {status, body} back.
Sidecar navigates to origin (anchor page, short 500ms settle vs full BROWSER_WAIT_MS), runs fetch via page.evaluate, mirrors _fetch_once lifecycle + recycle/relaunch; _do_fetch_json mirrors _do_fetch crash-retry; fetch_json_handler mirrors fetch_handler (400/503/500).
Client mirrors fetch's single-retry-on-HTTPError/TransportError pattern.
Purely additive: /fetch, _fetch_once, _do_fetch, fetch_handler, BrowserFetcher.fetch untouched. Nothing in prod calls it yet — a follow-up PR (#915 Stage 3) wires avito IMV to it, which can unblock #562 (IMV via warm browser session instead of needing the #853 proxy).

Test plan

Sidecar browser/test_server_fetch_json.py — 8 tests (happy path, origin navigation, explicit origin, POST passthrough, 400 missing url, 400 bad json, 503 unavailable, 500 on evaluate error + page closed).
Backend tests/scrapers/test_browser_fetcher.py — 5 new fetch_json tests (returns dict, full payload, defaults, one retry, raise-after-two) → 13 passed.
ruff check clean on all changed files.
Existing sidecar smoke/pacing/resource_block green — no regressions.

Refs #915

## Summary - Add additive `POST /fetch-json` to the camoufox sidecar + `BrowserFetcher.fetch_json()` client method. - Lets backend run an in-page `fetch()` from the warm camoufox page (same-origin, real fingerprint + context cookies + per-provider proxy) and get `{status, body}` back. - Sidecar navigates to `origin` (anchor page, short 500ms settle vs full `BROWSER_WAIT_MS`), runs `fetch` via `page.evaluate`, mirrors `_fetch_once` lifecycle + recycle/relaunch; `_do_fetch_json` mirrors `_do_fetch` crash-retry; `fetch_json_handler` mirrors `fetch_handler` (400/503/500). - Client mirrors `fetch`'s single-retry-on-HTTPError/TransportError pattern. - **Purely additive**: `/fetch`, `_fetch_once`, `_do_fetch`, `fetch_handler`, `BrowserFetcher.fetch` untouched. Nothing in prod calls it yet — a follow-up PR (#915 Stage 3) wires avito IMV to it, which can unblock #562 (IMV via warm browser session instead of needing the #853 proxy). ## Test plan - [x] Sidecar `browser/test_server_fetch_json.py` — 8 tests (happy path, origin navigation, explicit origin, POST passthrough, 400 missing url, 400 bad json, 503 unavailable, 500 on evaluate error + page closed). - [x] Backend `tests/scrapers/test_browser_fetcher.py` — 5 new fetch_json tests (returns dict, full payload, defaults, one retry, raise-after-two) → 13 passed. - [x] `ruff check` clean on all changed files. - [x] Existing sidecar smoke/pacing/resource_block green — no regressions. Refs #915

Rows
Columns