fix(tradein): camoufox fetch under app-user so runtime finds Firefox (#884) #899

Merged

lekss361 merged 1 commit from fix/884-camoufox-cache-path into main

2026-05-31 13:46:08 +00:00

bot-backend commented

2026-05-31 13:45:00 +00:00

Collaborator

Problem (found by real build-verify on the deploy host)

The #884 Dockerfile ran RUN python -m camoufox fetch as root, so the ~960MB Firefox build landed in /root/.cache/camoufox. But the container runs as app (uid 1000), and camoufox/platformdirs resolves its cache off $HOME → /home/app/.cache/camoufox. Result: the moment scraper_fetch_mode is flipped to browser, the runtime would not find the browser and would try to re-download 960MB on first scrape (or fail).

Verified on the prod build host (2026-05-31)

docker build --target runner → EXIT 0, image 3.15GB (content 1GB) — builds fine.
docker run … /app/.venv/bin/python -m camoufox path (as app) → /home/app/.cache/camoufox
that dir → No such file or directory; /root/.cache/camoufox → exists but Permission denied to app.

Fix

Move the fetch after USER app and set explicit ENV HOME=/home/app (Docker does not auto-set $HOME from USER, and the fetch needs the right $HOME at build time). Now both the build-time fetch and the runtime resolve to /home/app/.cache/camoufox.

Risk

Dormant — scraper_fetch_mode default stays curl_cffi, so no runtime behavior change. This just makes the image correct for when browser-mode is enabled (Phase 1 #883). Note: merging rebuilds the camoufox image layer on deploy (the 960MB fetch re-runs once, now under app). Reviewer may want to time the merge off demo-eve, but the change itself is safe.

Test plan

Root-cause reproduced + fix derived from a real image build+run on the deploy host (commands + output above).
Re-verify after merge: rebuild image, docker run … camoufox path as app shows the dir populated, and a headless AsyncCamoufox launch returns real HTML — unblocks Phase 1.

Refs #884, #883

## Problem (found by real build-verify on the deploy host) The #884 Dockerfile ran `RUN python -m camoufox fetch` **as root**, so the ~960MB Firefox build landed in `/root/.cache/camoufox`. But the container runs as `app` (uid 1000), and camoufox/platformdirs resolves its cache off `$HOME` → `/home/app/.cache/camoufox`. Result: the moment `scraper_fetch_mode` is flipped to `browser`, the runtime would not find the browser and would try to re-download 960MB on first scrape (or fail). ### Verified on the prod build host (2026-05-31) - `docker build --target runner` → **EXIT 0**, image 3.15GB (content 1GB) — builds fine. - `docker run … /app/.venv/bin/python -m camoufox path` (as app) → `/home/app/.cache/camoufox` - that dir → **`No such file or directory`**; `/root/.cache/camoufox` → exists but **`Permission denied`** to app. ## Fix Move the fetch **after `USER app`** and set explicit `ENV HOME=/home/app` (Docker does not auto-set `$HOME` from `USER`, and the fetch needs the right `$HOME` at build time). Now both the build-time fetch and the runtime resolve to `/home/app/.cache/camoufox`. ## Risk **Dormant** — `scraper_fetch_mode` default stays `curl_cffi`, so no runtime behavior change. This just makes the image correct for when browser-mode is enabled (Phase 1 #883). Note: merging rebuilds the camoufox image layer on deploy (the 960MB fetch re-runs once, now under app). Reviewer may want to time the merge off demo-eve, but the change itself is safe. ## Test plan - [x] Root-cause reproduced + fix derived from a real image build+run on the deploy host (commands + output above). - [ ] Re-verify after merge: rebuild image, `docker run … camoufox path` as app shows the dir **populated**, and a headless `AsyncCamoufox` launch returns real HTML — unblocks Phase 1. Refs #884, #883

bot-backend added 1 commit 2026-05-31 13:45:01 +00:00

fix(tradein): camoufox fetch under app-user so runtime finds Firefox (#884 ) 751059cb01

Build-verify on the deploy host (2026-05-31) showed the camoufox Firefox build
landed in /root/.cache/camoufox (RUN python -m camoufox fetch ran as root), but
the runtime app user (uid 1000) resolves the cache to /home/app/.cache/camoufox
→ 'browser not found' the moment scraper_fetch_mode flips to browser.

Fix: run the fetch AFTER USER app + set explicit ENV HOME=/home/app (Docker does
not auto-set HOME from USER, and camoufox/platformdirs keys the cache off $HOME).
Now build-time fetch and runtime both use /home/app/.cache/camoufox.

Verified: A) 'camoufox path' as app = /home/app/.cache/camoufox; B) that dir was
empty pre-fix while /root/.cache was root-only (Permission denied to app).
Dormant change — scraper_fetch_mode default stays curl_cffi (no runtime behavior
change), unblocks Phase 1 (#883).

Refs #884, #883