From 8aaa42f1019db54ee79c2a18276e31c86becc611 Mon Sep 17 00:00:00 2001 From: bot-backend Date: Sun, 28 Jun 2026 19:30:14 +0300 Subject: [PATCH] chore(tradein/deploy): don't recreate scraper on infra-only deploys + 120s graceful stop (phase 0+1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tradein-scraper container runs the same image as backend (python -m app.scheduler_main, in-app scheduler) and owns the in-flight scrape sweeps (a browser card takes ~15-27s). Two problems compounded to kill running jobs on every deploy: Phase 0 (deploy-tradein.yml): SCRAPER_CHANGED also fired on the `infra` paths-filter (compose / workflow / deploy/**), so any generic infra edit recreated the scraper container and SIGKILLed the running job. Dropped the `|| infra == 'true'` term — only real scraper-code paths (already covered by the `scraper` paths-filter) or a manual workflow_dispatch recreate it. Phase 1 (docker-compose.prod.yml): the scraper had no stop_grace_period, so Docker's default 10s window SIGKILLed an in-flight card (15-27s) before it could finish. Added stop_grace_period: 120s + explicit stop_signal: SIGTERM so a running unit can finish + checkpoint. The cooperative-drain handler that consumes SIGTERM lands in a later phase; this is the foundation. --- .forgejo/workflows/deploy-tradein.yml | 8 +++++++- tradein-mvp/docker-compose.prod.yml | 5 +++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/.forgejo/workflows/deploy-tradein.yml b/.forgejo/workflows/deploy-tradein.yml index eafe2980..6ae7a76c 100644 --- a/.forgejo/workflows/deploy-tradein.yml +++ b/.forgejo/workflows/deploy-tradein.yml @@ -276,7 +276,13 @@ jobs: env: IMAGE_TAG: latest GHCR_PAT: ${{ secrets.GHCR_PAT }} - SCRAPER_CHANGED: ${{ needs.changes.outputs.scraper == 'true' || needs.changes.outputs.infra == 'true' || github.event_name == 'workflow_dispatch' }} + # Phase 0: generic infra edits (compose / workflow / deploy/**) must NOT + # recreate the scraper and SIGKILL a running multi-hour job. Only genuine + # scraper-code paths (the `scraper` paths-filter already covers + # app/services/scrapers/**, scrape_pipeline.py, scheduler.py, + # scheduler_main.py, app/tasks/**) — or a manual workflow_dispatch — + # should trigger a scraper recreate. (infra term intentionally dropped.) + SCRAPER_CHANGED: ${{ needs.changes.outputs.scraper == 'true' || github.event_name == 'workflow_dispatch' }} GITHUB_SHA: ${{ github.sha }} with: host: ${{ secrets.DEPLOY_HOST }} diff --git a/tradein-mvp/docker-compose.prod.yml b/tradein-mvp/docker-compose.prod.yml index a349595b..d3a3d12f 100644 --- a/tradein-mvp/docker-compose.prod.yml +++ b/tradein-mvp/docker-compose.prod.yml @@ -129,6 +129,11 @@ services: browser: condition: service_started restart: unless-stopped + # Phase 1: give the in-flight scrape unit (browser card ~15-27s) time to finish + # + checkpoint on SIGTERM before Docker SIGKILLs. Default grace is only 10s, which + # SIGKILLs a running card mid-flight. (Cooperative-drain handler lands in a later phase.) + stop_grace_period: 120s + stop_signal: SIGTERM networks: - tradein-net