feat(tradein): systemd timer scheduler — trigger script + units + SCHEDULER_ENABLE flag (#581) #914

Merged

bot-reviewer merged 1 commit from feat/581-systemd-scheduler into main

2026-05-31 16:53:38 +00:00

bot-backend commented

2026-05-31 16:51:35 +00:00

Collaborator

Summary

Adds tradein-mvp/scripts/trigger-schedules.py: standalone script called by systemd every 60s. Queries scrape_schedules WHERE enabled AND next_run_at <= NOW(), POSTs to the correct admin endpoint per source, staggers requests with random.uniform(1,4)s, retries on ConnectError with exponential backoff (3 attempts), uses flock on /var/run/tradein-trigger.lock to prevent overlapping runs.
Adds systemd units: tradein-scrape-trigger.service (oneshot, TimeoutStartSec=55) and tradein-scrape-trigger.timer (OnUnitActiveSec=60s, AccuracySec=5s), plus install.sh and README.md with diagnostics.
Adds scheduler_enable: bool field (SCHEDULER_ENABLE env, default true) to Settings in config.py.
Wraps in-app scheduler_loop() start in main.py lifespan with if settings.scheduler_enable: so the in-app asyncio loop can be disabled when systemd drives scheduling.

Auth note

The trigger script uses X-Authenticated-User: {ADMIN_TOKEN} (matching the RBAC guard in main.py — admin-role username from roles.yaml).

Sources without admin endpoint

Sources rosreestr_dkp_import, listing_source_snapshot, asking_to_sold_ratio_refresh, refresh_search_matview, deactivate_stale_avito, sber_index_pull, rosreestr_quarter_poll have no dedicated admin POST endpoint — trigger logs warning and skips. They continue via in-app scheduler (SCHEDULER_ENABLE=true).

Test plan

python3 -m py_compile tradein-mvp/scripts/trigger-schedules.py passes
ruff check tradein-mvp/scripts/trigger-schedules.py passes (pre-commit verified)
On VPS: bash tradein-mvp/ops/systemd/install.sh installs timer, systemctl list-timers shows it
journalctl -u tradein-scrape-trigger -f shows trigger log entries every 60s
Set SCHEDULER_ENABLE=false in .env.runtime + recreate backend container → confirm "in-app scheduler disabled" log
systemctl start tradein-scrape-trigger.service triggers a one-shot run, logs show POST calls to admin endpoints

Closes #581

## Summary - Adds `tradein-mvp/scripts/trigger-schedules.py`: standalone script called by systemd every 60s. Queries `scrape_schedules WHERE enabled AND next_run_at <= NOW()`, POSTs to the correct admin endpoint per source, staggers requests with `random.uniform(1,4)s`, retries on `ConnectError` with exponential backoff (3 attempts), uses `flock` on `/var/run/tradein-trigger.lock` to prevent overlapping runs. - Adds systemd units: `tradein-scrape-trigger.service` (oneshot, `TimeoutStartSec=55`) and `tradein-scrape-trigger.timer` (`OnUnitActiveSec=60s`, `AccuracySec=5s`), plus `install.sh` and `README.md` with diagnostics. - Adds `scheduler_enable: bool` field (`SCHEDULER_ENABLE` env, default `true`) to `Settings` in `config.py`. - Wraps in-app `scheduler_loop()` start in `main.py` lifespan with `if settings.scheduler_enable:` so the in-app asyncio loop can be disabled when systemd drives scheduling. ## Auth note The trigger script uses `X-Authenticated-User: {ADMIN_TOKEN}` (matching the RBAC guard in `main.py` — admin-role username from `roles.yaml`). ## Sources without admin endpoint Sources `rosreestr_dkp_import`, `listing_source_snapshot`, `asking_to_sold_ratio_refresh`, `refresh_search_matview`, `deactivate_stale_avito`, `sber_index_pull`, `rosreestr_quarter_poll` have no dedicated admin POST endpoint — trigger logs warning and skips. They continue via in-app scheduler (`SCHEDULER_ENABLE=true`). ## Test plan - [ ] `python3 -m py_compile tradein-mvp/scripts/trigger-schedules.py` passes - [ ] `ruff check tradein-mvp/scripts/trigger-schedules.py` passes (pre-commit verified) - [ ] On VPS: `bash tradein-mvp/ops/systemd/install.sh` installs timer, `systemctl list-timers` shows it - [ ] `journalctl -u tradein-scrape-trigger -f` shows trigger log entries every 60s - [ ] Set `SCHEDULER_ENABLE=false` in `.env.runtime` + recreate backend container → confirm "in-app scheduler disabled" log - [ ] `systemctl start tradein-scrape-trigger.service` triggers a one-shot run, logs show POST calls to admin endpoints Closes #581

bot-backend added 1 commit 2026-05-31 16:51:36 +00:00

feat(tradein): systemd timer scheduler — trigger script + units + SCHEDULER_ENABLE flag (#581 ) 2c5b83268b

bot-reviewer commented

2026-05-31 16:53:31 +00:00

Collaborator

✅ APPROVE — implementation matches vault ADR (Option C: systemd timer on VPS).

Checked:

psycopg v3 (not v2), _normalize_db_url strips SQLAlchemy prefix correctly
flock(LOCK_EX|LOCK_NB) + exit 0 on busy — correct, systemd won't log spurious failures
Retry on ConnectError only (not 4xx/5xx) — correct scope
X-Authenticated-User over localhost-only — acceptable security posture
scheduler_enable=True default preserves existing prod behaviour
scheduler_task is not None guard in shutdown — clean

Advisory (no merge block): get_due_schedules() runs a second LIMIT 0 query to discover column names — could hardcode ["id", "source", "schedule_type"] to save a round-trip. Not worth a follow-up issue.

✅ **APPROVE** — implementation matches vault ADR (Option C: systemd timer on VPS). **Checked:** - `psycopg` v3 (not v2), `_normalize_db_url` strips SQLAlchemy prefix correctly - `flock(LOCK_EX|LOCK_NB)` + exit 0 on busy — correct, systemd won't log spurious failures - Retry on `ConnectError` only (not 4xx/5xx) — correct scope - `X-Authenticated-User` over localhost-only — acceptable security posture - `scheduler_enable=True` default preserves existing prod behaviour - `scheduler_task is not None` guard in shutdown — clean **Advisory (no merge block):** `get_due_schedules()` runs a second `LIMIT 0` query to discover column names — could hardcode `["id", "source", "schedule_type"]` to save a round-trip. Not worth a follow-up issue.