gendesign/tradein-mvp/deploy
lekss361 9c75ec0f78 feat(tradein): city sweep — auto ЕКБ pipeline (anchors × pages + enrichment + cancel)
- avito.py: fetch_around(pages=N, delay_override_sec) — pagination 1..N (backward compat default=1)
- scrape_pipeline.py:
  * EKB_ANCHORS const (5 точек ЕКБ)
  * run_avito_pipeline: pages + request_delay_sec params (backward compat defaults)
  * CitySweepCounters dataclass с anchors_done/total + aggregate counters
  * run_avito_city_sweep(db, run_id, *, pages_per_anchor, detail_top_n, request_delay_sec)
  * Cooperative cancel: checks scrape_runs.status каждый anchor
  * Per-anchor heartbeat + counters refresh
- scrape_runs.py NEW: utility module (create_run/update_heartbeat/mark_done/failed/cancelled/list_recent/is_cancelled)
- admin.py: 3 NEW endpoints:
  * POST /admin/scrape/avito-city-sweep — start (returns run_id, FastAPI BackgroundTasks)
  * GET  /admin/scrape/avito-city-sweep/runs?limit=N — recent runs list
  * POST /admin/scrape/avito-city-sweep/{run_id}/cancel — cooperative cancel
- 051_scrape_runs_extend.sql: ADD COLUMN params/counters/error/finished_at + cancelled в status enum
- avito-city-sweep.sh: cron wrapper с random 0-3h delay (anti-pattern-detection)
- tests/test_city_sweep.py: 16 offline smoke tests (counters, scrape_runs utils, endpoints)

Crontab entry: 0 2 * * * /opt/.../avito-city-sweep.sh (runs 02:00-05:00 UTC daily)
2026-05-23 17:33:34 +03:00
..
avito-city-sweep.sh feat(tradein): city sweep — auto ЕКБ pipeline (anchors × pages + enrichment + cancel) 2026-05-23 17:33:34 +03:00
backup-tradein-db.sh feat(tradein): бэкап tradein-postgres по cron (#397) 2026-05-22 11:22:30 +05:00
Caddyfile feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
Caddyfile.tradein-fragment feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
cron-scrape.sh fix(tradein): убрать require_admin полностью — Caddy basic_auth достаточен (#474) 2026-05-23 14:21:01 +00:00
import-cadastre.sh feat(tradein): кадастр зданий как источник house_metadata (#393) 2026-05-22 10:54:42 +05:00
import-rosreestr.sh feat(tradein): импорт реальных сделок Росреестра (#4) 2026-05-21 19:57:20 +03:00