lekss361
2914ff2ef2
Stage 4d: full city sweep ЕКБ (5 anchors × N pages + enrichment + cooperative cancel). - avito.py fetch_around(pages=N) — pagination backward compat default=1 - scrape_pipeline.run_avito_city_sweep — anchors loop + heartbeat + cancel check - scrape_runs.py utility (create/heartbeat/done/failed/cancelled/list_recent) - admin.py 3 endpoints: start (BackgroundTasks) / runs / cancel - migration 051 ADD COLUMN IF NOT EXISTS + DROP/ADD CHECK constraint (text+CHECK, не enum — safe в transaction) - deploy/avito-city-sweep.sh с random delay 0-3h (anti-pattern) - 16 offline tests Deep review approved: migration idempotent, cancel priority correct (mark_done guarded by status='running'), no SQL injection, psycopg v3 compliant. |
||
|---|---|---|
| .. | ||
| exporters | ||
| matching | ||
| scrapers | ||
| __init__.py | ||
| brand.py | ||
| cian_session.py | ||
| estimator.py | ||
| geocoder.py | ||
| house_metadata.py | ||
| scrape_pipeline.py | ||
| scrape_runs.py | ||