5 commits

Author	SHA1	Message	Date
bot-backend	8204802d36	feat(scrapers): wire N1.ru city sweep — pipeline + scheduler + admin endpoint (#575) ... n1.py scraper existed and was wired into the generic point-/scrape endpoint, but had no city-sweep + schedule entry. Mirror the yandex sweep (N1 is a simple regional SERP — search → save, no houses/detail enrich): - scrape_pipeline.run_n1_city_sweep + N1CitySweepCounters (anchors × pages → save_listings; cooperative cancel; abort after N consecutive anchor failures). - scheduler.trigger_n1_city_sweep_run + dispatch (source='n1_city_sweep'). - data/sql/084: scrape_schedules seed, enabled=true (vs dormant avito/yandex) — #575 activates N1 as a source; one-line UPDATE to disable if it blocks. - POST /api/v1/admin/scrape/n1 (+ /scrape/n1/runs) for on-demand city-sweep. estimator.sources_used is derived from {lot.source} — 'n1' flows in automatically once n1 listings exist (no estimator change). Tests: test_n1_city_sweep.py (5) — aggregate, run_id threading, empty-page stop, mid-sweep cancel, consecutive-failure abort. ruff clean; 32 scheduler/sweep regression tests pass. Refs #575	2026-05-30 18:39:36 +03:00
Light1YT	7ec19b3536	feat(scheduler): dormant Yandex Realty city-sweep (#561) (#642)	2026-05-29 08:50:10 +00:00
lekss361	222389cba9	fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487)	2026-05-23 20:08:40 +00:00
lekss361	2914ff2ef2	feat(tradein): city sweep — auto ЕКБ pipeline (#477) ... Stage 4d: full city sweep ЕКБ (5 anchors × N pages + enrichment + cooperative cancel). - avito.py fetch_around(pages=N) — pagination backward compat default=1 - scrape_pipeline.run_avito_city_sweep — anchors loop + heartbeat + cancel check - scrape_runs.py utility (create/heartbeat/done/failed/cancelled/list_recent) - admin.py 3 endpoints: start (BackgroundTasks) / runs / cancel - migration 051 ADD COLUMN IF NOT EXISTS + DROP/ADD CHECK constraint (text+CHECK, не enum — safe в transaction) - deploy/avito-city-sweep.sh с random delay 0-3h (anti-pattern) - 16 offline tests Deep review approved: migration idempotent, cancel priority correct (mark_done guarded by status='running'), no SQL injection, psycopg v3 compliant.	2026-05-23 14:38:07 +00:00
lekss361	eb4764f75a	feat(tradein): scrape_pipeline.py — Avito full orchestrator (search → houses → detail) (#451)	2026-05-23 12:55:13 +00:00