3 commits

Author	SHA1	Message	Date
lekss361	ee87575052	fix(tradein/avito): anti-bot hardening — sleep + abort + shared session ... Production был заблокирован 23.05 17:48-17:55 MSK из-за tight loop без delay в detail enrichment (~20 req/sec к Avito). Все 3 подряд runs получили 403. Bundled fix: 1. AvitoBlockedError / AvitoRateLimitedError (новый avito_exceptions.py) 2. fetch_around, fetch_detail, fetch_house_catalog — raise на HTTP 403/429 3. scrape_pipeline: shared AsyncSession через весь sweep (один TLS handshake) 4. asyncio.sleep с +-20% random jitter между detail requests (default 7s) 5. Early abort: после 3 consecutive blocks — abort sweep + propagate 6. scrape_runs.mark_rate_limited (status=rate_limited vs failed/done) 7. detail_top_n default 20->10 в CitySweepStartRequest 8. logger.error вместо warning для 403/429 -> GlitchTip alert Confirmed root cause: prod logs показали 403 в 40-60ms intervals (blast pattern без delay между fetch_detail calls).	2026-05-23 23:00:23 +03:00
lekss361	2914ff2ef2	feat(tradein): city sweep — auto ЕКБ pipeline (#477) ... Stage 4d: full city sweep ЕКБ (5 anchors × N pages + enrichment + cooperative cancel). - avito.py fetch_around(pages=N) — pagination backward compat default=1 - scrape_pipeline.run_avito_city_sweep — anchors loop + heartbeat + cancel check - scrape_runs.py utility (create/heartbeat/done/failed/cancelled/list_recent) - admin.py 3 endpoints: start (BackgroundTasks) / runs / cancel - migration 051 ADD COLUMN IF NOT EXISTS + DROP/ADD CHECK constraint (text+CHECK, не enum — safe в transaction) - deploy/avito-city-sweep.sh с random delay 0-3h (anti-pattern) - 16 offline tests Deep review approved: migration idempotent, cancel priority correct (mark_done guarded by status='running'), no SQL injection, psycopg v3 compliant.	2026-05-23 14:38:07 +00:00
lekss361	eb4764f75a	feat(tradein): scrape_pipeline.py — Avito full orchestrator (search → houses → detail) (#451)	2026-05-23 12:55:13 +00:00