gendesign/tradein-mvp/backend/app/services
lekss361 ee87575052 fix(tradein/avito): anti-bot hardening — sleep + abort + shared session
Production был заблокирован 23.05 17:48-17:55 MSK из-за tight loop без delay
в detail enrichment (~20 req/sec к Avito). Все 3 подряд runs получили 403.

Bundled fix:
1. AvitoBlockedError / AvitoRateLimitedError (новый avito_exceptions.py)
2. fetch_around, fetch_detail, fetch_house_catalog — raise на HTTP 403/429
3. scrape_pipeline: shared AsyncSession через весь sweep (один TLS handshake)
4. asyncio.sleep с +-20% random jitter между detail requests (default 7s)
5. Early abort: после 3 consecutive blocks — abort sweep + propagate
6. scrape_runs.mark_rate_limited (status=rate_limited vs failed/done)
7. detail_top_n default 20->10 в CitySweepStartRequest
8. logger.error вместо warning для 403/429 -> GlitchTip alert

Confirmed root cause: prod logs показали 403 в 40-60ms intervals
(blast pattern без delay между fetch_detail calls).
2026-05-23 23:00:23 +03:00
..
exporters fix(tradein): security hotfix — C-3 config default + C-4 empty estimate persist + C-6 PDF URL allowlist (#435) 2026-05-23 09:38:23 +00:00
matching feat(tradein): cross-source matching service (3-tier: cadastr / fingerprint / geo / composite) (#470) 2026-05-23 14:12:16 +00:00
scrapers fix(tradein/avito): anti-bot hardening — sleep + abort + shared session 2026-05-23 23:00:23 +03:00
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
brand.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
cache.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00
cian_session.py fix(tradein): CIAN_REQUIRED_COOKIES — add real auth cookies (DMIR_AUTH + Cian session) (#480) 2026-05-23 14:52:24 +00:00
estimator.py feat(tradein): estimator — Cian Valuation as on-demand 7th source (Stage 9) (#476) 2026-05-23 14:29:52 +00:00
geocoder.py fix(tradein): map-picker подставляет уличный адрес, а не название POI 2026-05-22 13:32:39 +05:00
house_metadata.py feat(tradein): кадастр зданий как источник house_metadata (#393) 2026-05-22 10:54:42 +05:00
scheduler.py feat(tradein): in-app scheduler — UI-managed schedule (replaces SSH crontab) (#482) 2026-05-23 14:54:25 +00:00
scrape_pipeline.py fix(tradein/avito): anti-bot hardening — sleep + abort + shared session 2026-05-23 23:00:23 +03:00
scrape_runs.py fix(tradein/avito): anti-bot hardening — sleep + abort + shared session 2026-05-23 23:00:23 +03:00
scraper_settings.py feat(tradein): global scraper delay setting (applies across all scrapers) (#485) 2026-05-23 15:43:28 +00:00
search_query.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00