lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/tradein-mvp/backend/app/services
History
bot-backend dc7a84cf2e feat(tradein): per-anchor watchdog timeout in city sweeps (#880) 
run_{avito,cian,yandex}_city_sweep had no per-anchor watchdog: a stalled HTTP
request (proxy/connection hang) blocked the asyncio task forever — heartbeat
froze mid-anchor, run stayed 'running' until the 6h zombie-detect. Live-repro:
cian sweep #24 hung on anchor 3, #29 on anchor 1.

Wrap each anchor's processing in asyncio.wait_for(coro, ANCHOR_TIMEOUT_SEC=240).
TimeoutError -> logger.warning + errors_count++ + continue (sweep finishes,
reaches mark_done). Timeout branch sits before the existing except Exception.
yandex/cian also bump consecutive_failures -> existing MAX_CONSECUTIVE_FAILURES
abort fires on 3 consecutive hung anchors. CianScraper SERP session already
has timeout=30. 5 unit tests (all 3 sweeps), network-free.

Refs #880, #581, #559
2026-05-31 13:33:39 +03:00
..
exporters fix(pdf): brand-spoof + median=0 empty-state + SSRF + дисклеймер + sources (Refs #772) (#789) 2026-05-30 16:55:56 +00:00
matching fix(matching): Tier 2b alias sync + collapse double SAVEPOINT (#849) (#870) 2026-05-31 08:05:48 +00:00
scrapers fix(tradein): avito house-link из detail SSR — SERP-карточка JS-only (#871) (#873) 2026-05-31 08:27:47 +00:00
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
account_quota.py fix(tradein): atomic conditional quota increment — close TOCTOU (#747) (#749) 2026-05-30 15:52:34 +00:00
brand.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
cache.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00
cian_price_history.py feat(tradein): admin backfill endpoints — cian price-history (T8a) + yandex address (T10) + house IMV (T7) (#856) 2026-05-31 06:37:10 +00:00
cian_session.py feat(scrapers): route cian sessions through mobile proxy (Refs #806) (#818) 2026-05-30 19:10:36 +00:00
dadata.py feat(tradein): DaData /suggest tier в /suggest endpoint (PR Q2) 2026-05-27 19:08:58 +05:00
estimator.py fix(estimator): sanity-clamp quarter-index factor (#859) (#863) 2026-05-31 07:20:59 +00:00
geocoder.py feat(tradein): DaData /suggest tier в /suggest endpoint (PR Q2) 2026-05-27 19:08:58 +05:00
house_imv_backfill.py feat(tradein): avito city-sweep — IMV-фаза (один прогон = SERP+detail+houses+IMV) (#861) 2026-05-31 07:09:42 +00:00
house_metadata.py feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) 2026-05-24 08:57:30 +00:00
image_sanitizer.py fix(tradein-photos): sanitize uploaded images via Pillow re-encode (#511) 2026-05-24 11:32:04 +00:00
scheduler.py feat(tradein): yandex city-sweep — один прогон SERP+address-enrich (#867) 2026-05-31 07:43:13 +00:00
scrape_pipeline.py feat(tradein): per-anchor watchdog timeout in city sweeps (#880) 2026-05-31 13:33:39 +03:00
scrape_runs.py feat(observability): sentry alert on N consecutive scraper failures (Refs #827) (#833) 2026-05-30 20:10:29 +00:00
scraper_settings.py feat(tradein): global scraper delay setting (applies across all scrapers) (#485) 2026-05-23 15:43:28 +00:00
search_query.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00
yandex_address_backfill.py fix(tradein): yandex address-enrich — city-agnostic title regex (спутники ЕКБ) (#875) 2026-05-31 09:08:29 +00:00