gendesign/backend/app/services/scrapers
lekss361 1db19b3b8a feat(scrape-kn): parallel Phase B/C + wire obj_checks + documents + is_ekb auto-derive
4 changes:
1. asyncio.gather() в Phase B/C — 6 endpoints/obj параллельно
   через _fetch_*_safe wrappers. ~6x speedup (3.5h -> ~35min).
   BrowserSession._sem(3) bounds concurrent requests автоматически.
2. Wire documents extractor (services/scrapers/documents.py) —
   per-object call в parallel gather. Заполняет domrf_kn_documents
   (был пуст после PR #305 schema-only).
3. Create services/scrapers/obj_checks.py extractor + wire в loop.
   Заполняет domrf_obj_checks (был пуст после PR #303 schema-only).
   Endpoint URL /checks не верифицирован devtools — выведен по
   паттерну /infrastructure; при ошибке пишет в kn_scrape_failures.
4. Phase E (после Phase D) — UPDATE is_ekb=TRUE для свежего snapshot
   по district_name ILIKE екатеринбург/свердловск. Best-effort
   (exception caught, scrape не прерывается).
2026-05-17 22:34:58 +03:00
..
__init__.py add interactive parser of наш.дом.рф ЖК+flats with stealth Playwright 2026-04-27 17:58:40 +03:00
documents.py feat(22i): domrf_kn_documents new table + parser (#305) 2026-05-17 15:21:31 +00:00
domrf_catalog.py feat(22d): domrf_catalog.py scraper for per-flat price/status from SSR HTML (#309) 2026-05-17 15:40:01 +00:00
domrf_kn.py feat(scrape-kn): parallel Phase B/C + wire obj_checks + documents + is_ekb auto-derive 2026-05-17 22:34:58 +03:00
ekburg_permits.py fix(#242): ekburg_permits SSL — verify=False для CA Минцифры РФ 2026-05-17 08:57:42 +03:00
flat_plans.py feat(22c): domrf_kn_flat_plans new table + parser stub (#308) 2026-05-17 15:39:34 +00:00
nspd_client.py fix(#231): repair test_rate_limit_semaphore regression + docstring sync 2026-05-17 10:22:16 +03:00
nspd_denorm.py feat(#126): Sub-PR A — NSPDClient.get_features_in_bbox_grid + engineering classifier (#247) 2026-05-17 05:54:30 +00:00
nspd_lite.py fix(nspd-geo): rosreestr2coord v5 compat — delay arg, tmp cache, dumps default 2026-05-11 14:43:44 +03:00
obj_checks.py feat(scrape-kn): parallel Phase B/C + wire obj_checks + documents + is_ekb auto-derive 2026-05-17 22:34:58 +03:00
objective.py fix(scrape-objective): stream-parse lots_pf via ijson to avoid OOM (#315) 2026-05-17 16:37:30 +00:00
stealth.py add domrf extras parser (sale_graph + sales_agg + infra + photos) + 2026-04-27 18:35:46 +03:00