gendesign/backend/app/services/scrapers
Light1YT ea42d29a4a feat(site-finder): live CBR key-rate scraper → macro_indicator (#945 PR B)
Site Finder v2 / GG-форсайт. Deterministic, no LLM. Fills the key_rate series
(empty after PR A) into macro_indicator from the live CBR source.

- services/scrapers/cbr_macro.py: fetch_key_rate via CBR SOAP DailyInfoWebServ
  KeyRateXML(fromDate, ToDate) (POST soap+xml; ToDate capital-T + dateTime
  suffix required — flat GET returns empty). Pure parse_key_rate_xml (TZ-safe
  date, empty/malformed → []). Live-verified: 85 rows, current rate 14.5%.
- workers/tasks/cbr_macro_sync.py: upsert (key_rate, rf, daily, %) into
  macro_indicator ON CONFLICT DO UPDATE; SAVEPOINT per row; fetch-fail → raise.
- beat: cbr-macro-sync-weekly (Mon 05:30 МСК, Europe/Moscow tz).
- 15 tests (pure parse + mocked upsert). No migration (table from #963 PR A).
2026-06-03 00:05:25 +05:00
..
__init__.py add interactive parser of наш.дом.рф ЖК+flats with stealth Playwright 2026-04-27 17:58:40 +03:00
cbr_macro.py feat(site-finder): live CBR key-rate scraper → macro_indicator (#945 PR B) 2026-06-03 00:05:25 +05:00
documents.py feat(22i): domrf_kn_documents new table + parser (#305) 2026-05-17 15:21:31 +00:00
domrf_catalog.py feat(22d): domrf_catalog.py scraper for per-flat price/status from SSR HTML (#309) 2026-05-17 15:40:01 +00:00
domrf_catalog_object.py fix(scrapers): DOM.РФ KN extras — bypass WAF + update renamed endpoints (#503) 2026-05-24 10:53:28 +00:00
domrf_kn.py fix(tradein): backfill kn obj_class (parser + 3-pass) (#572) (#628) 2026-05-28 15:04:32 +00:00
ekburg_permits.py fix(#242): ekburg_permits SSL — verify=False для CA Минцифры РФ 2026-05-17 08:57:42 +03:00
flat_plans.py feat(22c): domrf_kn_flat_plans new table + parser stub (#308) 2026-05-17 15:39:34 +00:00
nspd_client.py fix(#231): repair test_rate_limit_semaphore regression + docstring sync 2026-05-17 10:22:16 +03:00
nspd_denorm.py feat(#126): Sub-PR A — NSPDClient.get_features_in_bbox_grid + engineering classifier (#247) 2026-05-17 05:54:30 +00:00
nspd_lite.py fix(nspd-geo): rosreestr2coord v5 compat — delay arg, tmp cache, dumps default 2026-05-11 14:43:44 +03:00
obj_checks.py feat(scrape-kn): parallel Phase B/C + wire obj_checks + documents + is_ekb auto-derive (#322) 2026-05-17 19:40:16 +00:00
objective.py fix(scrape-objective): stream-parse lots_pf via ijson to avoid OOM (#315) 2026-05-17 16:37:30 +00:00
stealth.py fix(scrapers): DOM.РФ KN extras — bypass WAF + update renamed endpoints (#503) 2026-05-24 10:53:28 +00:00