Commit graph

3 commits

Author SHA1 Message Date
3c00af227e fix(scrapers): rate-limit/backoff/circuit-breaker для ОКН EGRKN-скрапера (#2445-D3)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 2m1s
CI / backend-tests (pull_request) Successful in 14m54s
WAF-гигиена (#2443): okn-mk.mkrf.ru имеет тот же unthrottled-request risk shape,
что забанил VPS в WAF ДОМ.РФ. okn_egrkn_client не имел пейсинга/backoff,
okn_objects_sync шлёт до 1500 последовательных GET без пауз.

Добавлено: _wait_rate_limit() (shared 400ms gate между searchMap+detail на одном
WAF-хосте), 429/5xx экспоненциальный backoff (base 2с, cap 60с, Retry-After
honored) в _do_request/_do_detail_request (recursion bounded _MAX_RETRIES=3),
circuit-breaker _DETAIL_FAIL_BREAKER=20 (consecutive fails, reset on success,
break только detail-цикла). Идиомы из nspd_client/objective/gisogd66.

Refs #2445
2026-07-06 00:50:57 +05:00
c3c6e2cef1 fix(okn): expand registry coverage + load detail-card name/address (#1159)
okn_objects_sync собирал 428 ОКН из ~798 ЕГРОКН и сохранял raw_props={id}
без названия/адреса — analyze показывал 'ОКН рядом' без указания КАКОГО.

Coverage: расширил _ADDRESS_PASSES с 2 до 11 подстрок (Екатеринбург,
г. Екатеринбург, 7 районов ЕКБ, 2 формы Свердловской обл.). _PASSES =
декартово (address × category). first-wins дедуп сохраняет высшую категорию.

Detail: fetch_okn_detail(source_id) — GET Show/Show?id=, BeautifulSoup
парсит HTML (dl/dt/dd и tr/td), извлекает name, address, protection_category,
object_kind, dating. _enrich_with_detail сливает в raw_props с graceful
failover (cap=1500, exception per-object не валит прогон).

Lookup: parcel_okn_objects возвращает name/address из raw_props->>'name'.
Existing ird_analyze тесты не сломаны.

49 okn-related тестов passed, ruff clean.

Closes #1159
2026-06-13 15:02:50 +05:00
50fcba1ca0 feat(sf): ingest ОКН-объектов ЕКБ из АИС ЕГРКН (точки+категория) → okn_objects (#1141)
All checks were successful
Deploy / changes (push) Successful in 5s
Deploy / build-backend (push) Successful in 1m23s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m21s
Deploy / deploy (push) Successful in 1m15s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-06-07 12:51:18 +00:00