9dbfb03a3d
fix(scrapers): disambiguate EMISS yearly vs Q1 in dedup key ( #1606 )
...
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m38s
CI / backend-tests (pull_request) Failing after 8m41s
_YEAR_PERIODS ('год') → month=1 collided with Q1 ('I квартал') → month=1
for the same region/year: both produced obs_date=YYYY-01-01, so the second
series in the XML silently overwrote the first in the in-memory by_key dict.
Fix: add _emiss_period_granularity() and extend the dedup key from
(region, obs_date) to (region, obs_date, granularity). 'year' and 'quarter'
are now distinct slots, so both rows survive the parser and reach the upsert.
Test: test_yearly_and_q1_both_survive_dedup verifies len==2 with both
values present; test_period_granularity covers all classification branches.
2026-06-17 20:39:52 +03:00
1f66dfd400
feat(macro): ЕМИСС среднедушевые доходы (id=57039) -> macro_indicator ( #946 part2)
...
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
Deploy / changes (push) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
Deploy / build-backend (push) Successful in 1m31s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m33s
Deploy / deploy (push) Successful in 1m12s
CI / backend-tests (push) Successful in 6m23s
CI / backend-tests (pull_request) Successful in 6m31s
Расширяет Росстат-скрейпер ЕМИСС/fedstat-рядом «Среднедушевые денежные доходы
населения» (fedstat id=57039, OKATO 65=Свердл, квартальный, руб). Добавляет pure
SDMX-ML v1.0 GenericData-парсер (stdlib ElementTree, dev-тестируем на фикстурах) +
EMISS-ветку Celery-таски rosstat_macro_sync (open-data + ЕМИСС, per-source guard,
source='emiss', frequency параметризован, CAST not ::, SAVEPOINT per-row).
income_per_capita проброшен в site_finder/macro (region='sverdl').
ЕМИСС за WAF с dev -> fetch исполняется на проде (verified: httpx POST из
gendesign-worker-1 -> 200 + SDMX, парсер извлёк 9 Свердл-строк). ИПЦ (id=31074) /
индекс цен СМР НЕ landed: многомерные, дефолт-экспорт data.do без Свердл, нужна
dataGrid-выборка (selectedFilterIds + filter-tree AJAX + reCAPTCHA) -- задокументировано.
Refs #946 .
2026-06-07 18:44:38 +05:00
