Adds period_type TEXT NOT NULL DEFAULT 'unknown' to macro_indicator and
widens the PRIMARY KEY to (indicator_type, region, obs_date, period_type).
Before: yearly aggregate ('год'→obs_date YYYY-01-01) and Q1 ('I квартал'→
same date) shared the same PK slot → ON CONFLICT DO UPDATE made them
overwrite each other despite the in-memory dedup fix in #1687.
After: each EmissRow carries period_type from _emiss_period_granularity
('year'/'quarter'/'month'); DB ON CONFLICT targets include it, so
yearly+Q1 rows genuinely coexist.
Non-EMISS sources (CBR, rosstat open-data, domrf) use period_type='unknown'
(literal in SQL) — they are disambiguated by obs_date already, so the
wider PK is backward-compatible. All ON CONFLICT clauses in
cbr_macro_sync.py and rosstat_macro_sync.py updated accordingly.
Migration: 163_emiss_pk_period_type.sql (idempotent BEGIN/COMMIT).
Tests: 49 passed (emiss + rosstat suite), ruff clean, py_compile OK.
_YEAR_PERIODS ('год') → month=1 collided with Q1 ('I квартал') → month=1
for the same region/year: both produced obs_date=YYYY-01-01, so the second
series in the XML silently overwrote the first in the in-memory by_key dict.
Fix: add _emiss_period_granularity() and extend the dedup key from
(region, obs_date) to (region, obs_date, granularity). 'year' and 'quarter'
are now distinct slots, so both rows survive the parser and reach the upsert.
Test: test_yearly_and_q1_both_survive_dedup verifies len==2 with both
values present; test_period_granularity covers all classification branches.
Closes the last #946 gap (СМР). Fetches «Индексы цен производителей на
строительную продукцию по РФ» from the OPEN Rosstat xlsx
(rosstat.gov.ru/storage/mediabank/Invest_ind_stroitel_MM-YYYY.xlsx),
parses the monthly «к предыдущему месяцу» section → macro_indicator
(indicator_type='construction_price_index', region='rf', monthly, %).
The reCAPTCHA-gated fedstat dataGrid.do path is DELIBERATELY NOT used —
the open Rosstat source needs no captcha. EMISS id=31108 documented as a
prod-only SDMX alternative (not wired).
URL is date-derived with bounded look-back (filename changes monthly;
_fetch_construction_latest walks current month back to 6, first 200 wins;
year-boundary handled, exhaustion raises, 404 breaks fast — no hang).
Wired into rosstat_macro_sync with per-source try/except (can't break
demography/income) + SAVEPOINT-per-row. None-not-0 on blank months.
33 tests (incl look-back + year-boundary + graceful), ruff clean.
Note: no construction-cost channel in macro_coefficient yet (demand-side
only) — series is ingested/available for future cost-side use + display.
Refs #946
fedstat ИПЦ is reCAPTCHA-blocked; CBR publishes inflation openly. Add
fetch_inflation + parse_inflation_xlsx (CBR UniDbQuery DownloadExcel/132934,
monthly % г/г, region=rf, source=cbr) to cbr_macro.py; upsert
indicator_type=inflation_yoy via the existing cbr_macro_sync task (per-series
guard, SAVEPOINT-per-row, CAST not ::, ON CONFLICT on the PK).
Surface inflation_yoy in MonthlyMacro (frozen, carry-forward) and ACTIVATE the
reserved §9.5 inflation channel (macro_coefficient f_inflation: level-vs-4%-target
nudge, non-positive to avoid double-counting f_rate, excluded from
_RATE_DRIVEN_FACTORS). Channel was DEGRADED (no data) -> now BACKED + consumed;
_CONF_HIGH_MIN_BACKED 4->5. Deterministic (§16/§26); renorm claims the reserved
0.08 slice as designed. Live-verified (2026-04 5.58%); 194 macro + 902 forecasting
tests green. No migration, no new deps.
Refs #946.
Re-review feedback (PR #260 head 8191a85):
1. tests/scrapers/test_nspd_bulk_client.py:360 — test referenced bulk_mod._SEMAPHORE
which was renamed to _SEMAPHORE_LIMIT in previous commit. AttributeError на
полном pytest run. Заменено: assert bulk_mod._SEMAPHORE_LIMIT == 3 (smoke).
max_concurrent <= 3 уже валидирует throttling — capacity check был лишним.
2. app/services/scrapers/nspd_client.py docstrings (lines 483, 506) — обновлены
ссылки NSPDBulkClient._SEMAPHORE(3) → NSPDBulkClient._sem (per-instance,
capacity=3) чтобы соответствовать новой архитектуре.
* fix(cadastre): NSPD response parse — properties.category + top-level meta (#168)
* fix(cadastre): test fixtures use real NSPD shape — properties.category + top-level meta (#168)
Bot review of #177 flagged that fixtures still used legacy properties.categoryId
+ data.meta — meaning CI exercised only fallback branches, not primary paths
where the actual bug lived. Pattern of 3 consecutive "live-only" parse fixes
(#175 verify, #176 headers, #177 parse) confirmed need for test coverage.
Changes:
- test_nspd_bulk_client.py: sample_quarter_response → properties.category +
meta moved to top-level (real NSPD shape verified live)
- test_cadastre_bulk.py: 7 fixtures categoryId → category (regex replace)
- test_nspd_bulk_feature_parse_basic: primary path now exercised
Plus schema hardening per bot review:
- NSPDBulkFeature.category_id: `is not None` check (not truthy `or`)
to avoid edge case category=0; int() wrapped in try/except so
non-numeric ID (e.g. "ЗУ") doesn't crash upsert_features loop.
35/35 tests pass locally.
---------
Co-authored-by: lekss361 <claudestars@proton.me>
