fix(sql): EMISS PK includes period_type so yearly+Q1 coexist (#1606 follow-up) #1706
No reviewers
Labels
No labels
admin
analytics
auth
automation
bug
business
chore
ci
compliance
data
data-moat
docs
duplicate
dx
enhancement
Fable 5 ревью
feedback/max
generative
GG-форсайт
needs-discussion
needs-human
observability
pause-bots
performance
priority/p0
priority/p1
priority/p2
priority/p3
scope/backend
scope/db
scope/devops
scope/frontend
scope/qa
scrapers
security
site-finder
stage/1
stage/2
status/blocked
status/done
status/needs-analysis
status/needs-fix
status/qa
status/ready
status/review
status/wip
tech-debt
tradein
ux
week ревью 1
wontfix
вторичка
ИРД
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: lekss361/gendesign#1706
Loading…
Add table
Reference in a new issue
No description provided.
Delete branch "fix/emiss-pk-granularity-1687"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Проблема (ревью #1687, решение владельца)
#1606 починил in-memory dedup, но DB-PK
(indicator_type, region, obs_date)всё равно коллидировал год/Q1 наON CONFLICT(оба month=1) → один перетирал другой.Решение
Миграция
163_emiss_pk_period_type.sql:ADD COLUMN period_type TEXT NOT NULL DEFAULT 'unknown', backfill emiss→'quarterly', PK →(indicator_type, region, obs_date, period_type), индекс пересоздан.rosstat_emiss.pyEmissRow.period_typeиз granularity; upsert-SQL (rosstat_macro_sync.py,cbr_macro_sync.py) пишут period_type + новый conflict target.Verify (live DB через MCP)
PK подтверждён на 4 колонки; year+quarter строки для одной region/date сосуществуют; ON CONFLICT обновляет только свою granularity.
pytest(emiss/rosstat) → 49 passed, 1 skipped.⚠️ Миграция меняет PK на проде (auto-apply)
Существующие схлопнутые строки backfill'ятся в 'quarterly'; годовые перезаполнятся при следующем scrape.
Adds period_type TEXT NOT NULL DEFAULT 'unknown' to macro_indicator and widens the PRIMARY KEY to (indicator_type, region, obs_date, period_type). Before: yearly aggregate ('год'→obs_date YYYY-01-01) and Q1 ('I квартал'→ same date) shared the same PK slot → ON CONFLICT DO UPDATE made them overwrite each other despite the in-memory dedup fix in #1687. After: each EmissRow carries period_type from _emiss_period_granularity ('year'/'quarter'/'month'); DB ON CONFLICT targets include it, so yearly+Q1 rows genuinely coexist. Non-EMISS sources (CBR, rosstat open-data, domrf) use period_type='unknown' (literal in SQL) — they are disambiguated by obs_date already, so the wider PK is backward-compatible. All ON CONFLICT clauses in cbr_macro_sync.py and rosstat_macro_sync.py updated accordingly. Migration: 163_emiss_pk_period_type.sql (idempotent BEGIN/COMMIT). Tests: 49 passed (emiss + rosstat suite), ruff clean, py_compile OK.