add anal 2

2026-04-27 13:05:36 +03:00 · 2026-04-27 13:05:36 +03:00 · 16481868a6
commit 16481868a6
parent c13cbaef2e
18 changed files with 2493 additions and 23 deletions
--- a/.gitignore
+++ b/.gitignore
@ -13,6 +13,7 @@ node_modules/
 .env
 .env.local
 .env.*.local
+.mcp.json

 # IDE
 .vscode/
--- a/README.md
+++ b/README.md
@ -10,7 +10,7 @@ Acceptance criteria разбиты на 3 блока (функционально

 ---

-## Текущий статус (26 апреля 2026)
+## Текущий статус (27 апреля 2026)

 **Инфраструктура: live на HTTPS.** Прод доступен по [https://gendsgn.ru/](https://gendsgn.ru/), Let's Encrypt cert получен. Полный цикл автодеплоя из GitHub Actions работает. 26.04 прод полностью пересоздан (`down -v` + redeploy) — БД свежая, backend в multi-stage образе под non-root пользователем, healthcheck-и работают, ловит ошибки в Sentry.

@ -46,6 +46,251 @@ Acceptance criteria разбиты на 3 блока (функционально
 2. **ДОМ.РФ фронт-API** (`DataSources_Analysis_Apr26` + 12 вкладок) — открытые JSON без auth (либо `Basic MTpqd2U=` хардкоден), per-flat прайс-лист, реализация/готовность как velocity.
 3. **НСПД MarketAnalytics** (`NSPD_MarketAnalytics_FullArchitecture_Apr26`) — индикаторы цен по кадастровым кварталам, 8 лет longitudinal, GeoJSON-полигоны для heatmap. Архитектура полностью раскрыта в debug-браузере (saveUserFilters → graphEgrn*/tableEgrn/geomDataForHeatMap, серверная сессия фильтров через userFiltersId). ⚠ `graphEgrnCandleMedian`/`LineMedian` (абсолютные ₽/м²) дают 403 для physic-аккаунта — но эти данные мы уже получаем из CSV-датасета Росреестра.

+**27.04 — analytical layer полностью построен.** В прод-БД 43 `domrf_*/cbr_*/yandex_*/ekb_*/sv_*/rosreestr_*` таблицы (см. блок «Аналитический пласт данных» ниже). За день:
+- Загружено 5 lazy-load страниц ДОМ.РФ (`DomRF_Lazy_Loaded_Apr27_2026`) — `domrf_realization` 9.7K + stat_series XLSX 809K observations + 333 raw JSONB endpoints.
+- Догружена история 14 месяцев (Янв 2025 - Февр 2026) для realization + sold_ready через `repYear/repMonth` параметры (`DomRF_History_Loaded_Apr27_2026`) — `domrf_realization` итого **143 963** рядов, sold_ready_breakdown расширен до 7 chart-types с типизированными колонками.
+- Импортирован SQLite Антона (23 МБ) — 13 новых таблиц: `domrf_kn_flats` (27 563 поквартирно), `domrf_kn_objects` (1 516 ЖК), `cbr_mortgage_series` (3 308 ЦБ), `sber_ddf_metrics`, `yandex_realty_zk`, `ekb_districts`, `rosreestr_rent_deals`. Архив девелоперов смерджен (2 734 → 8 629). См. `Anton_Imported_Apr27_2026`.
+- Подключён MCP `crystaldba/postgres-mcp` (`.mcp.json`) — все DDL/SELECT теперь через `mcp__postgres__execute_sql` без `docker run psql` обёртки.
+- FK-аудит и cleanup: добавлены 4 missing FK, дропнуты 2 дублирующих индекса и 2 устаревшие таблицы (`sold_out_by_year`, `sold_out_by_progress` ⊂ `domrf_realization`), 8 `COMMENT ON TABLE` для документации источников.
+- Territory aliases для stat_series ↔ regions name mapping (`v_stat_series_resolved` view): 100% покрытие 809K observations через 101 alias.
+- Готовы 2 стратегических отчёта: `Sverdl_Quartirography_Report_Apr27` (общий по Свердл) + `PRINZIP_Strategy_Apr27` (что строить и куда смотреть).
+
+---
+
+## Аналитический пласт данных (data warehouse) — 27.04.2026
+
+В прод-БД `gendesign` живёт **43 предметных таблицы** + 5 helper views + Rosreestr partitions. Размер БД ~2.1 ГБ.
+
+### 🔌 Подключение
+
+| Метод | Команда |
+|---|---|
+| **MCP** (рекомендуется) | конфиг в `.mcp.json` (project root) — `mcp__postgres__execute_sql`, `analyze_db_health`, `list_objects`, `explain_query` и т.д. |
+| **psql via SSH-туннель** | `ssh -fN -L 15432:127.0.0.1:5432 gendesign` затем `PGPASSWORD=… psql -h 127.0.0.1 -p 15432 -U gendesign gendesign` |
+| **DBeaver / TablePlus** | localhost:15432 после открытия SSH-туннеля |
+
+⚠️ Туннель уже открыт на машинах команды (gendesign@46.173.16.127). `.mcp.json` в `.gitignore` (содержит пароль).
+
+### 📊 Карта таблиц (43 предметных)
+
+**ROSREESTR per-deal (1.1 ГБ)**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `rosreestr_deals` (+ 7 партиций по кварталам 2024Q3-2026Q1) | 4 975 398 | Сырые сделки ДКП+ДДУ из ФГИС ЕГРН, 89 регионов |
+| `rosreestr_deals_staging` | UNLOGGED | буфер для `\copy` из CSV |
+| `rosreestr_rent_deals` | 1 071 | Договоры аренды Свердл Q1 2026 (Антон) |
+
+**ROSREESTR materialized views (305 МБ)**
+| MV | Рядов | Описание |
+|---|---:|---|
+| `rr_agg_cad_quarter` | 952 799 | (region, cad_quarter, period, doc_type, area_bucket) → p10/median/p90 ₽/м² |
+| `rr_agg_settlement` | 422 852 | (region, settlement, period, doc_type, realestate_type) |
+| `rr_agg_region` | 3 549 | percentile-агрегаты по регионам |
+
+**DOM.РФ справочники / dimensions**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_regions` | 90 | PK region_id (0=РФ + 89 субъектов). FK target для 4 таблиц |
+| `domrf_developers` | **8 629** | реестр + архив (после merge с Антоном). PK developer_id ('6208_0'=PRINZIP) |
+| `domrf_developers_full` | 2 563 | XLSX-выгрузка с метриками area/permits/flats/share |
+| `domrf_snapshots` | 18 | snapshot_date dimension (4 базовых + 14 history Янв 2025-Февр 2026) |
+| `domrf_federal_districts` | 8 | 8 ФО как first-class lookup |
+| `domrf_territory_aliases` | 101 | text→(level, region_id\|fo_id) для нормализации имён |
+| `domrf_stat_series_indicators` | 123 | XLSX sheet codes → indicator_name |
+
+**DOM.РФ realization & sold_ready (history)**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_realization` | **143 963** | 4 endpoint_types (total/housing/ready_year/developer) × 89 регионов × 14 monthly snapshots |
+| `domrf_sold_ready_index` | 15 | РФ-итог helio метрики per snapshot |
+| `domrf_sold_ready_breakdown` | 40 044 | 7 chart_types × dim_value × 14 snapshots (regionChart/devChart/foChart/cityPopulationChart/classChart/devValueChart/readyYearChart) с типизированными колонками |
+| `domrf_sold_ready_dynamics` | 3 450 | timeseries 2020-period × 4 chart_types |
+
+**DOM.РФ JSON dashboard**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_region_aggregates` | 439 | (region, room_count_type) → flat_count/area/percent |
+| `domrf_developer_aggregates` | 11 063 | (developer × region × room_count) |
+| `domrf_flat_area_distribution` | 40 | РФ-уровень бакеты площади |
+
+**DOM.РФ XLSX/PDF снапшоты**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_developers_full` | 2 563 | реестр девелоперов из XLSX |
+| `domrf_escrow_banks` | 28 | банки эскроу (Сбер 61%) |
+| `domrf_guaranty_regions` | 99 | РФ + 8 ФО + 89 регионов разбивка |
+| `domrf_housing_summary` | 58 | разбивка по механизмам |
+| `domrf_planned_commissioning` | 10 | план ввода 2026-2035 |
+| `domrf_sold_out` | 9 | РФ + 8 топ-регионов (PDF, ради unique `attracted_funds_mln_rub`) |
+| `domrf_mortgage_rates` | 20 | ставки топ-20 банков (PDF) |
+
+**DOM.РФ launch / commissioning**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_launch_top` | 1 410 | топ-N developer/region/area/fo по запускам/вводу |
+| `domrf_launch_obj_class` | 4 | Типовой/Комфорт/Бизнес/Элитный |
+| `domrf_launch_monthly` | 75 | timeseries 2020-2026 |
+| `domrf_commissioning` | 1 | факт ввод (multifamily/private) |
+| `domrf_project_finance` | 99 | ПР финансирование РФ/ФО/регионы |
+
+**DOM.РФ stat_series (parsed XLSX 254 МБ)**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_stat_series_observations` | **809 236** | long-form panel: 99 территорий × 123 индикатора × ~70 месяцев (2020-2026) |
+| `domrf_xlsx_files` | 9 | BYTEA-архив исходных XLSX (8 МБ) |
+
+**DOM.РФ ipoteka / mortgage**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_mortgage_dashboard` | 1 | агрегат рынка ипотеки РФ |
+| `domrf_mortgage_details` | 2 | RUB/OTH валюты |
+
+**DOM.РФ /сервисы/api/kn/* (от Антона)**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_kn_flats` | **27 563** | поквартирный прайс РФ free/booked/sold с ценами |
+| `domrf_kn_objects` | 1 516 | реестр ЖК Свердл (1285 ЕКБ + 28 PRINZIP) с GPS, классом, эскроу |
+| `domrf_dev_growth` | 139 | динамика 2024→2025 девелоперов |
+
+**ЦБ РФ ипотека (от Антона)**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `cbr_mortgage_series` | 3 308 | 47 серий × {rf,sverdl} × Jan 2019 - Feb 2026 |
+| `cbr_mortgage_files` | 19 | XLSX-источники |
+
+**Внешние источники**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `sber_ddf_metrics` | 20 | Сбер ДомКлик DDF nov2025: affordability, demand index, миграция, повторные ипотечники, price gap |
+| `yandex_realty_zk` | 12 | Yandex Realty ЖК ЕКБ snapshot |
+| `yandex_realty_class_prices` | 3 | разбивка по классам econom/comfort/elite |
+
+**Geo Свердл/ЕКБ**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `ekb_districts` | 9 | 9 районов ЕКБ с zk_count/flat_count/area |
+| `ekb_metro` | 10 | станции метро ЕКБ |
+| `sv_cities` | 24 | не-ЕКБ города Свердл (Н.Тагил, Первоуральск, Арамиль) |
+| `sv_dev_sales` | 50 | ТОП-50 Свердл девелоперы Март 2026 |
+
+**Universal raw**
+| Таблица | Рядов | Описание |
+|---|---:|---|
+| `domrf_raw_endpoints` | 349 | universal JSONB store: каждый /api/* response с GIN-index на payload |
+
+### 🔍 Helper views (9 шт)
+
+**Базовые (legacy):**
+| View | Описание |
+|---|---|
+| `v_market_pulse_rf` | one-row live-метрика РФ: 118.7M м² стройка / sold 30.91% / Цена 216 316 ₽/м² / ставка 19.9% vs 6.83% |
+| `v_developer_master` | полный профиль девелопера (legacy). PRINZIP sold% 48.4% — ЛИДЕР Свердл |
+| `v_region_master` | one-row-per-region: квартирография + sold_out + sold_ready + project_finance + guaranty + launch |
+| `v_rosreestr_market_by_region` | Rosreestr p10/median/p90 + DOM.РФ имена |
+| `v_stat_series_resolved` | stat_series_observations + alias + indicator_name (`WHERE region_id=66` → Свердл) |
+
+**Связи квартиры↔ЖК↔застройщики (NEW 27.04 evening):**
+| View | Описание |
+|---|---|
+| `v_jk_with_developer` | ЖК Свердл + canonical dev_name + region_name (через FK `fk_kn_obj_developer/region`). 1 516 строк. |
+| `v_jk_sales_summary` | flats GROUP BY obj_name: total/sold/booked/free + sold_pct + avg_ppm + avg_area + total_value. Standalone-вид поквартирных снапшотов из `domrf_kn_flats`. |
+| `v_developer_portfolio` | Девелопер + все его корпуса/ЖК Свердл: count, flats_total, sqm + фильтры ekb/problem/escrow/completed/in_progress + earliest/latest ready_dt. |
+| `v_developer_full_metrics` | **⭐ ULTIMATE QUERY**: portfolio (ЖК) + квартирография (1к/2к/3к/4+) с процентами + realization Свердл sold%/цена. Лучший view для PRINZIP/Брусника/конкурентов сравнения. |
+
+### 🔗 FK relationships
+
+**35 FK constraints** (после relations v2):
+- `domrf_snapshots`: 24 child FKs (все таблицы со snapshot_date)
+- `domrf_regions`: 5 (developer_aggregates, flat_area_distribution, region_aggregates, realization, kn_objects⭐, rent_deals⭐, territory_aliases)
+- `domrf_developers`: 3 (developer_aggregates, **kn_objects** ⭐ NEW, **sv_dev_sales** ⭐ NEW)
+- `domrf_federal_districts`: 1 (territory_aliases)
+
+**Цепочка анализа продаж:**
+```
+domrf_kn_flats (поквартирно) ─── obj_name ──→ domrf_kn_objects (ЖК)
+                                                    │  dev_id (FK)
+                                                    ▼
+                                                domrf_developers (справочник)
+                                                    │  developer_id (FK)
+                                                    ▼
+                                            domrf_realization (sold%/price 14 мес)
+                                                    │
+                                                    │  region_code=66 ─→ domrf_regions
+```
+
+⚠ **Известное ограничение**: `domrf_kn_flats.obj_id` НЕ имеет FK на `domrf_kn_objects.obj_id` — два API endpoint используют разные ID-системы (27 563/27 563 orphans, текстовый match по `obj_name`/`comm_name` тоже даёт только 2 совпадения). Поквартирные снапшоты обрабатываются standalone через `v_jk_sales_summary`.
+
+### 📜 ETL artefacts (data/sql/)
+
+```
+01_schema_rosreestr_deals.sql       partitioned table + staging
+02_load_all_quarters.sh             \copy CSV (sep ~ и ;)
+03_materialized_views.sql           3 MV для аналитики
+
+10_schema_domrf.sql                 domrf_regions/developers/snapshots/aggregates
+11_load_domrf.py                    JSON dashboard loader
+20_schema_domrf_extras.sql          XLSX/PDF schemas
+21_load_domrf_extras.py             XLSX/PDF loader
+
+30_scrape_domrf.py                  Playwright headless Chromium с per-page direct API triggers
+31_schema_domrf_full.sql            launch/sold_ready/etc normalized
+31_schema_domrf_raw.sql             universal JSONB store
+32_load_domrf_full.py               normalize scraper output
+32_load_domrf_raw.py                JSONB raw store loader
+
+33_schema_domrf_lazy.sql            domrf_realization + xlsx_files (5 lazy pages)
+34_load_domrf_lazy.py               realization/xlsx/raw loader
+35_schema_domrf_stats.sql           stat_series_indicators + observations
+36_parse_xlsx_stats.py              parse 7 XLSX → 809K observations
+37_cleanup_redundant.sql            DROP sold_out_by_year/progress
+
+38_relations_fix.sql                +4 FK, миграция -1→0 RF, drop dup indexes, COMMENT'ы
+39_territory_aliases.sql            federal_districts + territory_aliases + v_stat_series_resolved
+40_relations.sql                    оригинальные FK + 4 helper views (legacy)
+40_scrape_history.py                14-month history scraper для realization+sold_ready
+41_load_history.py                  loader с SNAPSHOT_DATE = end of period
+42_schema_breakdown_widen.sql       drop JSONB extra → 6 typed dimension columns
+43_anton_import.sql                 13 new tables for Anton's SQLite
+44_import_anton_db.py               sqlite3 → postgres migration
+45_relations_v2.sql                 +4 FK (kn_obj→regions/developers, sv_dev_sales→developers, rent_deals→regions) + 4 analytical views (jk_with_developer, jk_sales_summary, developer_portfolio, developer_full_metrics)
+```
+
+### 🎯 Аналитика — что делать с этими данными
+
+**Готовые отчёты в графе памяти:**
+- `Sverdl_Quartirography_Report_Apr27` — полный отчёт по квартирографии Свердл (что строится, что продаётся, тренды 14 мес, топ-15 девелоперов)
+- `PRINZIP_Strategy_Apr27` — стратегические рекомендации для PRINZIP (что строить, куда смотреть, BENCHMARK с Брусникой/Форум-групп)
+
+**Использование (паттерны):**
+```sql
+-- Свердл realization timeseries за год
+SELECT TO_CHAR(snapshot_date,'YYYY-MM') AS p, total_square, sold_perc, price_avg
+FROM domrf_realization
+WHERE region_code=66 AND endpoint_type='total' AND type_square='total'
+ORDER BY snapshot_date;
+
+-- ЕКБ ДДУ-сделки по площадным сегментам
+SELECT
+  CASE WHEN area<30 THEN 'studio' WHEN area<45 THEN '1k' ... END,
+  COUNT(*), AVG(price_per_sqm)
+FROM rosreestr_deals
+WHERE region_code=66 AND city ILIKE '%катеринбург%'
+  AND period_start_date >= '2025-04-01' AND doc_type='ДДУ';
+
+-- Свердл stat_series (МКД 214-ФЗ, временной ряд) через resolved view
+SELECT obs_date, value
+FROM v_stat_series_resolved
+WHERE region_id=66 AND sheet_code='01_01_01';
+
+-- Поквартирный прайс ЕКБ (Антон)
+SELECT obj_name, rooms, total_area, price_per_m2, status
+FROM domrf_kn_flats WHERE city='Город Екатеринбург' AND status='free';
+
+-- Топ ЖК ЕКБ (Антон)
+SELECT comm_name, dev_name, flat_count, square_living, ready_dt
+FROM domrf_kn_objects WHERE is_ekb=true ORDER BY flat_count DESC;
+```
+
 ---

 ## Структура репозитория
@ -325,7 +570,16 @@ git push

 ⚠️ Эти документы — **снимок состояния на момент написания**. Свежее состояние стратегии — в графе памяти. При расхождении доверяем графу.

-Релевантные сущности графа на 26.04.2026:
+Релевантные сущности графа на 27.04.2026:
+
+**Аналитика и отчёты (27.04):**
+- `DomRF_Lazy_Loaded_Apr27_2026` ⭐ — итог дня: 5 lazy-load страниц + stat_series + cleanup + territory aliases + MCP
+- `DomRF_History_Loaded_Apr27_2026` ⭐ — 14 месяцев истории realization+sold_ready (143K рядов realization)
+- `Anton_Imported_Apr27_2026` ⭐ — 13 таблиц от Антона + merge developers (8 629 итого)
+- `Sverdl_Quartirography_Report_Apr27` 📊 — полный отчёт по Свердл квартирографии
+- `PRINZIP_Strategy_Apr27` 📊 — стратегия PRINZIP: что строить и куда смотреть
+
+Сущности 26.04.2026:

 **Стратегия / план:**
 - `Project_GenDesign` — общая шапка
--- a/data/sql/30_scrape_domrf.py
+++ b/data/sql/30_scrape_domrf.py
@ -60,6 +60,330 @@ def upgrade_size(url: str) -> str:
    return urlunsplit((parts.scheme, parts.netloc, parts.path, new_q, parts.fragment))


+# ---------------------------------------------------------------------------
+# Page-specific lazy-load triggers
+# ---------------------------------------------------------------------------
+# Most DOM.RF analytics pages render Material-UI tabs/selects; data fetches
+# fire on tab/option switch. The "kitchen sink" trigger walks every tab and
+# every Select option, with networkidle waits in between. Pages with custom
+# UI (tree pickers, multi-step filters) get dedicated triggers below.
+
+async def _wait_quiet(page, ms=2500):
+    try:
+        await page.wait_for_load_state('networkidle', timeout=8000)
+    except Exception:
+        pass
+    await page.wait_for_timeout(ms)
+
+
+async def _click_all_tabs(page):
+    """Click every Material-UI tab one-by-one, waiting between to capture XHR."""
+    tab_count = await page.evaluate('''() => document.querySelectorAll(
+        '[role="tab"], .MuiTab-root, button[data-tab]'
+    ).length''')
+    print(f'  tabs found: {tab_count}')
+    for i in range(tab_count):
+        try:
+            await page.evaluate(
+                '''(i) => {
+                  const tabs = document.querySelectorAll(
+                    '[role="tab"], .MuiTab-root, button[data-tab]'
+                  );
+                  if (tabs[i]) tabs[i].click();
+                }''',
+                i,
+            )
+            await _wait_quiet(page, 2000)
+        except Exception as e:
+            print(f'  tab[{i}] err: {e}')
+
+
+async def _iterate_selects(page, max_options=20):
+    """Open every MUI Select / native <select>, pick each option, wait."""
+    select_count = await page.evaluate('''() => document.querySelectorAll(
+        '[role="combobox"], [role="button"][aria-haspopup="listbox"], select'
+    ).length''')
+    print(f'  selects found: {select_count}')
+    for i in range(select_count):
+        try:
+            opt_count = await page.evaluate(
+                '''async (i) => {
+                  const sels = document.querySelectorAll(
+                    '[role="combobox"], [role="button"][aria-haspopup="listbox"], select'
+                  );
+                  const sel = sels[i];
+                  if (!sel) return 0;
+                  if (sel.tagName === 'SELECT') return sel.options.length;
+                  sel.click();
+                  await new Promise(r => setTimeout(r, 800));
+                  return document.querySelectorAll('[role="option"], li[data-value]').length;
+                }''',
+                i,
+            )
+            opt_count = min(opt_count, max_options)
+            for j in range(opt_count):
+                try:
+                    await page.evaluate(
+                        '''async ([i, j]) => {
+                          const sels = document.querySelectorAll(
+                            '[role="combobox"], [role="button"][aria-haspopup="listbox"], select'
+                          );
+                          const sel = sels[i];
+                          if (!sel) return;
+                          if (sel.tagName === 'SELECT') {
+                            sel.selectedIndex = j;
+                            sel.dispatchEvent(new Event('change', {bubbles: true}));
+                          } else {
+                            sel.click();
+                            await new Promise(r => setTimeout(r, 600));
+                            const opts = document.querySelectorAll('[role="option"], li[data-value]');
+                            if (opts[j]) opts[j].click();
+                          }
+                        }''',
+                        [i, j],
+                    )
+                    await _wait_quiet(page, 1500)
+                except Exception as e:
+                    print(f'  select[{i}].opt[{j}] err: {e}')
+                    break
+        except Exception as e:
+            print(f'  select[{i}] err: {e}')
+
+
+async def _click_all_safe_buttons(page, deny=('Войти', 'Регистр', 'Выйти', 'Меню')):
+    """Click every non-disabled button whose text doesn't match deny list."""
+    deny_js = json.dumps(list(deny))
+    btn_count = await page.evaluate(
+        '''(deny) => Array.from(document.querySelectorAll('button:not([disabled])'))
+            .filter(b => !deny.some(d => (b.textContent || '').includes(d))).length''',
+        deny,
+    )
+    print(f'  safe buttons: {btn_count}')
+    for i in range(min(btn_count, 30)):
+        try:
+            await page.evaluate(
+                '''([i, deny]) => {
+                  const bs = Array.from(document.querySelectorAll('button:not([disabled])'))
+                    .filter(b => !deny.some(d => (b.textContent || '').includes(d)));
+                  if (bs[i]) bs[i].click();
+                }''',
+                [i, list(deny)],
+            )
+            await _wait_quiet(page, 1500)
+        except Exception:
+            pass
+
+
+async def trigger_kitchen_sink(page):
+    """Generic: scroll, click all tabs, iterate all selects."""
+    for _ in range(2):
+        await page.evaluate('window.scrollTo(0, document.body.scrollHeight)')
+        await page.wait_for_timeout(1200)
+        await page.evaluate('window.scrollTo(0, 0)')
+        await page.wait_for_timeout(800)
+    await _click_all_tabs(page)
+    await _iterate_selects(page, max_options=10)
+    await _click_all_safe_buttons(page)
+
+
+# ---------------------------------------------------------------------------
+# Direct-API helpers — fetch from inside browser ctx so cookies/WAF survive
+# ---------------------------------------------------------------------------
+# These five pages don't fire XHR on user interaction — they're SSR-baked or
+# expose a documented REST API. Instead of clicking we just call the APIs
+# directly via page.evaluate(fetch()) and dump the JSON via the response hook.
+# Page spec source: knowledge graph entities DomRF_Tab*_*_Apr26.
+
+REGIONS_FULL_RU = list(range(1, 100))  # try region codes 1..99 (Rosstat-style)
+
+
+async def _fetch_and_save(page, url, out_dir, seen_urls, saved):
+    """Fetch URL inside browser context (so cookies/WAF stay valid), save body."""
+    if url in seen_urls:
+        return
+    try:
+        resp = await page.evaluate(
+            '''async (u) => {
+              const r = await fetch(u, {credentials: 'include'});
+              const t = await r.text();
+              return {status: r.status, ct: r.headers.get('content-type') || '', text: t};
+            }''',
+            url,
+        )
+    except Exception as e:
+        print(f'  direct err {url}: {e}')
+        return
+    if not resp or resp.get('status', 0) >= 400:
+        return
+    text = resp.get('text') or ''
+    if not text or len(text) < 20:
+        return
+    seen_urls.add(url)
+    fname = safe_filename(url)
+    with open(os.path.join(out_dir, fname), 'w', encoding='utf-8') as f:
+        f.write(text)
+    saved.append((fname, len(text), url))
+    print(f'  direct  {fname} ({len(text)}b)')
+
+
+async def _dump_next_data(page, out_dir, saved):
+    """Pull __NEXT_DATA__ JSON blob from the rendered HTML and save it."""
+    blob = await page.evaluate('''() => {
+      const el = document.getElementById('__NEXT_DATA__');
+      return el ? el.textContent : null;
+    }''')
+    if not blob:
+        return
+    fname = '__NEXT_DATA__.json'
+    with open(os.path.join(out_dir, fname), 'w', encoding='utf-8') as f:
+        f.write(blob)
+    saved.append((fname, len(blob), 'inline:__NEXT_DATA__'))
+    print(f'  ssr     {fname} ({len(blob)}b)')
+
+
+async def trigger_housing(page, out_dir, seen_urls, saved):
+    """Tab 1: показатели жилищного строительства — SSR + portal-analytics REST."""
+    await _dump_next_data(page, out_dir, saved)
+    base = '/portal-analytics/api/dashboard'
+    # Aggregate (no filter) variants first
+    for ep in ('by-region', 'by-developer', 'by-room-count', 'by-flat-area'):
+        await _fetch_and_save(page, f'{base}/{ep}', out_dir, seen_urls, saved)
+    # Per-region drilldown
+    for rid in REGIONS_FULL_RU:
+        await _fetch_and_save(page, f'{base}/by-region?regionId={rid}', out_dir, seen_urls, saved)
+        await _fetch_and_save(page, f'{base}/by-room-count?regionId={rid}', out_dir, seen_urls, saved)
+        await _fetch_and_save(page, f'{base}/by-flat-area?regionId={rid}', out_dir, seen_urls, saved)
+
+
+async def trigger_housing_dev(page, out_dir, seen_urls, saved):
+    """Tab 2: жилищное строительство — different Redux slice, same portal-analytics base."""
+    await _dump_next_data(page, out_dir, saved)
+    # Probe common analytics endpoints
+    bases = [
+        '/portal-analytics/api/housing',
+        '/portal-analytics/api/dashboard',
+        '/аналитика/api/housing',
+    ]
+    for b in bases:
+        for ep in ('summary', 'by-region', 'by-developer', 'by-period', 'overview', 'main'):
+            await _fetch_and_save(page, f'{b}/{ep}', out_dir, seen_urls, saved)
+    await trigger_kitchen_sink(page)
+
+
+async def trigger_realization(page, out_dir, seen_urls, saved):
+    """Tab 6: реализация — /аналитика/api/rpp/* documented in graph."""
+    await _dump_next_data(page, out_dir, saved)
+    base = '/аналитика/api/rpp'
+    rep_year, rep_month = 2026, 3
+    # RF-level (no regionCode)
+    for ep in ('total', 'housing', 'readyYear'):
+        await _fetch_and_save(
+            page, f'{base}/{ep}?repMonth={rep_month}&repYear={rep_year}',
+            out_dir, seen_urls, saved,
+        )
+    await _fetch_and_save(
+        page,
+        f'{base}/developer?typeSquare=total&repMonth={rep_month}&repYear={rep_year}'
+        f'&developerOrder=totalSquare:desc',
+        out_dir, seen_urls, saved,
+    )
+    # Per-region (89 регионов РФ — codes mostly in 1..99)
+    for rcode in REGIONS_FULL_RU:
+        for ep in ('total', 'housing', 'readyYear'):
+            await _fetch_and_save(
+                page,
+                f'{base}/{ep}?regionCode={rcode}&repMonth={rep_month}&repYear={rep_year}',
+                out_dir, seen_urls, saved,
+            )
+        for ts in ('total', 'living'):
+            await _fetch_and_save(
+                page,
+                f'{base}/developer?typeSquare={ts}&regionCode={rcode}'
+                f'&repMonth={rep_month}&repYear={rep_year}'
+                f'&developerOrder=totalSquare:desc',
+                out_dir, seen_urls, saved,
+            )
+
+
+async def trigger_mortgage_rates(page, out_dir, seen_urls, saved):
+    """Tab 10: ставки предложений по ипотеке — likely SSR; probe known APIs."""
+    await _dump_next_data(page, out_dir, saved)
+    bases = [
+        '/portal-analytics/api/mortgage-rates',
+        '/portal-analytics/api/mortgage',
+        '/аналитика/api/mortgage-rates',
+        '/аналитика/api/mortgage',
+    ]
+    for b in bases:
+        for ep in ('', '/list', '/banks', '/offers', '/dashboard', '/summary'):
+            await _fetch_and_save(page, f'{b}{ep}', out_dir, seen_urls, saved)
+    await trigger_kitchen_sink(page)
+
+
+async def trigger_stat_series(page, out_dir, seen_urls, saved):
+    """Tab 12: статистические ряды — index of XLSX files baked into __NEXT_DATA__.
+
+    Per graph: data lives in props.initialState.housingData.list (CMS hierarchy).
+    XLSX URLs: /site/binaries/content/assets/domrf/xlsdashboard/{filename}.xlsx
+    Strategy: dump __NEXT_DATA__ + extract every .xlsx href and download.
+    """
+    await _dump_next_data(page, out_dir, saved)
+    # Extract all .xlsx links from the rendered DOM and the __NEXT_DATA__ blob
+    links = await page.evaluate('''() => {
+      const set = new Set();
+      document.querySelectorAll('a[href]').forEach(a => {
+        const h = a.getAttribute('href');
+        if (h && h.toLowerCase().includes('.xlsx')) set.add(h);
+      });
+      const blob = document.getElementById('__NEXT_DATA__');
+      if (blob) {
+        const re = /["'](\\/site\\/binaries[^"']+\\.xlsx)["']/g;
+        let m;
+        while ((m = re.exec(blob.textContent)) !== null) set.add(m[1]);
+      }
+      return Array.from(set);
+    }''')
+    print(f'  xlsx links found: {len(links)}')
+    xlsx_dir = os.path.join(out_dir, 'xlsx')
+    os.makedirs(xlsx_dir, exist_ok=True)
+    for href in links:
+        url = href if href.startswith('http') else BASE + href
+        # Download via fetch in browser context, then write bytes locally
+        try:
+            b64 = await page.evaluate(
+                '''async (u) => {
+                  const r = await fetch(u, {credentials: 'include'});
+                  if (!r.ok) return null;
+                  const buf = await r.arrayBuffer();
+                  let s = '';
+                  const arr = new Uint8Array(buf);
+                  for (let i = 0; i < arr.length; i++) s += String.fromCharCode(arr[i]);
+                  return btoa(s);
+                }''',
+                url,
+            )
+            if not b64:
+                continue
+            import base64
+            data = base64.b64decode(b64)
+            fname = re.sub(r'[^A-Za-z0-9_.\-]', '_', os.path.basename(href.split('?')[0]))
+            with open(os.path.join(xlsx_dir, fname), 'wb') as f:
+                f.write(data)
+            saved.append((f'xlsx/{fname}', len(data), url))
+            print(f'  xlsx    {fname} ({len(data)}b)')
+        except Exception as e:
+            print(f'  xlsx err {href}: {e}')
+
+
+PAGE_TRIGGERS = {
+    'housing':        trigger_housing,
+    'housing_dev':    trigger_housing_dev,
+    'realization':    trigger_realization,
+    'mortgage_rates': trigger_mortgage_rates,
+    'stat_series':    trigger_stat_series,
+}
+
+
 async def scrape_page(browser, name, path):
    out_dir = os.path.join(ROOT, name)
    os.makedirs(out_dir, exist_ok=True)
@ -101,30 +425,26 @@ async def scrape_page(browser, name, path):
    full_url = BASE + path
    try:
        await page.goto(full_url, wait_until='domcontentloaded', timeout=60_000)
-        # WAF challenge usually completes within 1-3 seconds
        try:
            await page.wait_for_load_state('networkidle', timeout=30_000)
-        except:
-            pass
-        # Extra wait for lazy-loaded API calls
-        await page.wait_for_timeout(8000)
-        # Scroll multiple times to trigger any IntersectionObserver-bound fetches
-        for _ in range(3):
-            await page.evaluate('window.scrollTo(0, document.body.scrollHeight)')
-            await page.wait_for_timeout(1500)
-            await page.evaluate('window.scrollTo(0, document.body.scrollHeight / 2)')
-            await page.wait_for_timeout(1500)
-            await page.evaluate('window.scrollTo(0, 0)')
-            await page.wait_for_timeout(1500)
-        # Click first interactive button (triggers React state → API)
-        try:
-            await page.evaluate('''() => {
-              const btn = document.querySelector('button:not([disabled])');
-              if (btn) btn.click();
-            }''')
-            await page.wait_for_timeout(3000)
        except Exception:
            pass
+        await page.wait_for_timeout(5000)
+
+        # Per-page triggers (falls back to kitchen sink for everything new)
+        trigger = PAGE_TRIGGERS.get(name)
+        try:
+            if trigger is None:
+                await trigger_kitchen_sink(page)
+            else:
+                # New triggers take (page, out_dir, seen_urls, saved) so they
+                # can perform direct API fetches and dump SSR blobs themselves.
+                await trigger(page, out_dir, seen_urls, saved)
+        except Exception as e:
+            print(f'  trigger err: {e}')
+
+        # Final settle
+        await _wait_quiet(page, 3000)

        # Re-fetch with size=1000 for endpoints that have small page-size
        for url in list(seen_urls):
--- a/data/sql/33_schema_domrf_lazy.sql
+++ b/data/sql/33_schema_domrf_lazy.sql
@ -0,0 +1,60 @@
+-- DOM.RF lazy-load pages (housing, housing_dev, realization, mortgage_rates,
+-- stat_series) — schema for the 5 pages that didn't capture on the first
+-- scrape pass. Realization = 4 endpoints (rpp/total, /housing, /readyYear,
+-- /developer) all share the same row shape, so a single wide table works.
+
+-- ── REALIZATION ────────────────────────────────────────────────────────────
+-- Per-region (and RF-level when region_code IS NULL) realization data from
+-- /аналитика/api/rpp/{total,housing,readyYear,developer}. The "subject"
+-- field is overloaded:
+--   - total    : RF code (always "100")
+--   - housing  : sold-progress bucket id ("0".."6")
+--   - ready_year: year of planned readiness ("2026"..)
+--   - developer: developer_id (e.g. "6208_0" = PRINZIP)
+-- subject_desc carries the human-readable label.
+CREATE TABLE IF NOT EXISTS domrf_realization (
+    snapshot_date    DATE NOT NULL,
+    endpoint_type    TEXT NOT NULL,           -- 'total' | 'housing' | 'ready_year' | 'developer'
+    region_code      INT  NOT NULL DEFAULT -1, -- Rosstat code; -1 sentinel for RF-only call
+    rep_year         INT  NOT NULL,
+    rep_month        INT  NOT NULL,
+    type_square      TEXT NOT NULL,           -- 'total' | 'living'
+    subject          TEXT NOT NULL,           -- bucket id / year / developer_id
+    subject_desc     TEXT,
+    share_total      NUMERIC,                 -- доля от итогового объёма (для housing/ready_year)
+    total_square     NUMERIC,                 -- м² × тыс
+    open_square      NUMERIC,
+    sold_square      NUMERIC,
+    unsold_square    NUMERIC,
+    unopened_square  NUMERIC,
+    sold_perc        NUMERIC,
+    unsold_perc      NUMERIC,
+    unopened_perc    NUMERIC,
+    sold_amount      NUMERIC,                 -- # квартир
+    price_avg        NUMERIC,                 -- ₽/м² средний
+    PRIMARY KEY (snapshot_date, endpoint_type, region_code, type_square, subject)
+);
+CREATE INDEX IF NOT EXISTS idx_realization_region ON domrf_realization (region_code, endpoint_type);
+CREATE INDEX IF NOT EXISTS idx_realization_dev    ON domrf_realization (subject) WHERE endpoint_type='developer';
+
+-- ── STAT_SERIES XLSX FILE INDEX ────────────────────────────────────────────
+-- Catalogue of the public XLSX statistics files served at
+-- /site/binaries/content/assets/domrf/xlsdashboard/*. Stores download metadata
+-- and a copy of the bytes (for offline reload). XLSX bodies live in BYTEA so
+-- the prod DB is the single source of truth — no filesystem dependency.
+CREATE TABLE IF NOT EXISTS domrf_xlsx_files (
+    snapshot_date    DATE NOT NULL,
+    filename         TEXT NOT NULL,
+    source_url       TEXT NOT NULL,
+    bytes            INT  NOT NULL,
+    sha256           TEXT NOT NULL,
+    content          BYTEA NOT NULL,
+    fetched_at       TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+    PRIMARY KEY (snapshot_date, filename)
+);
+
+-- ── SSR BLOBS for housing / housing_dev / mortgage_rates ────────────────────
+-- These pages bake their data into Next.js __NEXT_DATA__ instead of XHR.
+-- We dump the whole blob into the existing universal store so future
+-- normalization can mine it without re-scraping.
+-- (no new table — uses existing domrf_raw_endpoints from 31_schema_domrf_raw.sql)
--- a/data/sql/34_load_domrf_lazy.py
+++ b/data/sql/34_load_domrf_lazy.py
@ -0,0 +1,222 @@
+"""Load lazy-load DOM.RF pages (housing, housing_dev, realization,
+mortgage_rates, stat_series) into prod PG.
+
+Reads:  data/raw/domrf_full/{realization,housing,housing_dev,mortgage_rates,stat_series}/*
+Writes: domrf_realization (new),
+        domrf_xlsx_files (new),
+        domrf_raw_endpoints (existing universal store, append-only).
+
+Connect via SSH tunnel: PG_HOST=host.docker.internal PG_PORT=15432 (default).
+"""
+import os, json, re, sys, hashlib, base64, subprocess, datetime
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+ROOT = os.path.join(HERE, '..', 'raw', 'domrf_full')
+PG_HOST = os.environ.get('PG_HOST', 'host.docker.internal')
+PG_PORT = os.environ.get('PG_PORT', '15432')
+PG_USER = os.environ.get('PG_USER', 'gendesign')
+PG_DB   = os.environ.get('PG_DB',   'gendesign')
+PG_PASS = os.environ.get('PGPASSWORD', '2J2SBPMKuS998fiwhtQqDhMI')
+SNAP    = os.environ.get('SNAPSHOT_DATE', datetime.date.today().isoformat())
+
+
+def psql(sql, capture=True):
+    cmd = ['docker', 'run', '--rm', '-i', '-e', f'PGPASSWORD={PG_PASS}',
+           'postgres:16-alpine', 'psql',
+           '-h', PG_HOST, '-p', PG_PORT, '-U', PG_USER, '-d', PG_DB,
+           '-v', 'ON_ERROR_STOP=1', '--quiet']
+    res = subprocess.run(cmd, input=sql, capture_output=capture, text=True, encoding='utf-8')
+    if res.returncode != 0:
+        print('psql FAILED:', (res.stderr or '')[-2000:], file=sys.stderr)
+        raise SystemExit(res.returncode)
+    return res.stdout
+
+
+def esc(s):
+    if s is None:
+        return 'NULL'
+    return "'" + str(s).replace("'", "''") + "'"
+
+
+def num(v):
+    if v is None or v == '-' or v == '':
+        return 'NULL'
+    try:
+        return str(float(v))
+    except (ValueError, TypeError):
+        return 'NULL'
+
+
+def integer(v):
+    if v is None:
+        return 'NULL'
+    try:
+        return str(int(float(v)))
+    except (ValueError, TypeError):
+        return 'NULL'
+
+
+def upsert_chunked(rows, table, cols, conflict_cols, batch=500):
+    """Insert rows in batches with ON CONFLICT DO UPDATE."""
+    if not rows:
+        return 0
+    update = ', '.join(f'{c}=EXCLUDED.{c}' for c in cols if c not in conflict_cols)
+    total = 0
+    for i in range(0, len(rows), batch):
+        chunk = rows[i:i + batch]
+        sql = (f"INSERT INTO {table} ({','.join(cols)}) VALUES "
+               + ',\n'.join(chunk)
+               + f"\nON CONFLICT ({','.join(conflict_cols)}) DO UPDATE SET {update};")
+        psql(sql)
+        total += len(chunk)
+    return total
+
+
+# ── 1. REALIZATION (rpp/{total,housing,readyYear,developer}) ─────────────────
+RPP_FNAME_RE = re.compile(
+    r'rpp_(total|housing|readyYear|developer)__'
+    r'(?:typeSquare_(total|living)_)?'
+    r'(?:regionCode_(\d+)_)?'
+    r'repMonth_(\d+)_repYear_(\d+)'
+)
+
+
+def load_realization():
+    sec_dir = os.path.join(ROOT, 'realization')
+    if not os.path.isdir(sec_dir):
+        print('  realization dir missing')
+        return 0
+    cols = ['snapshot_date', 'endpoint_type', 'region_code', 'rep_year', 'rep_month',
+            'type_square', 'subject', 'subject_desc', 'share_total',
+            'total_square', 'open_square', 'sold_square', 'unsold_square', 'unopened_square',
+            'sold_perc', 'unsold_perc', 'unopened_perc', 'sold_amount', 'price_avg']
+    rows = []
+    seen = set()
+    for fname in sorted(os.listdir(sec_dir)):
+        if not fname.endswith('.json') or not fname.startswith('rpp_'):
+            continue
+        m = RPP_FNAME_RE.match(fname)
+        if not m:
+            continue
+        ep_raw, ts_filename, rcode_filename, rmonth, ryear = m.groups()
+        endpoint_type = {'total': 'total', 'housing': 'housing',
+                         'readyYear': 'ready_year', 'developer': 'developer'}[ep_raw]
+        region_code = int(rcode_filename) if rcode_filename else -1
+        try:
+            doc = json.load(open(os.path.join(sec_dir, fname), encoding='utf-8'))
+        except Exception as e:
+            print(f'  parse err {fname}: {e}')
+            continue
+        for it in doc.get('data') or []:
+            ts = ts_filename or it.get('typeSquare') or 'total'
+            subject = str(it.get('subject', ''))
+            if not subject:
+                continue
+            key = (endpoint_type, region_code, ts, subject)
+            if key in seen:
+                continue
+            seen.add(key)
+            rows.append(
+                f"({esc(SNAP)},{esc(endpoint_type)},{region_code},"
+                f"{int(ryear)},{int(rmonth)},{esc(ts)},{esc(subject)},"
+                f"{esc(it.get('subjectDesc'))},{num(it.get('shareTotal'))},"
+                f"{num(it.get('totalSquare'))},{num(it.get('openSquare'))},"
+                f"{num(it.get('soldSquare'))},{num(it.get('unsoldSquare'))},"
+                f"{num(it.get('unopenedSquare'))},{num(it.get('soldPerc'))},"
+                f"{num(it.get('unsoldPerc'))},{num(it.get('unopenedPerc'))},"
+                f"{num(it.get('soldAmount'))},{num(it.get('priceAvg'))})"
+            )
+    return upsert_chunked(
+        rows, 'domrf_realization', cols,
+        conflict_cols=['snapshot_date', 'endpoint_type', 'region_code', 'type_square', 'subject'],
+    )
+
+
+# ── 2. STAT_SERIES XLSX FILES ────────────────────────────────────────────────
+def load_xlsx_files():
+    xlsx_dir = os.path.join(ROOT, 'stat_series', 'xlsx')
+    if not os.path.isdir(xlsx_dir):
+        print('  xlsx dir missing')
+        return 0
+    base_url = 'https://xn--80az8a.xn--d1aqf.xn--p1ai/site/binaries/content/assets/domrf/xlsdashboard/'
+    rows = []
+    cols = ['snapshot_date', 'filename', 'source_url', 'bytes', 'sha256', 'content']
+    for fname in sorted(os.listdir(xlsx_dir)):
+        if not fname.endswith('.xlsx'):
+            continue
+        path = os.path.join(xlsx_dir, fname)
+        with open(path, 'rb') as f:
+            blob = f.read()
+        sha = hashlib.sha256(blob).hexdigest()
+        # PG bytea hex format: \xDEADBEEF
+        hex_blob = '\\\\x' + blob.hex()
+        rows.append(
+            f"({esc(SNAP)},{esc(fname)},{esc(base_url + fname)},"
+            f"{len(blob)},{esc(sha)},E{esc(hex_blob)})"
+        )
+    return upsert_chunked(
+        rows, 'domrf_xlsx_files', cols,
+        conflict_cols=['snapshot_date', 'filename'],
+        batch=5,  # XLSX bytes are big — keep batches tiny to avoid huge SQL strings
+    )
+
+
+# ── 3. RAW JSONB store for SSR + housing per-region ──────────────────────────
+def load_raw_endpoints():
+    """Dump every captured JSON file from the 5 lazy pages into
+    domrf_raw_endpoints (existing universal store). Keeps all data accessible
+    for future normalization without re-scraping."""
+    cols = ['snapshot_date', 'section', 'endpoint', 'source_url', 'payload',
+            'payload_size']
+    rows = []
+    for section in ('housing', 'housing_dev', 'mortgage_rates', 'stat_series'):
+        sec_dir = os.path.join(ROOT, section)
+        if not os.path.isdir(sec_dir):
+            continue
+        for fname in sorted(os.listdir(sec_dir)):
+            path = os.path.join(sec_dir, fname)
+            if not os.path.isfile(path) or not fname.endswith('.json'):
+                continue
+            with open(path, encoding='utf-8') as f:
+                text = f.read()
+            # Validate JSON; skip if parse fails (rare).
+            try:
+                json.loads(text)
+            except Exception:
+                continue
+            endpoint = fname[:-5]  # drop .json
+            rows.append(
+                f"({esc(SNAP)},{esc(section)},{esc(endpoint)},NULL,"
+                f"{esc(text)}::jsonb,{len(text)})"
+            )
+    return upsert_chunked(
+        rows, 'domrf_raw_endpoints', cols,
+        conflict_cols=['snapshot_date', 'section', 'endpoint'],
+        batch=50,  # payloads can be 100KB+
+    )
+
+
+def main():
+    print(f'Snapshot date: {SNAP}')
+    print(f'PG: {PG_USER}@{PG_HOST}:{PG_PORT}/{PG_DB}')
+
+    schema = open(os.path.join(HERE, '33_schema_domrf_lazy.sql'), encoding='utf-8').read()
+    psql(schema)
+    print('schema applied')
+
+    # Stamp snapshot in domrf_snapshots (FK target)
+    psql(f"INSERT INTO domrf_snapshots(snapshot_date) VALUES ({esc(SNAP)}) "
+         f"ON CONFLICT DO NOTHING;")
+
+    n_real = load_realization()
+    print(f'  realization: {n_real} rows')
+
+    n_xlsx = load_xlsx_files()
+    print(f'  xlsx_files:  {n_xlsx} rows')
+
+    n_raw = load_raw_endpoints()
+    print(f'  raw_endpoints: {n_raw} rows')
+
+
+if __name__ == '__main__':
+    main()
--- a/data/sql/35_schema_domrf_stats.sql
+++ b/data/sql/35_schema_domrf_stats.sql
@ -0,0 +1,29 @@
+-- Normalized stat_series observations parsed from DOM.RF XLSX files.
+-- Source: domrf_xlsx_files (BYTEA archive) → parsed via 36_parse_xlsx_stats.py.
+-- Long-form panel: (territory × indicator × month) → numeric value.
+
+-- sheet_code alone isn't unique — '01_01_01' appears in both
+-- stockvariablesexsales (МКД 214-ФЗ без эскроу) and stockvariableskrt (МКД КРТ).
+-- Composite PK on (file_name, sheet_code) keeps them separate.
+CREATE TABLE IF NOT EXISTS domrf_stat_series_indicators (
+    file_name        TEXT NOT NULL,               -- source xlsx
+    sheet_code       TEXT NOT NULL,               -- '01_01_01' .. '01_04_NN'
+    indicator_name   TEXT NOT NULL,               -- human label from row 2
+    section_title    TEXT,                        -- top-of-file context
+    n_periods        INT,                         -- # of monthly columns
+    n_territories    INT,
+    PRIMARY KEY (file_name, sheet_code)
+);
+
+CREATE TABLE IF NOT EXISTS domrf_stat_series_observations (
+    snapshot_date    DATE NOT NULL,
+    file_name        TEXT NOT NULL,
+    sheet_code       TEXT NOT NULL,
+    territory        TEXT NOT NULL,                -- 'Российская Федерация' | ФО | регион
+    obs_date         DATE NOT NULL,                -- first day of month
+    value            NUMERIC,
+    PRIMARY KEY (snapshot_date, file_name, sheet_code, territory, obs_date)
+);
+CREATE INDEX IF NOT EXISTS idx_stat_obs_sheet     ON domrf_stat_series_observations (file_name, sheet_code);
+CREATE INDEX IF NOT EXISTS idx_stat_obs_territory ON domrf_stat_series_observations (territory, obs_date);
+CREATE INDEX IF NOT EXISTS idx_stat_obs_date      ON domrf_stat_series_observations (obs_date);
--- a/data/sql/36_parse_xlsx_stats.py
+++ b/data/sql/36_parse_xlsx_stats.py
@ -0,0 +1,218 @@
+"""Parse DOM.RF stat_series XLSX files into long-form observations.
+
+Reads:  data/raw/domrf_full/stat_series/xlsx/*.xlsx (4 standard-format files)
+Writes: domrf_stat_series_indicators, domrf_stat_series_observations
+        (created by 35_schema_domrf_stats.sql).
+
+Standard format (files 01_01..01_04):
+    Sheet '*_00' = index (skipped — derive from individual sheets)
+    Sheets '*_NN' = one indicator × all territories × all months
+        row 1: section title (long sentence)
+        row 2: indicator name ("Количество МКД...")
+        row 4: header — col 0 = "Регион", col 1..N = month dates
+        row 5+: territory + numeric values
+
+Files with bespoke format (01_05_*, 01_06_*) are skipped — left in
+domrf_xlsx_files BYTEA archive for ad-hoc analysis.
+"""
+import os, re, sys, subprocess, datetime
+import openpyxl
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+ROOT = os.path.join(HERE, '..', 'raw', 'domrf_full', 'stat_series', 'xlsx')
+PG_HOST = os.environ.get('PG_HOST', 'host.docker.internal')
+PG_PORT = os.environ.get('PG_PORT', '15432')
+PG_USER = os.environ.get('PG_USER', 'gendesign')
+PG_DB   = os.environ.get('PG_DB',   'gendesign')
+PG_PASS = os.environ.get('PGPASSWORD', '2J2SBPMKuS998fiwhtQqDhMI')
+SNAP    = os.environ.get('SNAPSHOT_DATE', datetime.date.today().isoformat())
+
+# Sheet name pattern: NN_NN_NN (e.g. '01_01_01', '01_02_05'). The '*_00' index
+# is skipped (last group != '00').
+SHEET_RE = re.compile(r'^(\d\d)_(\d\d)_(\d\d)$')
+
+# Standard-format files only. Others have bespoke layouts.
+STANDARD_FILES = {
+    '01_01_stockvariablesexsales.xlsx',
+    '01_01_stockvariableskrt.xlsx',
+    '01_02_flowvariables.xlsx',
+    '01_03_sales.xlsx',
+    '01_04_flowvariables.xlsx',
+    '01_04_stockvariables.xlsx',
+    '01_06_stockvariables_izs.xlsx',
+}
+
+
+def psql(sql, capture=True):
+    cmd = ['docker', 'run', '--rm', '-i', '-e', f'PGPASSWORD={PG_PASS}',
+           'postgres:16-alpine', 'psql',
+           '-h', PG_HOST, '-p', PG_PORT, '-U', PG_USER, '-d', PG_DB,
+           '-v', 'ON_ERROR_STOP=1', '--quiet']
+    res = subprocess.run(cmd, input=sql, capture_output=capture, text=True, encoding='utf-8')
+    if res.returncode != 0:
+        print('psql FAILED:', (res.stderr or '')[-2000:], file=sys.stderr)
+        raise SystemExit(res.returncode)
+    return res.stdout
+
+
+def esc(s):
+    if s is None:
+        return 'NULL'
+    return "'" + str(s).replace("'", "''") + "'"
+
+
+def num(v):
+    if v is None or v == '':
+        return 'NULL'
+    try:
+        f = float(v)
+        # Strip trailing .0 noise but keep precision
+        if f != f:  # NaN
+            return 'NULL'
+        return repr(f)
+    except (ValueError, TypeError):
+        return 'NULL'
+
+
+def to_date(v):
+    if v is None:
+        return None
+    if isinstance(v, datetime.datetime):
+        return v.date().isoformat()
+    if isinstance(v, datetime.date):
+        return v.isoformat()
+    s = str(v)
+    # Sometimes the cell is a string like '2024-03-01'
+    m = re.match(r'(\d{4})-(\d{2})-(\d{2})', s)
+    if m:
+        return f'{m.group(1)}-{m.group(2)}-{m.group(3)}'
+    return None
+
+
+def parse_sheet(ws, file_name, sheet_code):
+    """Return (indicator_name, [(territory, obs_date, value)...]). Stream rows."""
+    rows = list(ws.iter_rows(values_only=True))
+    if len(rows) < 5:
+        return None, []
+    # Row 1 (index 0) — section title; row 2 — indicator; rows 4..end — data
+    indicator_name = (rows[1][0] or '').strip() if rows[1] and rows[1][0] else ''
+    header = rows[3]
+    if not header or len(header) < 2:
+        return indicator_name, []
+    # Skip first 'Регион' column; remaining are month dates
+    date_cols = []
+    for col_idx, cell in enumerate(header[1:], start=1):
+        d = to_date(cell)
+        if d:
+            date_cols.append((col_idx, d))
+    obs = []
+    for r in rows[4:]:
+        if not r or r[0] is None:
+            continue
+        territory = str(r[0]).strip()
+        if not territory or territory.lower() == 'перейти в начало':
+            continue
+        for col_idx, obs_date in date_cols:
+            if col_idx >= len(r):
+                continue
+            val = r[col_idx]
+            if val is None or val == '':
+                continue
+            obs.append((territory, obs_date, val))
+    return indicator_name, obs
+
+
+def main():
+    if not os.path.isdir(ROOT):
+        print('xlsx dir missing:', ROOT)
+        return
+    schema = open(os.path.join(HERE, '35_schema_domrf_stats.sql'),
+                  encoding='utf-8').read()
+    psql(schema)
+    print('schema applied')
+
+    indicator_rows = []  # (file_name, sheet_code, indicator_name, n_periods, n_territories)
+    obs_rows = []        # (file_name, sheet_code, territory, obs_date, value)
+    parsed_sheets = 0
+    skipped_files = []
+
+    for fname in sorted(os.listdir(ROOT)):
+        if not fname.endswith('.xlsx'):
+            continue
+        if fname not in STANDARD_FILES:
+            skipped_files.append(fname)
+            continue
+        path = os.path.join(ROOT, fname)
+        print(f'  parsing {fname}')
+        wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
+        for sn in wb.sheetnames:
+            m = SHEET_RE.match(sn)
+            if not m or m.group(3) == '00':
+                continue
+            ws = wb[sn]
+            indicator_name, obs = parse_sheet(ws, fname, sn)
+            if not obs:
+                continue
+            territories = {t for t, _, _ in obs}
+            periods = {d for _, d, _ in obs}
+            indicator_rows.append((fname, sn, indicator_name, len(periods), len(territories)))
+            for t, d, v in obs:
+                obs_rows.append((fname, sn, t, d, v))
+            parsed_sheets += 1
+        wb.close()
+
+    print(f'parsed {parsed_sheets} sheets; skipped (bespoke format): {skipped_files}')
+    print(f'  indicators: {len(indicator_rows)}')
+    print(f'  observations: {len(obs_rows)}')
+
+    # ── INDICATORS upsert ──
+    if indicator_rows:
+        sql = ("INSERT INTO domrf_stat_series_indicators "
+               "(file_name, sheet_code, indicator_name, n_periods, n_territories) VALUES "
+               + ','.join(
+                    f"({esc(f)},{esc(c)},{esc(n)},{p},{t})"
+                    for f, c, n, p, t in indicator_rows)
+               + " ON CONFLICT (file_name, sheet_code) DO UPDATE SET "
+                 "indicator_name=EXCLUDED.indicator_name, "
+                 "n_periods=EXCLUDED.n_periods, "
+                 "n_territories=EXCLUDED.n_territories;")
+        psql(sql)
+        print(f'  indicators upserted: {len(indicator_rows)}')
+
+    # ── OBSERVATIONS upsert (chunked — ~810K rows). Dedupe within batch
+    # because the same (file, sheet, territory, obs_date) can appear twice
+    # in the source XLSX (rarely — duplicate header dates) and ON CONFLICT
+    # rejects in-batch duplicates.
+    seen = set()
+    deduped = []
+    for f, c, t, d, v in obs_rows:
+        key = (f, c, t, d)
+        if key in seen:
+            continue
+        seen.add(key)
+        deduped.append((f, c, t, d, v))
+    if len(deduped) != len(obs_rows):
+        print(f'  deduped {len(obs_rows) - len(deduped)} duplicate observations')
+    obs_rows = deduped
+
+    BATCH = 2000
+    total = 0
+    for i in range(0, len(obs_rows), BATCH):
+        chunk = obs_rows[i:i + BATCH]
+        values = ','.join(
+            f"({esc(SNAP)},{esc(f)},{esc(c)},{esc(t)},{esc(d)},{num(v)})"
+            for f, c, t, d, v in chunk
+        )
+        sql = ("INSERT INTO domrf_stat_series_observations "
+               "(snapshot_date, file_name, sheet_code, territory, obs_date, value) VALUES "
+               + values
+               + " ON CONFLICT (snapshot_date, file_name, sheet_code, territory, obs_date) "
+                 "DO UPDATE SET value=EXCLUDED.value;")
+        psql(sql)
+        total += len(chunk)
+        if total % 20000 == 0 or total == len(obs_rows):
+            print(f'    obs upserted: {total}/{len(obs_rows)}')
+
+
+if __name__ == '__main__':
+    main()
--- a/data/sql/37_cleanup_redundant.sql
+++ b/data/sql/37_cleanup_redundant.sql
@ -0,0 +1,19 @@
+-- Drop tables fully superseded by domrf_realization (loaded 27.04.2026).
+-- Coverage check confirmed:
+--   • domrf_sold_out_by_year (6 rows, RF-only) ⊂
+--     domrf_realization WHERE endpoint_type='ready_year' AND region_code=-1
+--     (+ × 89 regions in same source). Values match exactly: e.g. 2027 sold 30%,
+--     2028 sold 17%, 2029 sold 9%. API data is more complete (PDF had nulls).
+--   • domrf_sold_out_by_progress (6 rows, RF-only) ⊂
+--     domrf_realization WHERE endpoint_type='housing' AND region_code=-1
+--     (+ × 89 regions). Same buckets: not_open / до 20% / ... / 80+.
+--
+-- domrf_sold_out (9 rows) is KEPT — has unique attracted_funds_mln_rub column
+-- not present in the API response.
+--
+-- Run manually:
+--   docker run --rm -e PGPASSWORD=... postgres:16-alpine psql \
+--     -h host.docker.internal -p 15432 -U gendesign -d gendesign \
+--     -f data/sql/37_cleanup_redundant.sql
+DROP TABLE IF EXISTS domrf_sold_out_by_year;
+DROP TABLE IF EXISTS domrf_sold_out_by_progress;
--- a/data/sql/38_relations_fix.sql
+++ b/data/sql/38_relations_fix.sql
@ -0,0 +1,63 @@
+-- Fix missing relations + housekeeping after 27.04.2026 loads.
+-- Audit findings:
+--   1. domrf_realization, domrf_xlsx_files, domrf_stat_series_observations
+--      have snapshot_date but NO FK to domrf_snapshots.
+--   2. domrf_realization.region_code uses sentinel -1 for RF-level rows; the
+--      RF placeholder already exists in domrf_regions as region_id=0. Migrate
+--      sentinel and add FK to keep data consistent.
+--   3. Two duplicate indexes (covered by PK):
+--      - idx_domrf_developer_aggregates_dev
+--      - rr_agg_settl_search (on rr_agg_settlement)
+--   4. domrf_stat_series_observations.territory is TEXT and uses naming that
+--      does NOT line up with domrf_regions.region_name (e.g. 'г. Москва' vs
+--      'Город Москва', 'Республика Адыгея (Адыгея)' vs 'Республика Адыгея',
+--      plus 8 federal districts and a few synthetic rows). FK is not added;
+--      lookup-by-alias work tracked as a separate TODO.
+--   5. ~8 700 stat_series rows have territory='-' or 'г. Москва1' (parser
+--      noise). NOT deleted here — keep raw for future cleaning pass.
+
+BEGIN;
+
+-- 1. Migrate RF sentinel -1 → 0 (matches existing RF placeholder in regions)
+UPDATE domrf_realization SET region_code = 0 WHERE region_code = -1;
+ALTER TABLE domrf_realization ALTER COLUMN region_code SET DEFAULT 0;
+
+-- 2. Add missing FKs
+ALTER TABLE domrf_realization
+  ADD CONSTRAINT fk_snap_realization
+    FOREIGN KEY (snapshot_date) REFERENCES domrf_snapshots(snapshot_date) ON UPDATE CASCADE,
+  ADD CONSTRAINT fk_realization_region
+    FOREIGN KEY (region_code) REFERENCES domrf_regions(region_id) ON UPDATE CASCADE;
+
+ALTER TABLE domrf_xlsx_files
+  ADD CONSTRAINT fk_snap_xlsx_files
+    FOREIGN KEY (snapshot_date) REFERENCES domrf_snapshots(snapshot_date) ON UPDATE CASCADE;
+
+ALTER TABLE domrf_stat_series_observations
+  ADD CONSTRAINT fk_snap_stat_series_observations
+    FOREIGN KEY (snapshot_date) REFERENCES domrf_snapshots(snapshot_date) ON UPDATE CASCADE;
+
+-- 3. Drop duplicate indexes (functionally covered by PK)
+DROP INDEX IF EXISTS idx_domrf_developer_aggregates_dev;
+DROP INDEX IF EXISTS rr_agg_settl_search;
+
+-- 4. Document tables with COMMENT — helps future maintainers (and AI agents)
+--    distinguish overlapping data sources (PDF snapshot vs API daily etc).
+COMMENT ON TABLE domrf_realization IS
+  'RPP API per-region realization (sold/unsold/price). 4 endpoint_types: total, housing, ready_year, developer. region_code=0 = RF total. SUPERSET of the dropped sold_out_by_year / sold_out_by_progress (kept domrf_sold_out for unique attracted_funds_mln_rub).';
+COMMENT ON TABLE domrf_sold_out IS
+  'PDF-snapshot of RF + 8 top regions (manual export 26.04). Kept ONLY for unique attracted_funds_mln_rub column. For full 89-region coverage use domrf_realization endpoint_type=total.';
+COMMENT ON TABLE domrf_stat_series_observations IS
+  'Long-form panel of monthly Rosstat-form stats from 7 XLSX files (parsed 27.04). Indicators: см. domrf_stat_series_indicators. Coverage: 2020-2026, ~70 territories incl. 8 federal districts. NB: territory naming differs from domrf_regions.region_name (FK not enforced).';
+COMMENT ON TABLE domrf_stat_series_indicators IS
+  'Lookup of XLSX sheet codes to human indicator names. PK (file_name, sheet_code) — codes like 01_01_01 repeat across files with different semantics.';
+COMMENT ON TABLE domrf_xlsx_files IS
+  'BYTEA archive of raw stat_series XLSX. Acts as backup for re-parsing (domrf_stat_series_observations). 2 of 9 files (01_05_*) bespoke format — not yet normalized.';
+COMMENT ON TABLE domrf_raw_endpoints IS
+  'Universal JSONB store: every captured /api/* response from DOM.RF analytics scrape. Source-of-truth backup for future normalization.';
+COMMENT ON TABLE domrf_developers IS
+  'Developer dictionary from JSON dashboard (with developer_id). FK target for domrf_developer_aggregates and domrf_realization (when endpoint_type=developer; not enforced).';
+COMMENT ON TABLE domrf_developers_full IS
+  'XLSX export of full developer registry (manual 26.04). Has metrics (area, permits, flats, market_share) but NO developer_id — name-only join to domrf_developers possible.';
+
+COMMIT;
--- a/data/sql/39_territory_aliases.sql
+++ b/data/sql/39_territory_aliases.sql
@ -0,0 +1,120 @@
+-- Resolve stat_series.territory ↔ domrf_regions.region_name name mismatches.
+-- Stat_series uses Rosstat-style names that differ in 18 of 99 territories:
+--   • 9 regions: "г. Москва" vs "Город Москва", "Республика Адыгея (Адыгея)"
+--     vs "Республика Адыгея", "- Алания"/"(Татарстан)"/"- Чувашия" suffixes;
+--   • 8 federal districts ("Центральный ФО") — not in domrf_regions at all;
+--   • 1 synthetic row "Российская Федерация (без г. Москва)";
+--   • 2 noise rows '-' and 'г. Москва1' (XLSX parser artefacts).
+--
+-- Solution: an alias lookup that maps any incoming text to a structured target
+-- (level + region_id|fo_id). View v_stat_series_resolved joins observations
+-- through it for clean territory resolution without rewriting 809K rows.
+
+BEGIN;
+
+-- ── 1. Federal districts as a first-class table ─────────────────────────────
+CREATE TABLE IF NOT EXISTS domrf_federal_districts (
+    fo_id    INT  PRIMARY KEY,
+    fo_name  TEXT NOT NULL UNIQUE
+);
+INSERT INTO domrf_federal_districts (fo_id, fo_name) VALUES
+    (1, 'Центральный ФО'),
+    (2, 'Северо-Западный ФО'),
+    (3, 'Южный ФО'),
+    (4, 'Северо-Кавказский ФО'),
+    (5, 'Приволжский ФО'),
+    (6, 'Уральский ФО'),
+    (7, 'Сибирский ФО'),
+    (8, 'Дальневосточный ФО')
+ON CONFLICT (fo_id) DO NOTHING;
+
+COMMENT ON TABLE domrf_federal_districts IS
+    '8 федеральных округов РФ. Связывается с domrf_regions.federal_district по имени, и используется как target для domrf_territory_aliases (level=fo).';
+
+-- ── 2. Generic territory alias table ────────────────────────────────────────
+CREATE TABLE IF NOT EXISTS domrf_territory_aliases (
+    territory_text   TEXT PRIMARY KEY,
+    level            TEXT NOT NULL CHECK (level IN ('rf','fo','region','synthetic','noise')),
+    region_id        INT  REFERENCES domrf_regions(region_id),
+    fo_id            INT  REFERENCES domrf_federal_districts(fo_id),
+    canonical_name   TEXT NOT NULL,
+    notes            TEXT,
+    CONSTRAINT alias_target_chk CHECK (
+        (level='rf'        AND region_id=0          AND fo_id IS NULL) OR
+        (level='region'    AND region_id IS NOT NULL AND fo_id IS NULL) OR
+        (level='fo'        AND region_id IS NULL    AND fo_id IS NOT NULL) OR
+        (level='synthetic' AND region_id IS NULL    AND fo_id IS NULL) OR
+        (level='noise'     AND region_id IS NULL    AND fo_id IS NULL)
+    )
+);
+CREATE INDEX IF NOT EXISTS idx_territory_aliases_region ON domrf_territory_aliases (region_id);
+CREATE INDEX IF NOT EXISTS idx_territory_aliases_fo     ON domrf_territory_aliases (fo_id);
+
+COMMENT ON TABLE domrf_territory_aliases IS
+    'Text→(level, region_id|fo_id) lookup. Resolves stat_series territory naming variants to canonical regions/FO. Use v_stat_series_resolved for joined views.';
+
+-- ── 3a. Auto-populate identity aliases for the 81 regions whose names match ─
+--     (and the RF row at region_id=0)
+INSERT INTO domrf_territory_aliases (territory_text, level, region_id, canonical_name, notes)
+SELECT
+    r.region_name,
+    CASE WHEN r.region_id = 0 THEN 'rf' ELSE 'region' END,
+    r.region_id,
+    r.region_name,
+    'auto: identity alias from domrf_regions.region_name'
+FROM domrf_regions r
+WHERE r.region_name IN (SELECT DISTINCT territory FROM domrf_stat_series_observations)
+ON CONFLICT (territory_text) DO NOTHING;
+
+-- ── 3b. Manual aliases for 9 regions with format differences ────────────────
+INSERT INTO domrf_territory_aliases (territory_text, level, region_id, canonical_name, notes) VALUES
+    ('г. Москва',                              'region', 77, 'Город Москва',                'г./Город prefix mismatch'),
+    ('г. Санкт-Петербург',                     'region', 78, 'Город Санкт-Петербург',       'г./Город prefix mismatch'),
+    ('г. Севастополь',                         'region', 92, 'Город Севастополь',           'г./Город prefix mismatch'),
+    ('Республика Адыгея (Адыгея)',             'region',  1, 'Республика Адыгея',           'parenthetical suffix in Rosstat naming'),
+    ('Республика Татарстан (Татарстан)',       'region', 16, 'Республика Татарстан',        'parenthetical suffix in Rosstat naming'),
+    ('Республика Северная Осетия - Алания',    'region', 15, 'Республика Северная Осетия',  '- Алания suffix in Rosstat naming'),
+    ('Чувашская Республика - Чувашия',         'region', 21, 'Чувашская Республика',        '- Чувашия suffix in Rosstat naming')
+ON CONFLICT (territory_text) DO NOTHING;
+-- Note: "Республика Саха (Якутия)" and "Кемеровская область - Кузбасс" use the
+-- same parenthetical convention BUT match exactly between sources, so identity
+-- aliases above already cover them.
+
+-- ── 3c. Federal districts — 8 aliases pointing at the new FO table ──────────
+INSERT INTO domrf_territory_aliases (territory_text, level, fo_id, canonical_name, notes)
+SELECT fo_name, 'fo', fo_id, fo_name, 'auto: identity alias from domrf_federal_districts'
+FROM domrf_federal_districts
+ON CONFLICT (territory_text) DO NOTHING;
+
+-- ── 3d. Synthetic + noise rows ──────────────────────────────────────────────
+INSERT INTO domrf_territory_aliases (territory_text, level, canonical_name, notes) VALUES
+    ('Российская Федерация (без г. Москва)', 'synthetic', 'РФ без Москвы',
+     'Synthetic row in some XLSX summaries — RF totals excluding Moscow.'),
+    ('-',          'noise', '-',         'XLSX parser artefact: empty separator row swallowed by parser.'),
+    ('г. Москва1', 'noise', 'г. Москва1','XLSX source typo (extra "1" suffix). Real "г. Москва" present separately.')
+ON CONFLICT (territory_text) DO NOTHING;
+
+-- ── 4. Resolved view: stat_series_observations + alias target ───────────────
+CREATE OR REPLACE VIEW v_stat_series_resolved AS
+SELECT
+    o.snapshot_date,
+    o.file_name,
+    o.sheet_code,
+    i.indicator_name,
+    o.territory,
+    a.level         AS territory_level,
+    a.region_id,
+    a.fo_id,
+    a.canonical_name,
+    o.obs_date,
+    o.value
+FROM domrf_stat_series_observations o
+LEFT JOIN domrf_territory_aliases a
+       ON a.territory_text = o.territory
+LEFT JOIN domrf_stat_series_indicators i
+       ON i.file_name = o.file_name AND i.sheet_code = o.sheet_code;
+
+COMMENT ON VIEW v_stat_series_resolved IS
+    'stat_series_observations + indicator name + resolved territory (level/region_id/fo_id). Filter WHERE territory_level=''region'' for clean region-level analysis.';
+
+COMMIT;
--- a/data/sql/40_scrape_history.py
+++ b/data/sql/40_scrape_history.py
@ -0,0 +1,151 @@
+"""Historical scrape: 12 months of 2025 + Jan/Feb 2026 for realization +
+sold_ready. The DOM.RF API accepts repYear/repMonth params and returns
+historical state — already verified by probe.
+
+Output: data/raw/domrf_history/<section>/<YYYY-MM>/<endpoint>.json
+
+Usage:
+    PYTHONIOENCODING=utf-8 python data/sql/40_scrape_history.py
+"""
+import asyncio, json, os, re, sys
+from urllib.parse import urlsplit, parse_qs, urlencode, urlunsplit
+from playwright.async_api import async_playwright
+
+ROOT = os.path.join(os.path.dirname(__file__), '..', 'raw', 'domrf_history')
+BASE = 'https://xn--80az8a.xn--d1aqf.xn--p1ai'
+
+# (year, month) pairs to backfill. 2025 full year + 2026 Jan/Feb.
+# 2026-03 already has snapshot_date=2026-04-27 in domrf_realization.
+PERIODS = [(2025, m) for m in range(1, 13)] + [(2026, 1), (2026, 2)]
+
+REGIONS = list(range(1, 100))  # Rosstat-style 1..99
+
+
+def safe_filename(url):
+    parts = urlsplit(url)
+    base = parts.path.split('/api/')[-1].strip('/')
+    base = re.sub(r'[^A-Za-z0-9_\-]', '_', base)[:120]
+    if parts.query:
+        q = re.sub(r'[^A-Za-z0-9_\-]', '_', parts.query)[:80]
+        base = f'{base}__{q}'
+    return base + '.json'
+
+
+async def warm_up(page, url):
+    """Goto a section page so Chromium passes WAF for that subdomain segment."""
+    await page.goto(BASE + url, wait_until='domcontentloaded', timeout=60_000)
+    try:
+        await page.wait_for_load_state('networkidle', timeout=30_000)
+    except Exception:
+        pass
+    await page.wait_for_timeout(5000)
+
+
+async def fetch_save(page, url, out_dir, saved):
+    full = url if url.startswith('http') else BASE + url
+    try:
+        resp = await page.evaluate(
+            '''async (u) => {
+              const r = await fetch(u, {credentials: 'include'});
+              const t = await r.text();
+              return {s: r.status, ct: r.headers.get('content-type')||'', t};
+            }''',
+            full,
+        )
+    except Exception as e:
+        print(f'  ERR {full}: {e}')
+        return False
+    if not resp or resp.get('s', 0) != 200:
+        return False
+    text = resp.get('t') or ''
+    if len(text) < 20:
+        return False
+    fname = safe_filename(full)
+    with open(os.path.join(out_dir, fname), 'w', encoding='utf-8') as f:
+        f.write(text)
+    saved.append((fname, len(text)))
+    return True
+
+
+async def scrape_realization_period(page, year, month):
+    out_dir = os.path.join(ROOT, 'realization', f'{year}-{month:02d}')
+    os.makedirs(out_dir, exist_ok=True)
+    saved = []
+    base = '/аналитика/api/rpp'
+    # RF-level total (no regionCode)
+    for ep in ('total', 'housing', 'readyYear'):
+        await fetch_save(page, f'{base}/{ep}?repMonth={month}&repYear={year}', out_dir, saved)
+    await fetch_save(
+        page,
+        f'{base}/developer?typeSquare=total&repMonth={month}&repYear={year}'
+        f'&developerOrder=totalSquare:desc',
+        out_dir, saved,
+    )
+    # Per-region
+    for rcode in REGIONS:
+        for ep in ('total', 'housing', 'readyYear'):
+            await fetch_save(
+                page,
+                f'{base}/{ep}?regionCode={rcode}&repMonth={month}&repYear={year}',
+                out_dir, saved,
+            )
+        for ts in ('total', 'living'):
+            await fetch_save(
+                page,
+                f'{base}/developer?typeSquare={ts}&regionCode={rcode}'
+                f'&repMonth={month}&repYear={year}&developerOrder=totalSquare:desc',
+                out_dir, saved,
+            )
+    print(f'  realization {year}-{month:02d}: {len(saved)} files')
+    return len(saved)
+
+
+async def scrape_sold_ready_period(page, year, month):
+    out_dir = os.path.join(ROOT, 'sold_ready', f'{year}-{month:02d}')
+    os.makedirs(out_dir, exist_ok=True)
+    saved = []
+    base = '/portal-analytics/api/ready-construction'
+    for ep_path in [
+        f'/index?repYear={year}&repMonth={month}',
+        f'/charts?repYear={year}&repMonth={month}',
+        f'/filters?repYear={year}&repMonth={month}',
+        f'/ready-year-charts?repYear={year}&repMonth={month}',
+        '/dynamics?dynamicChartType=readyPercChart,soldPercChart,soldReadyPercChart,squareSum',
+    ]:
+        await fetch_save(page, base + ep_path, out_dir, saved)
+    print(f'  sold_ready  {year}-{month:02d}: {len(saved)} files')
+    return len(saved)
+
+
+async def main():
+    os.makedirs(ROOT, exist_ok=True)
+    async with async_playwright() as pw:
+        browser = await pw.chromium.launch(headless=True)
+        ctx = await browser.new_context(
+            user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) '
+                       'AppleWebKit/537.36 (KHTML, like Gecko) '
+                       'Chrome/147.0.0.0 Safari/537.36',
+            locale='ru-RU',
+            viewport={'width': 1920, 'height': 1080},
+        )
+        page = await ctx.new_page()
+
+        # Warm up realization page first
+        print('Warming up: realization page')
+        await warm_up(page, '/аналитика/реализация_строящихся_квартир')
+
+        # Realization for all periods (uses /аналитика/api/rpp/*)
+        for year, month in PERIODS:
+            await scrape_realization_period(page, year, month)
+
+        # Switch to sold_ready warm-up
+        print('Warming up: sold_ready page')
+        await warm_up(page, '/аналитика/распроданность-стройготовность?repYear=2026&repMonth=3')
+        for year, month in PERIODS:
+            await scrape_sold_ready_period(page, year, month)
+
+        await browser.close()
+
+
+if __name__ == '__main__':
+    asyncio.run(main())
--- a/data/sql/41_load_history.py
+++ b/data/sql/41_load_history.py
@ -0,0 +1,283 @@
+"""Load historical scrape into prod (realization + sold_ready).
+
+Reads:  data/raw/domrf_history/<section>/<YYYY-MM>/*.json
+Writes: domrf_realization, domrf_sold_ready_index/breakdown/dynamics
+        + a fresh row in domrf_snapshots per period.
+
+SNAPSHOT_DATE for each period = last day of repMonth (so PK collisions are
+avoided across multiple historical pulls done on the same calendar day).
+"""
+import os, json, re, sys, subprocess, calendar, datetime
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+ROOT = os.path.join(HERE, '..', 'raw', 'domrf_history')
+PG_HOST = os.environ.get('PG_HOST', 'host.docker.internal')
+PG_PORT = os.environ.get('PG_PORT', '15432')
+PG_USER = os.environ.get('PG_USER', 'gendesign')
+PG_DB   = os.environ.get('PG_DB',   'gendesign')
+PG_PASS = os.environ.get('PGPASSWORD', '2J2SBPMKuS998fiwhtQqDhMI')
+
+
+def psql(sql):
+    cmd = ['docker', 'run', '--rm', '-i', '-e', f'PGPASSWORD={PG_PASS}',
+           'postgres:16-alpine', 'psql',
+           '-h', PG_HOST, '-p', PG_PORT, '-U', PG_USER, '-d', PG_DB,
+           '-v', 'ON_ERROR_STOP=1', '--quiet']
+    res = subprocess.run(cmd, input=sql, capture_output=True, text=True, encoding='utf-8')
+    if res.returncode != 0:
+        print('psql FAILED:', (res.stderr or '')[-2000:], file=sys.stderr)
+        raise SystemExit(res.returncode)
+    return res.stdout
+
+
+def esc(s):
+    if s is None:
+        return 'NULL'
+    return "'" + str(s).replace("'", "''") + "'"
+
+
+def num(v):
+    if v is None or v == '' or v == '-':
+        return 'NULL'
+    try:
+        f = float(v)
+        if f != f:
+            return 'NULL'
+        return repr(f)
+    except (ValueError, TypeError):
+        return 'NULL'
+
+
+def upsert_chunked(rows, table, cols, conflict_cols, batch=500):
+    if not rows:
+        return 0
+    update = ', '.join(f'{c}=EXCLUDED.{c}' for c in cols if c not in conflict_cols)
+    total = 0
+    for i in range(0, len(rows), batch):
+        chunk = rows[i:i + batch]
+        sql = (f"INSERT INTO {table} ({','.join(cols)}) VALUES "
+               + ','.join(chunk)
+               + (f" ON CONFLICT ({','.join(conflict_cols)}) DO UPDATE SET {update};"
+                  if update else f" ON CONFLICT ({','.join(conflict_cols)}) DO NOTHING;"))
+        psql(sql)
+        total += len(chunk)
+    return total
+
+
+# ── REALIZATION ─────────────────────────────────────────────────────────────
+RPP_FNAME_RE = re.compile(
+    r'rpp_(total|housing|readyYear|developer)__'
+    r'(?:typeSquare_(total|living)_)?'
+    r'(?:regionCode_(\d+)_)?'
+    r'repMonth_(\d+)_repYear_(\d+)'
+)
+
+
+def load_realization(period_dir, snap):
+    cols = ['snapshot_date', 'endpoint_type', 'region_code', 'rep_year', 'rep_month',
+            'type_square', 'subject', 'subject_desc', 'share_total',
+            'total_square', 'open_square', 'sold_square', 'unsold_square', 'unopened_square',
+            'sold_perc', 'unsold_perc', 'unopened_perc', 'sold_amount', 'price_avg']
+    rows, seen = [], set()
+    for fname in sorted(os.listdir(period_dir)):
+        if not fname.endswith('.json') or not fname.startswith('rpp_'):
+            continue
+        m = RPP_FNAME_RE.match(fname)
+        if not m:
+            continue
+        ep_raw, ts_filename, rcode_filename, rmonth, ryear = m.groups()
+        endpoint_type = {'total': 'total', 'housing': 'housing',
+                         'readyYear': 'ready_year', 'developer': 'developer'}[ep_raw]
+        region_code = int(rcode_filename) if rcode_filename else 0
+        try:
+            doc = json.load(open(os.path.join(period_dir, fname), encoding='utf-8'))
+        except Exception:
+            continue
+        for it in doc.get('data') or []:
+            ts = ts_filename or it.get('typeSquare') or 'total'
+            subject = str(it.get('subject', ''))
+            if not subject:
+                continue
+            key = (endpoint_type, region_code, ts, subject)
+            if key in seen:
+                continue
+            seen.add(key)
+            rows.append(
+                f"({esc(snap)},{esc(endpoint_type)},{region_code},"
+                f"{int(ryear)},{int(rmonth)},{esc(ts)},{esc(subject)},"
+                f"{esc(it.get('subjectDesc'))},{num(it.get('shareTotal'))},"
+                f"{num(it.get('totalSquare'))},{num(it.get('openSquare'))},"
+                f"{num(it.get('soldSquare'))},{num(it.get('unsoldSquare'))},"
+                f"{num(it.get('unopenedSquare'))},{num(it.get('soldPerc'))},"
+                f"{num(it.get('unsoldPerc'))},{num(it.get('unopenedPerc'))},"
+                f"{num(it.get('soldAmount'))},{num(it.get('priceAvg'))})"
+            )
+    return upsert_chunked(
+        rows, 'domrf_realization', cols,
+        conflict_cols=['snapshot_date', 'endpoint_type', 'region_code', 'type_square', 'subject'],
+    )
+
+
+# ── SOLD_READY ──────────────────────────────────────────────────────────────
+def load_sold_ready(period_dir, snap, year, month):
+    n_idx = n_brk = n_dyn = 0
+    # index file
+    for fname in os.listdir(period_dir):
+        path = os.path.join(period_dir, fname)
+        if not fname.startswith('ready-construction_index') or not fname.endswith('.json'):
+            continue
+        try:
+            d = json.load(open(path, encoding='utf-8'))
+        except Exception:
+            continue
+        row = (
+            f"({esc(snap)},{int(year)},{int(month)},"
+            f"{num(d.get('squareSumIndex'))},{num(d.get('soldPercIndex'))},"
+            f"{num(d.get('soldSumIndex'))},{num(d.get('soldReadyPercIndex'))},"
+            f"{num(d.get('readyPercIndex'))})"
+        )
+        n_idx = upsert_chunked(
+            [row], 'domrf_sold_ready_index',
+            ['snapshot_date', 'rep_year', 'rep_month',
+             'square_sum', 'sold_perc', 'sold_sum', 'sold_ready_perc', 'ready_perc'],
+            conflict_cols=['snapshot_date', 'rep_year', 'rep_month'],
+        )
+    # charts → breakdown (shape: {"charts":[{"chartType":"...","data":[...]}]} )
+    # Each chart_type has exactly one discriminator field — extracted to its
+    # own typed column instead of dumped into JSONB.
+    chart_rows = []
+    seen_keys = set()
+    for fname in os.listdir(period_dir):
+        if not (fname.startswith('ready-construction_charts')
+                or fname.startswith('ready-construction_ready-year-charts')) or not fname.endswith('.json'):
+            continue
+        try:
+            d = json.load(open(os.path.join(period_dir, fname), encoding='utf-8'))
+        except Exception:
+            continue
+        # Format A: {"charts":[{"chartType":"foChart","data":[...]}, ...]}
+        for chart_obj in (d.get('charts') or []):
+            chart_type = chart_obj.get('chartType')
+            if not chart_type:
+                continue
+            for it in chart_obj.get('data') or []:
+                if not isinstance(it, dict):
+                    continue
+                key = it.get('key') or it.get('id') or it.get('subject') or it.get('name') or ''
+                if not key:
+                    continue
+                pk = (chart_type, str(key))
+                if pk in seen_keys:
+                    continue
+                seen_keys.add(pk)
+                chart_rows.append(
+                    f"({esc(snap)},{int(year)},{int(month)},{esc(chart_type)},{esc(str(key))},"
+                    f"{num(it.get('squareSum'))},{num(it.get('soldPerc'))},"
+                    f"{num(it.get('readyPerc'))},{num(it.get('soldReadyPerc'))},"
+                    f"{esc(it.get('foCd'))},"
+                    f"{esc(it.get('regionCd'))},"
+                    f"{esc(it.get('devCalcGroupId'))},"
+                    f"{esc(it.get('devGroupValue'))},"
+                    f"{esc(it.get('cityPopulation'))},"
+                    f"{esc(it.get('objClass'))},"
+                    f"NULL)"  # ready_year — only for readyYearChart format below
+                )
+        # Format B: {"readyYears":[{"readyYear":2030,...}]} → readyYearChart
+        for it in (d.get('readyYears') or []):
+            if not isinstance(it, dict):
+                continue
+            ry = it.get('readyYear')
+            if ry is None:
+                continue
+            key = str(ry)
+            pk = ('readyYearChart', key)
+            if pk in seen_keys:
+                continue
+            seen_keys.add(pk)
+            chart_rows.append(
+                f"({esc(snap)},{int(year)},{int(month)},{esc('readyYearChart')},{esc(key)},"
+                f"{num(it.get('squareSum'))},{num(it.get('soldPerc'))},"
+                f"{num(it.get('readyPerc'))},{num(it.get('soldReadyPerc'))},"
+                f"NULL,NULL,NULL,NULL,NULL,NULL,{int(ry)})"
+            )
+    n_brk = upsert_chunked(
+        chart_rows, 'domrf_sold_ready_breakdown',
+        ['snapshot_date', 'rep_year', 'rep_month', 'chart_type', 'entity_key',
+         'square_sum', 'sold_perc', 'ready_perc', 'sold_ready_perc',
+         'fo_cd', 'region_cd', 'dev_calc_group_id', 'dev_group_value',
+         'city_population', 'obj_class', 'ready_year'],
+        conflict_cols=['snapshot_date', 'rep_year', 'rep_month', 'chart_type', 'entity_key'],
+    )
+    # dynamics: {"dynamicCharts":[{"dynamicChartType":"soldPercChart",
+    #   "repYears":[{"repYear":2020,"repMonths":[{"repMonth":1,"value":41.54}]}]}]}
+    dyn_rows = []
+    dyn_seen = set()
+    for fname in os.listdir(period_dir):
+        if not fname.startswith('ready-construction_dynamics') or not fname.endswith('.json'):
+            continue
+        try:
+            d = json.load(open(os.path.join(period_dir, fname), encoding='utf-8'))
+        except Exception:
+            continue
+        for chart_obj in (d.get('dynamicCharts') or []):
+            ch_type = chart_obj.get('dynamicChartType')
+            if not ch_type:
+                continue
+            for ry_obj in chart_obj.get('repYears') or []:
+                ry = ry_obj.get('repYear')
+                if ry is None:
+                    continue
+                for pt in ry_obj.get('repMonths') or []:
+                    rm = pt.get('repMonth')
+                    if rm is None:
+                        continue
+                    pk = (ch_type, ry, rm)
+                    if pk in dyn_seen:
+                        continue
+                    dyn_seen.add(pk)
+                    dyn_rows.append(
+                        f"({esc(snap)},{esc(ch_type)},{int(ry)},{int(rm)},{num(pt.get('value'))})"
+                    )
+    n_dyn = upsert_chunked(
+        dyn_rows, 'domrf_sold_ready_dynamics',
+        ['snapshot_date', 'dynamic_chart_type', 'rep_year', 'rep_month', 'value'],
+        conflict_cols=['snapshot_date', 'dynamic_chart_type', 'rep_year', 'rep_month'],
+    )
+    return n_idx, n_brk, n_dyn
+
+
+def main():
+    if not os.path.isdir(ROOT):
+        print(f'{ROOT} not found, run 40_scrape_history.py first')
+        return
+
+    # Discover periods from realization subdirs
+    realization_root = os.path.join(ROOT, 'realization')
+    periods = sorted(
+        d for d in os.listdir(realization_root)
+        if re.match(r'^\d{4}-\d{2}$', d) and os.path.isdir(os.path.join(realization_root, d))
+    )
+    print(f'Periods to load: {periods}')
+
+    for period in periods:
+        year, month = map(int, period.split('-'))
+        last_day = calendar.monthrange(year, month)[1]
+        snap = f'{year:04d}-{month:02d}-{last_day:02d}'
+
+        # Ensure parent snapshot row
+        psql(f"INSERT INTO domrf_snapshots(snapshot_date) VALUES ({esc(snap)}) ON CONFLICT DO NOTHING;")
+
+        period_dir = os.path.join(realization_root, period)
+        n_real = load_realization(period_dir, snap)
+
+        sr_dir = os.path.join(ROOT, 'sold_ready', period)
+        n_idx = n_brk = n_dyn = 0
+        if os.path.isdir(sr_dir):
+            n_idx, n_brk, n_dyn = load_sold_ready(sr_dir, snap, year, month)
+
+        print(f'{period} (snap={snap}):  realization={n_real}  '
+              f'sr_index={n_idx} sr_breakdown={n_brk} sr_dynamics={n_dyn}')
+
+
+if __name__ == '__main__':
+    main()
--- a/data/sql/42_schema_breakdown_widen.sql
+++ b/data/sql/42_schema_breakdown_widen.sql
@ -0,0 +1,37 @@
+-- Replace JSONB `extra` column in domrf_sold_ready_breakdown with explicit
+-- typed columns. Each chart_type has exactly one discriminator field:
+--   • foChart            → fo_cd (already existed)
+--   • regionChart        → region_cd
+--   • devChart           → dev_calc_group_id (e.g. '6072_0' = Самолет)
+--   • devValueChart      → dev_group_value  ('do10','do25','do50',...)
+--   • cityPopulationChart→ city_population  ('do50','do100','do250',...)
+--   • classChart         → obj_class        ('typical','comfort','business','elite')
+--   • readyYearChart     → ready_year (INT, plan year)
+-- The extra JSONB was a dumping ground for these — split them out.
+
+BEGIN;
+
+ALTER TABLE domrf_sold_ready_breakdown
+    ADD COLUMN IF NOT EXISTS region_cd          INT,
+    ADD COLUMN IF NOT EXISTS dev_calc_group_id  TEXT,
+    ADD COLUMN IF NOT EXISTS dev_group_value    TEXT,
+    ADD COLUMN IF NOT EXISTS city_population    TEXT,
+    ADD COLUMN IF NOT EXISTS obj_class          TEXT,
+    ADD COLUMN IF NOT EXISTS ready_year         INT;
+
+ALTER TABLE domrf_sold_ready_breakdown DROP COLUMN IF EXISTS extra;
+
+COMMENT ON COLUMN domrf_sold_ready_breakdown.region_cd IS
+    'Rosstat region code, populated for chart_type=regionChart';
+COMMENT ON COLUMN domrf_sold_ready_breakdown.dev_calc_group_id IS
+    'Developer ID (matches domrf_developers.developer_id), for chart_type=devChart';
+COMMENT ON COLUMN domrf_sold_ready_breakdown.dev_group_value IS
+    'Size bucket key for chart_type=devValueChart (do10/do25/do50/...)';
+COMMENT ON COLUMN domrf_sold_ready_breakdown.city_population IS
+    'Population bucket key for chart_type=cityPopulationChart (do50/do100/...)';
+COMMENT ON COLUMN domrf_sold_ready_breakdown.obj_class IS
+    'Housing class for chart_type=classChart (typical/comfort/business/elite)';
+COMMENT ON COLUMN domrf_sold_ready_breakdown.ready_year IS
+    'Plan-completion year for chart_type=readyYearChart';
+
+COMMIT;
--- a/data/sql/43_anton_import.sql
+++ b/data/sql/43_anton_import.sql
@ -0,0 +1,193 @@
+-- Import unique tables from Anton's domrf.db (SQLite, 23.5MB).
+-- Skipped duplicates: regions, federal_districts, mortgage_rates (empty),
+-- escrow_banks, rc_classes/dev_size/city_pop, rc_dynamics_rf, egrn_deals
+-- (we have raw rosreestr_deals — 4.97M rows, 32× larger).
+-- developers (8 618) handled separately via merge into domrf_developers.
+
+-- ── DOM.РФ /сервисы/api/kn/* — поквартирный прайс + ЖК ─────────────────────
+CREATE TABLE IF NOT EXISTS domrf_kn_flats (
+    id              BIGINT PRIMARY KEY,
+    ods_id          TEXT,
+    flat_type       TEXT,
+    flat_number     TEXT,
+    is_studio       BOOLEAN,
+    total_area      NUMERIC,
+    living_area     NUMERIC,
+    rooms           INT,
+    status          TEXT,                -- free | booked | sold
+    price_rub       BIGINT,
+    price_per_m2    NUMERIC,
+    floor           INT,
+    num_floors      INT,
+    obj_id          BIGINT,              -- FK target → domrf_kn_objects.obj_id
+    city            TEXT,
+    region_cd       INT,
+    obj_name        TEXT,
+    snapshot_date   DATE NOT NULL DEFAULT CURRENT_DATE
+);
+CREATE INDEX IF NOT EXISTS idx_kn_flats_obj    ON domrf_kn_flats (obj_id);
+CREATE INDEX IF NOT EXISTS idx_kn_flats_status ON domrf_kn_flats (status);
+CREATE INDEX IF NOT EXISTS idx_kn_flats_region ON domrf_kn_flats (region_cd);
+CREATE INDEX IF NOT EXISTS idx_kn_flats_rooms  ON domrf_kn_flats (rooms);
+
+CREATE TABLE IF NOT EXISTS domrf_kn_objects (
+    obj_id          BIGINT PRIMARY KEY,
+    hobj_id         BIGINT,
+    comm_name       TEXT,
+    addr            TEXT,
+    short_addr      TEXT,
+    region_cd       INT,
+    dev_id          TEXT,
+    dev_name        TEXT,
+    dev_inn         TEXT,
+    floor_min       INT,
+    floor_max       INT,
+    flat_count      INT,
+    square_living   NUMERIC,
+    ready_dt        DATE,
+    problem_flag    TEXT,
+    site_status     TEXT,
+    green_house     BOOLEAN,
+    escrow          BOOLEAN,
+    obj_class       TEXT,
+    wall_type       TEXT,
+    energy_eff      TEXT,
+    latitude        NUMERIC,
+    longitude       NUMERIC,
+    obj_status      INT,
+    is_prinzip      BOOLEAN DEFAULT FALSE,
+    is_ekb          BOOLEAN DEFAULT FALSE,
+    is_problem      BOOLEAN DEFAULT FALSE,
+    snapshot_date   DATE NOT NULL DEFAULT CURRENT_DATE
+);
+CREATE INDEX IF NOT EXISTS idx_kn_obj_dev      ON domrf_kn_objects (dev_id);
+CREATE INDEX IF NOT EXISTS idx_kn_obj_region   ON domrf_kn_objects (region_cd);
+CREATE INDEX IF NOT EXISTS idx_kn_obj_prinzip  ON domrf_kn_objects (is_prinzip);
+CREATE INDEX IF NOT EXISTS idx_kn_obj_ekb      ON domrf_kn_objects (is_ekb);
+
+-- ── ЦБ РФ ипотечная статистика (cbr.ru/statistics/bank_sector/mortgage/) ──
+CREATE TABLE IF NOT EXISTS cbr_mortgage_files (
+    filename     TEXT PRIMARY KEY,
+    size_kb      INT,
+    description  TEXT,
+    url          TEXT
+);
+
+CREATE TABLE IF NOT EXISTS cbr_mortgage_series (
+    series_id    TEXT NOT NULL,            -- e.g. '02_05_Debt_ind_в_рублях'
+    region       TEXT NOT NULL,            -- 'rf' | 'sverdl'
+    period       TEXT NOT NULL,            -- 'Feb 2019', 'Mar 2026', ...
+    value        NUMERIC,
+    title        TEXT,
+    PRIMARY KEY (series_id, region, period)
+);
+CREATE INDEX IF NOT EXISTS idx_cbr_period ON cbr_mortgage_series (period);
+CREATE INDEX IF NOT EXISTS idx_cbr_region ON cbr_mortgage_series (region);
+
+-- ── Сбер ДомКлик DDF (PDF nov2025) — ключевые метрики ────────────────────
+CREATE TABLE IF NOT EXISTS sber_ddf_metrics (
+    metric_key   TEXT PRIMARY KEY,
+    region       TEXT,
+    value_2019   NUMERIC,
+    value_2025   NUMERIC,
+    unit         TEXT,
+    source       TEXT
+);
+
+-- ── Yandex Realty ЕКБ ─────────────────────────────────────────────────────
+CREATE TABLE IF NOT EXISTS yandex_realty_zk (
+    yid             BIGINT PRIMARY KEY,
+    name            TEXT,
+    developer       TEXT,
+    developer_id    BIGINT,
+    finished_obj    INT,
+    unfinished_obj  INT,
+    price_from      NUMERIC,
+    price_to        NUMERIC,
+    address         TEXT,
+    obj_class       TEXT,
+    latitude        NUMERIC,
+    longitude       NUMERIC,
+    snapshot_date   DATE NOT NULL DEFAULT CURRENT_DATE
+);
+
+CREATE TABLE IF NOT EXISTS yandex_realty_class_prices (
+    obj_class           TEXT PRIMARY KEY,
+    total_sites         INT,
+    price_per_m2_min    NUMERIC,
+    price_per_m2_max    NUMERIC,
+    snapshot_date       DATE NOT NULL DEFAULT CURRENT_DATE
+);
+
+-- ── Geo Свердл/ЕКБ ─────────────────────────────────────────────────────────
+CREATE TABLE IF NOT EXISTS ekb_districts (
+    district_name        TEXT PRIMARY KEY,
+    population           INT,
+    zk_count             INT,
+    flat_count           INT,
+    area_m2              NUMERIC,
+    median_price_per_m2  NUMERIC,
+    mean_price_per_m2    NUMERIC
+);
+
+CREATE TABLE IF NOT EXISTS ekb_metro (
+    station_name         TEXT PRIMARY KEY,
+    zk_count             INT,
+    median_price_per_m2  NUMERIC
+);
+
+CREATE TABLE IF NOT EXISTS sv_cities (
+    city_name            TEXT PRIMARY KEY,
+    zk_count             INT,
+    flat_count           INT,
+    area_m2              NUMERIC,
+    median_price_per_m2  NUMERIC
+);
+
+-- ── Pre-aggregated Свердл developer rankings (топ-50, март 2026) ──────────
+CREATE TABLE IF NOT EXISTS sv_dev_sales (
+    rank              INT,
+    dev_id            TEXT NOT NULL,
+    name              TEXT,
+    total_square      INT,
+    sold_perc         NUMERIC,
+    unsold_perc       NUMERIC,
+    unopened_perc     NUMERIC,
+    rep_period        TEXT NOT NULL,         -- 'YYYY-MM' (e.g. '2026-03')
+    PRIMARY KEY (dev_id, rep_period)
+);
+
+-- ── Developer growth 2024→2025 (РФ + Свердл) ───────────────────────────────
+CREATE TABLE IF NOT EXISTS domrf_dev_growth (
+    id                BIGSERIAL PRIMARY KEY,
+    dev_id            TEXT,
+    dev_name          TEXT NOT NULL,
+    area_2024         NUMERIC,
+    area_2025         NUMERIC,
+    delta_pct         NUMERIC,
+    region            TEXT,                 -- 'rf' | 'sverdl'
+    UNIQUE (dev_name, region)
+);
+
+-- ── Росреестр rent deals Свердл (1071 договоров аренды) ──────────────────
+CREATE TABLE IF NOT EXISTS rosreestr_rent_deals (
+    id                BIGSERIAL PRIMARY KEY,
+    quarter           TEXT NOT NULL,        -- 'Q1 2026'
+    obj_type          TEXT,                 -- '002001001000' и т.п.
+    cad_quarter       TEXT,                 -- '66:02:1401003'
+    area              NUMERIC,
+    rent_rub          NUMERIC,
+    rent_per_m2       NUMERIC,
+    duration_months   INT,
+    region_cd         INT DEFAULT 66
+);
+CREATE INDEX IF NOT EXISTS idx_rent_quarter   ON rosreestr_rent_deals (quarter);
+CREATE INDEX IF NOT EXISTS idx_rent_cadq      ON rosreestr_rent_deals (cad_quarter);
+CREATE INDEX IF NOT EXISTS idx_rent_region    ON rosreestr_rent_deals (region_cd);
+
+COMMENT ON TABLE domrf_kn_flats IS 'DOM.РФ /сервисы/api/kn/flats per-flat catalog. 27 563 квартир РФ status=free/booked/sold с ценами. Источник: Anton domrf.db imported 27.04.2026.';
+COMMENT ON TABLE domrf_kn_objects IS 'DOM.РФ /сервисы/api/kn/object — реестр ЖК Свердл. 1 516 объектов с GPS, классом, эскроу-флагом. is_prinzip flag для 28 PRINZIP ЖК. Источник: Anton.';
+COMMENT ON TABLE cbr_mortgage_series IS 'ЦБ РФ статистика по ипотеке (XLSX cbr.ru/statistics/bank_sector/mortgage/). 47 серий × {rf|sverdl} × месяцы 2019-2026. Свердл Фев-2026: ставка 11%, +318 б.п. за месяц.';
+COMMENT ON TABLE sber_ddf_metrics IS 'Сбер ДомКлик DDF nov2025 PDF: affordability, demand index, миграция, повторные ипотечники, price gap 79/21%, quality shift.';
+COMMENT ON TABLE yandex_realty_zk IS 'Yandex Realty snapshot — 12 ЖК ЕКБ из INITIAL_STATE с классами/ценами. Полный snapshot пользовательских listings.';
+COMMENT ON TABLE rosreestr_rent_deals IS 'Договоры аренды Свердл Q1 2026 (1071) из Росреестра data-sets.';
--- a/data/sql/44_import_anton_db.py
+++ b/data/sql/44_import_anton_db.py
@ -0,0 +1,298 @@
+"""Migrate Anton's domrf.db (SQLite, 23.5 MB) into prod Postgres.
+
+Reads:  C:\\Users\\user\\Downloads\\Telegram Desktop\\db\\db\\domrf.db
+Writes: 13 new tables created by 43_anton_import.sql + merges developers.
+
+Skipped tables (duplicates of our existing data):
+  • regions, federal_districts (we have domrf_regions/_federal_districts)
+  • mortgage_rates (empty in Anton's DB, we have domrf_mortgage_rates)
+  • escrow_banks (we have domrf_escrow_banks, 28 vs 27)
+  • rc_classes/dev_size/city_pop (in our domrf_sold_ready_breakdown)
+  • rc_dynamics_rf (in our domrf_sold_ready_dynamics)
+  • egrn_deals (we have raw rosreestr_deals — 4.97M rows, far richer)
+  • _meta, _sources, sqlite_sequence, launch_5y(empty)
+
+Special handling:
+  • developers (8 618) — merge missing rows into domrf_developers (we have 2 734)
+"""
+import os, sys, sqlite3, subprocess, datetime
+
+SQLITE_PATH = r"C:\Users\user\Downloads\Telegram Desktop\db\db\domrf.db"
+PG_HOST = os.environ.get('PG_HOST', 'host.docker.internal')
+PG_PORT = os.environ.get('PG_PORT', '15432')
+PG_USER = os.environ.get('PG_USER', 'gendesign')
+PG_DB   = os.environ.get('PG_DB',   'gendesign')
+PG_PASS = os.environ.get('PGPASSWORD', '2J2SBPMKuS998fiwhtQqDhMI')
+
+
+def psql(sql):
+    cmd = ['docker', 'run', '--rm', '-i', '-e', f'PGPASSWORD={PG_PASS}',
+           'postgres:16-alpine', 'psql',
+           '-h', PG_HOST, '-p', PG_PORT, '-U', PG_USER, '-d', PG_DB,
+           '-v', 'ON_ERROR_STOP=1', '--quiet']
+    res = subprocess.run(cmd, input=sql, capture_output=True, text=True, encoding='utf-8')
+    if res.returncode != 0:
+        print('psql FAILED:', (res.stderr or '')[-2000:], file=sys.stderr)
+        raise SystemExit(res.returncode)
+    return res.stdout
+
+
+def esc_val(v):
+    if v is None:
+        return 'NULL'
+    if isinstance(v, bool):
+        return 'TRUE' if v else 'FALSE'
+    if isinstance(v, (int, float)):
+        if isinstance(v, float) and v != v:  # NaN
+            return 'NULL'
+        return str(v)
+    return "'" + str(v).replace("'", "''") + "'"
+
+
+def to_pg_bool(v):
+    if v is None:
+        return 'NULL'
+    return 'TRUE' if v else 'FALSE'
+
+
+def to_pg_date(v):
+    """SQLite stores dates as TEXT. Try parse YYYY-MM-DD; else NULL."""
+    if v is None or v == '':
+        return 'NULL'
+    s = str(v)[:10]
+    try:
+        datetime.date.fromisoformat(s)
+        return f"'{s}'"
+    except Exception:
+        return 'NULL'
+
+
+def chunked_insert(rows, table, cols, conflict_cols=None, batch=500, do_update=False):
+    if not rows:
+        return 0
+    total = 0
+    on_conflict = ''
+    if conflict_cols:
+        if do_update:
+            update = ', '.join(f'{c}=EXCLUDED.{c}' for c in cols if c not in conflict_cols)
+            on_conflict = f' ON CONFLICT ({",".join(conflict_cols)}) DO UPDATE SET {update}'
+        else:
+            on_conflict = f' ON CONFLICT ({",".join(conflict_cols)}) DO NOTHING'
+    for i in range(0, len(rows), batch):
+        chunk = rows[i:i + batch]
+        sql = (f"INSERT INTO {table} ({','.join(cols)}) VALUES "
+               + ','.join(chunk) + on_conflict + ';')
+        psql(sql)
+        total += len(chunk)
+    return total
+
+
+def migrate_kn_flats(con):
+    rows = []
+    for r in con.execute('SELECT id, ods_id, type, number, is_studio, total_area, '
+                         'living_area, rooms, status, price_rub, price_per_m2, '
+                         'floor, num_floors, obj_id, city, region_cd, obj_name '
+                         'FROM flats'):
+        (fid, ods, ftype, fnum, studio, ta, la, rooms, status, pr, ppm,
+         fl, nfl, obj_id, city, region, obj_name) = r
+        rows.append(
+            f"({fid},{esc_val(ods)},{esc_val(ftype)},{esc_val(fnum)},"
+            f"{to_pg_bool(studio)},{esc_val(ta)},{esc_val(la)},{esc_val(rooms)},"
+            f"{esc_val(status)},{esc_val(pr)},{esc_val(ppm)},{esc_val(fl)},"
+            f"{esc_val(nfl)},{esc_val(obj_id)},{esc_val(city)},{esc_val(region)},"
+            f"{esc_val(obj_name)},DEFAULT)"
+        )
+    return chunked_insert(
+        rows, 'domrf_kn_flats',
+        ['id','ods_id','flat_type','flat_number','is_studio','total_area','living_area',
+         'rooms','status','price_rub','price_per_m2','floor','num_floors','obj_id',
+         'city','region_cd','obj_name','snapshot_date'],
+        conflict_cols=['id'], do_update=True,
+    )
+
+
+def migrate_kn_objects(con):
+    rows = []
+    for r in con.execute('SELECT obj_id, hobj_id, comm_name, addr, short_addr, region_cd, '
+                         'dev_id, dev_name, dev_inn, floor_min, floor_max, flat_count, '
+                         'square_living, ready_dt, problem_flag, site_status, green_house, '
+                         'escrow, obj_class, wall_type, energy_eff, latitude, longitude, '
+                         'obj_status, is_prinzip, is_ekb, is_problem FROM objects'):
+        (oid, hid, cn, addr, sa, rc, did, dn, dinn, fmn, fmx, fc, sl, rdt,
+         pf, ss, gh, esc, oc, wt, ee, lat, lon, ostat, ipr, iekb, iprob) = r
+        rows.append(
+            f"({oid},{esc_val(hid)},{esc_val(cn)},{esc_val(addr)},{esc_val(sa)},"
+            f"{esc_val(rc)},{esc_val(did)},{esc_val(dn)},{esc_val(dinn)},"
+            f"{esc_val(fmn)},{esc_val(fmx)},{esc_val(fc)},{esc_val(sl)},"
+            f"{to_pg_date(rdt)},{esc_val(pf)},{esc_val(ss)},"
+            f"{to_pg_bool(gh)},{to_pg_bool(esc)},{esc_val(oc)},{esc_val(wt)},"
+            f"{esc_val(ee)},{esc_val(lat)},{esc_val(lon)},{esc_val(ostat)},"
+            f"{to_pg_bool(ipr)},{to_pg_bool(iekb)},{to_pg_bool(iprob)},DEFAULT)"
+        )
+    return chunked_insert(
+        rows, 'domrf_kn_objects',
+        ['obj_id','hobj_id','comm_name','addr','short_addr','region_cd','dev_id','dev_name',
+         'dev_inn','floor_min','floor_max','flat_count','square_living','ready_dt',
+         'problem_flag','site_status','green_house','escrow','obj_class','wall_type',
+         'energy_eff','latitude','longitude','obj_status','is_prinzip','is_ekb','is_problem',
+         'snapshot_date'],
+        conflict_cols=['obj_id'], do_update=True,
+    )
+
+
+def migrate_simple(con, src, dst, cols_src, cols_dst, conflict_cols, do_update=True,
+                   transforms=None):
+    """Generic migration: SELECT cols from src → INSERT into dst."""
+    transforms = transforms or {}
+    rows = []
+    for r in con.execute(f'SELECT {",".join(cols_src)} FROM "{src}"'):
+        vals = []
+        for col_name, raw in zip(cols_src, r):
+            if col_name in transforms:
+                vals.append(transforms[col_name](raw))
+            else:
+                vals.append(esc_val(raw))
+        rows.append('(' + ','.join(vals) + ')')
+    return chunked_insert(rows, dst, cols_dst, conflict_cols=conflict_cols, do_update=do_update)
+
+
+def merge_developers(con):
+    """Anton has 8 618 developers (incl. archived); we have 2 734.
+    INSERT only IDs that don't exist in our table — no update on existing."""
+    # Pull existing developer_ids from prod via psql
+    existing_ids = set()
+    res = psql('SELECT developer_id FROM domrf_developers;')
+    for line in res.splitlines():
+        line = line.strip()
+        if line and not line.startswith('---') and 'developer_id' not in line and 'rows' not in line and '(' not in line:
+            existing_ids.add(line)
+    print(f'  existing developers in prod: {len(existing_ids)}')
+
+    new_rows = []
+    skipped = 0
+    cur = con.execute('SELECT id, name FROM developers WHERE id IS NOT NULL')
+    for did, name in cur:
+        if not did or not name:
+            continue
+        if str(did) in existing_ids:
+            skipped += 1
+            continue
+        new_rows.append(f"({esc_val(did)},{esc_val(name)})")
+    print(f'  skipped (already in prod): {skipped}')
+    print(f'  to INSERT (new from Anton): {len(new_rows)}')
+
+    return chunked_insert(
+        new_rows, 'domrf_developers', ['developer_id', 'developer_name'],
+        conflict_cols=['developer_id'], do_update=False,
+    )
+
+
+def main():
+    con = sqlite3.connect(SQLITE_PATH)
+
+    print('--- domrf_kn_flats ---')
+    n = migrate_kn_flats(con)
+    print(f'  loaded {n} flats')
+
+    print('--- domrf_kn_objects ---')
+    n = migrate_kn_objects(con)
+    print(f'  loaded {n} objects')
+
+    print('--- cbr_mortgage_files ---')
+    n = migrate_simple(con, 'cbr_files', 'cbr_mortgage_files',
+        ['filename', 'size_kb', 'description', 'url'],
+        ['filename', 'size_kb', 'description', 'url'],
+        conflict_cols=['filename'])
+    print(f'  loaded {n}')
+
+    print('--- cbr_mortgage_series ---')
+    n = migrate_simple(con, 'cbr_series', 'cbr_mortgage_series',
+        ['series_id', 'region', 'period', 'value', 'title'],
+        ['series_id', 'region', 'period', 'value', 'title'],
+        conflict_cols=['series_id', 'region', 'period'])
+    print(f'  loaded {n}')
+
+    print('--- sber_ddf_metrics ---')
+    n = migrate_simple(con, 'ddf_metrics', 'sber_ddf_metrics',
+        ['metric_key', 'region', 'value_2019', 'value_2025', 'unit', 'source'],
+        ['metric_key', 'region', 'value_2019', 'value_2025', 'unit', 'source'],
+        conflict_cols=['metric_key'])
+    print(f'  loaded {n}')
+
+    print('--- yandex_realty_zk ---')
+    n = migrate_simple(con, 'yandex_zk', 'yandex_realty_zk',
+        ['yid', 'name', 'developer', 'developer_id', 'finished_obj', 'unfinished_obj',
+         'price_from', 'price_to', 'address', 'obj_class', 'latitude', 'longitude'],
+        ['yid', 'name', 'developer', 'developer_id', 'finished_obj', 'unfinished_obj',
+         'price_from', 'price_to', 'address', 'obj_class', 'latitude', 'longitude'],
+        conflict_cols=['yid'])
+    print(f'  loaded {n}')
+
+    print('--- yandex_realty_class_prices ---')
+    n = migrate_simple(con, 'yandex_class_prices', 'yandex_realty_class_prices',
+        ['obj_class', 'total_sites', 'price_per_m2_min', 'price_per_m2_max'],
+        ['obj_class', 'total_sites', 'price_per_m2_min', 'price_per_m2_max'],
+        conflict_cols=['obj_class'])
+    print(f'  loaded {n}')
+
+    print('--- ekb_districts ---')
+    n = migrate_simple(con, 'ekb_districts', 'ekb_districts',
+        ['name', 'population', 'zk_count', 'flat_count', 'area_m2',
+         'median_price_per_m2', 'mean_price_per_m2'],
+        ['district_name', 'population', 'zk_count', 'flat_count', 'area_m2',
+         'median_price_per_m2', 'mean_price_per_m2'],
+        conflict_cols=['district_name'])
+    print(f'  loaded {n}')
+
+    print('--- ekb_metro ---')
+    n = migrate_simple(con, 'ekb_metro', 'ekb_metro',
+        ['station_name', 'zk_count', 'median_price_per_m2'],
+        ['station_name', 'zk_count', 'median_price_per_m2'],
+        conflict_cols=['station_name'])
+    print(f'  loaded {n}')
+
+    print('--- sv_cities ---')
+    n = migrate_simple(con, 'sv_cities', 'sv_cities',
+        ['city_name', 'zk_count', 'flat_count', 'area_m2', 'median_price_per_m2'],
+        ['city_name', 'zk_count', 'flat_count', 'area_m2', 'median_price_per_m2'],
+        conflict_cols=['city_name'])
+    print(f'  loaded {n}')
+
+    print('--- sv_dev_sales ---')
+    n = migrate_simple(con, 'sv_dev_sales', 'sv_dev_sales',
+        ['rank', 'dev_id', 'name', 'total_square', 'sold_perc', 'unsold_perc',
+         'unopened_perc', 'rep_period'],
+        ['rank', 'dev_id', 'name', 'total_square', 'sold_perc', 'unsold_perc',
+         'unopened_perc', 'rep_period'],
+        conflict_cols=['dev_id', 'rep_period'])
+    print(f'  loaded {n}')
+
+    print('--- domrf_dev_growth ---')
+    n = migrate_simple(con, 'dev_growth', 'domrf_dev_growth',
+        ['dev_id', 'dev_name', 'area_2024', 'area_2025', 'delta_pct', 'region'],
+        ['dev_id', 'dev_name', 'area_2024', 'area_2025', 'delta_pct', 'region'],
+        conflict_cols=['dev_name', 'region'])
+    print(f'  loaded {n}')
+
+    print('--- rosreestr_rent_deals ---')
+    rows = []
+    for r in con.execute('SELECT quarter, obj_type, cad_quarter, area, rent_rub, '
+                         'rent_per_m2, duration_months, region_cd FROM rent_deals'):
+        rows.append('(' + ','.join(esc_val(v) for v in r) + ')')
+    n = chunked_insert(
+        rows, 'rosreestr_rent_deals',
+        ['quarter','obj_type','cad_quarter','area','rent_rub','rent_per_m2',
+         'duration_months','region_cd'],
+        conflict_cols=None,  # no PK conflict — id is BIGSERIAL
+    )
+    print(f'  loaded {n}')
+
+    print('--- merge developers ---')
+    n = merge_developers(con)
+    print(f'  inserted {n} new developers')
+
+    con.close()
+
+
+if __name__ == '__main__':
+    main()
--- a/data/sql/45_relations_v2.sql
+++ b/data/sql/45_relations_v2.sql
@ -0,0 +1,130 @@
+-- Дополнительные связи между таблицами (27.04.2026 evening) + analytical views
+-- для сквозного анализа: квартиры ↔ ЖК ↔ застройщики ↔ регионы.
+--
+-- Pre-checked orphans:
+--   • domrf_kn_objects.region_cd → regions: 1516/1516 valid ✓
+--   • domrf_kn_objects.dev_id    → developers: 34 orphans (NULLed before FK)
+--   • sv_dev_sales.dev_id        → developers: 50/50 valid ✓
+--   • rosreestr_rent_deals.region_cd → regions: 1071/1071 valid ✓
+--   • dev_growth.dev_id          → developers: ALL NULL — no FK possible
+--   • domrf_kn_flats.obj_id      → kn_objects.obj_id: ALL 27563 orphans
+--     (different ID systems: /api/kn/flats uses hobj_id-style codes, only
+--     2 unique obj_id from flats match objects.hobj_id). NO FK; join only
+--     via obj_name text match (best-effort) or aggregate by obj_name.
+
+BEGIN;
+
+-- 1. NULL out 34 orphan dev_id values in kn_objects before adding FK
+UPDATE domrf_kn_objects SET dev_id=NULL
+WHERE dev_id IS NOT NULL
+  AND NOT EXISTS (SELECT 1 FROM domrf_developers d WHERE d.developer_id=domrf_kn_objects.dev_id);
+
+-- 2. Add 4 clean FKs (31 → 35 total)
+ALTER TABLE domrf_kn_objects
+  ADD CONSTRAINT fk_kn_obj_region
+    FOREIGN KEY (region_cd) REFERENCES domrf_regions(region_id) ON UPDATE CASCADE,
+  ADD CONSTRAINT fk_kn_obj_developer
+    FOREIGN KEY (dev_id) REFERENCES domrf_developers(developer_id) ON UPDATE CASCADE;
+
+ALTER TABLE sv_dev_sales
+  ADD CONSTRAINT fk_sv_dev_sales_developer
+    FOREIGN KEY (dev_id) REFERENCES domrf_developers(developer_id) ON UPDATE CASCADE;
+
+ALTER TABLE rosreestr_rent_deals
+  ADD CONSTRAINT fk_rent_deals_region
+    FOREIGN KEY (region_cd) REFERENCES domrf_regions(region_id) ON UPDATE CASCADE;
+
+-- 3. Analytical view: ЖК + dev + регион (use FK joins)
+CREATE OR REPLACE VIEW v_jk_with_developer AS
+SELECT o.obj_id, o.hobj_id, o.comm_name AS jk_name,
+       o.addr, o.short_addr, o.region_cd, r.region_name,
+       o.dev_id, COALESCE(d.developer_name, o.dev_name) AS dev_name,
+       o.flat_count, o.square_living,
+       o.obj_class, o.wall_type, o.energy_eff, o.escrow, o.green_house,
+       o.ready_dt, o.site_status,
+       o.is_prinzip, o.is_ekb, o.is_problem,
+       o.latitude, o.longitude
+FROM domrf_kn_objects o
+LEFT JOIN domrf_developers d ON o.dev_id = d.developer_id
+LEFT JOIN domrf_regions r    ON o.region_cd = r.region_id;
+
+COMMENT ON VIEW v_jk_with_developer IS
+  'ЖК Свердл с обогащением dev_name из канонического справочника + region_name. Использует FK fk_kn_obj_developer/region.';
+
+-- 4. ЖК + поквартирные продажи (text match obj_name → only 2 ЖК matchятся,
+--    но view удобен для standalone-анализа kn_flats по ЖК).
+CREATE OR REPLACE VIEW v_jk_sales_summary AS
+SELECT obj_name AS jk_name,
+  MIN(city) AS city,
+  COUNT(*) AS flats_total,
+  COUNT(*) FILTER (WHERE status='sold')   AS flats_sold,
+  COUNT(*) FILTER (WHERE status='booked') AS flats_booked,
+  COUNT(*) FILTER (WHERE status='free')   AS flats_free,
+  ROUND((COUNT(*) FILTER (WHERE status='sold') * 100.0 / NULLIF(COUNT(*),0))::numeric, 1) AS sold_pct,
+  ROUND(AVG(price_per_m2)::numeric, 0) AS avg_ppm,
+  ROUND(AVG(total_area)::numeric, 1) AS avg_area_sqm,
+  SUM(price_rub) AS total_value_rub
+FROM domrf_kn_flats
+WHERE obj_name IS NOT NULL
+GROUP BY obj_name;
+
+COMMENT ON VIEW v_jk_sales_summary IS
+  'Поквартирная агрегация по ЖК (имя): кол-во × free/booked/sold + avg цена/площадь. Источник: domrf_kn_flats (РФ-snapshot).';
+
+-- 5. Портфель ЖК у каждого девелопера (через FK)
+CREATE OR REPLACE VIEW v_developer_portfolio AS
+SELECT d.developer_id, d.developer_name,
+  COUNT(DISTINCT o.obj_id) AS jk_count,
+  SUM(o.flat_count) AS jk_flats_total,
+  ROUND(SUM(o.square_living)::numeric, 0) AS jk_sqm_total,
+  COUNT(DISTINCT o.obj_id) FILTER (WHERE o.is_ekb=true) AS jk_ekb,
+  COUNT(DISTINCT o.obj_id) FILTER (WHERE o.is_problem=true) AS jk_problem,
+  COUNT(DISTINCT o.obj_id) FILTER (WHERE o.escrow=true) AS jk_escrow,
+  COUNT(DISTINCT o.obj_id) FILTER (WHERE o.site_status='Сданные') AS jk_completed,
+  COUNT(DISTINCT o.obj_id) FILTER (WHERE o.site_status='Строящиеся') AS jk_in_progress,
+  MIN(o.ready_dt) AS earliest_ready, MAX(o.ready_dt) AS latest_ready
+FROM domrf_developers d
+LEFT JOIN domrf_kn_objects o ON o.dev_id = d.developer_id
+GROUP BY d.developer_id, d.developer_name;
+
+COMMENT ON VIEW v_developer_portfolio IS
+  'Портфель ЖК у каждого девелопера: total ЖК / квартир / м² + разбивка ЕКБ / problem / эскроу / Сданы/Строятся. Только Свердл (что в kn_objects).';
+
+-- 6. Полный профиль девелопера = portfolio + квартирография + realization Свердл
+CREATE OR REPLACE VIEW v_developer_full_metrics AS
+SELECT
+  p.developer_id, p.developer_name,
+  p.jk_count, p.jk_flats_total, p.jk_sqm_total, p.jk_ekb,
+  p.jk_completed, p.jk_in_progress, p.jk_escrow,
+  da_total.flat_count AS agg_flats_total,
+  da_one.flat_count   AS agg_one_room,
+  da_two.flat_count   AS agg_two_room,
+  da_three.flat_count AS agg_three_room,
+  da_four.flat_count  AS agg_four_plus,
+  ROUND((da_one.flat_count * 100.0 / NULLIF(da_total.flat_count,0))::numeric, 1) AS pct_one,
+  ROUND(((COALESCE(da_three.flat_count,0) + COALESCE(da_four.flat_count,0)) * 100.0
+         / NULLIF(da_total.flat_count,0))::numeric, 1) AS pct_three_plus,
+  ROUND((da_total.area_sqm / NULLIF(da_total.flat_count,0))::numeric, 1) AS avg_area_sqm,
+  rl.total_square AS sverdl_sqm,
+  rl.sold_perc    AS sverdl_sold_pct,
+  rl.unsold_perc  AS sverdl_unsold_pct,
+  rl.price_avg    AS sverdl_price_avg
+FROM v_developer_portfolio p
+LEFT JOIN domrf_developer_aggregates da_total
+  ON da_total.developer_id=p.developer_id AND da_total.room_count_type='TOTAL'  AND da_total.region_id=0
+LEFT JOIN domrf_developer_aggregates da_one
+  ON da_one.developer_id=p.developer_id   AND da_one.room_count_type='ONE'    AND da_one.region_id=0
+LEFT JOIN domrf_developer_aggregates da_two
+  ON da_two.developer_id=p.developer_id   AND da_two.room_count_type='TWO'    AND da_two.region_id=0
+LEFT JOIN domrf_developer_aggregates da_three
+  ON da_three.developer_id=p.developer_id AND da_three.room_count_type='THREE'AND da_three.region_id=0
+LEFT JOIN domrf_developer_aggregates da_four
+  ON da_four.developer_id=p.developer_id  AND da_four.room_count_type='FOUR' AND da_four.region_id=0
+LEFT JOIN domrf_realization rl
+  ON rl.subject=p.developer_id AND rl.endpoint_type='developer' AND rl.region_code=66
+     AND rl.type_square='total' AND rl.snapshot_date='2026-04-27';
+
+COMMENT ON VIEW v_developer_full_metrics IS
+  'Единый профиль девелопера: portfolio (ЖК) + квартирография (1-к/2-к/3-к доли) + realization Свердл. Best query для PRINZIP/Брусника/конкурентов сравнения.';
+
+COMMIT;
--- a/memory/memory-gendesign.jsonl
+++ b/memory/memory-gendesign.jsonl
--- a/sshkey.txt
+++ b/sshkey.txt
@ -80,3 +80,13 @@ pre-commit install
    $env:PATH + ";$env:LOCALAPPDATA\Python\pythoncore-3.14-64\Scripts",
    "User"
 )
+
+Host gendesign
+    HostName 46.173.16.127
+    User gendesign
+    Port 22
+    IdentityFile ~/.ssh/gendesign_deploy
+    IdentitiesOnly yes
+    ServerAliveInterval 60
+    LocalForward 15432 127.0.0.1:5432
+    LocalForward 8000 127.0.0.1:8000