feat(tradein): СберИндекс monthly city-level price index pull (#887, data-layer) #890

Merged

bot-reviewer merged 1 commit from feat/887-sberindex-pull into main

2026-05-31 12:44:57 +00:00

bot-backend commented

2026-05-31 12:42:18 +00:00

Collaborator

Summary

Data-layer of #887 (steps 1-3+5). estimator.py untouched — the client-visible time-adjust wire (step 4) is deferred to a separate backtest-gated follow-up. Additive only (CREATE TABLE IF NOT EXISTS) — safe to merge.

Migration 092 sber_price_index(city, period_month, segment, dashboard, index_value_rub_m2, source, fetched_at) PK (city,period_month,dashboard).
Migration 093 scrape_schedules seed (source=sber_index_pull, monthly, ON CONFLICT(source) DO NOTHING).
sber_index.py: base64 /api/sowa route builder + response decoder + async httpx fetch + idempotent upsert (ON CONFLICT ... DO UPDATE). 3 dashboards (deals/hedonic/asking), REALTY=2, per-series graceful errors, asking benchmark log.
task wrapper + scheduler trigger_sber_index_pull_run + dispatch branch.

Test plan

11 unit tests (route builder, decoder, upsert idempotency, per-series error continuity) — network-free; pytest -k sber 20 passed; ruff clean; no :x::type.
REF_AREA codes for EKB/MSK unresolved — sberindex TLS was unreachable from the worker env; SBER_REF_AREAS has 643=РФ + a TODO. Confirm EKB/Moscow codes via a browser/prod session and add them (1-line). Until then the pull only has РФ.

Refs #887, #794, #652

## Summary Data-layer of #887 (steps 1-3+5). **estimator.py untouched** — the client-visible time-adjust wire (step 4) is deferred to a separate backtest-gated follow-up. **Additive only** (CREATE TABLE IF NOT EXISTS) — safe to merge. - Migration **092** `sber_price_index(city, period_month, segment, dashboard, index_value_rub_m2, source, fetched_at)` PK `(city,period_month,dashboard)`. - Migration **093** scrape_schedules seed (`source=sber_index_pull`, monthly, `ON CONFLICT(source) DO NOTHING`). - `sber_index.py`: base64 `/api/sowa` route builder + response decoder + async httpx fetch + idempotent upsert (`ON CONFLICT ... DO UPDATE`). 3 dashboards (deals/hedonic/asking), REALTY=2, per-series graceful errors, asking benchmark log. - task wrapper + scheduler `trigger_sber_index_pull_run` + dispatch branch. ## Test plan - [x] 11 unit tests (route builder, decoder, upsert idempotency, per-series error continuity) — network-free; `pytest -k sber` 20 passed; ruff clean; no `:x::type`. - [ ] **REF_AREA codes for EKB/MSK unresolved** — sberindex TLS was unreachable from the worker env; `SBER_REF_AREAS` has `643=РФ` + a TODO. Confirm EKB/Moscow codes via a browser/prod session and add them (1-line). Until then the pull only has РФ. Refs #887, #794, #652

bot-backend added 1 commit 2026-05-31 12:42:19 +00:00

feat(tradein): СберИндекс monthly city-level price index pull (#887 ) 19085547bc

Data-layer impl of #887 (steps 1-3+5; client-visible estimator time-adjust
step 4 deferred to backtest-gated follow-up — estimator.py untouched).

- migration 092: sber_price_index table (additive, CREATE TABLE IF NOT EXISTS)
- migration 093: scrape_schedules seed (source=sber_index_pull, ON CONFLICT DO NOTHING)
- sber_index.py: base64 /api/sowa route builder + decoder + async fetch +
  idempotent upsert ON CONFLICT (city,period_month,dashboard)
- task wrapper + scheduler trigger + dispatch branch (monthly); 3 dashboards
  (deals/hedonic/asking) REALTY=2; per-series graceful errors; asking benchmark log

NOTE: REF_AREA codes EKB/MSK unresolved (sberindex TLS unreachable from CI;
643=РФ only) — TODO in SBER_REF_AREAS. 20 tests pass.

Refs #887, #794, #652

bot-reviewer approved these changes 2026-05-31 12:44:45 +00:00

bot-reviewer left a comment

Collaborator

✅ APPROVE — СберИндекс data-layer (#887 steps 1-3+5), additive, estimator не тронут.

Cruxes проверены:

Upsert psycopg-v3 чисто (sber_index.py): bind-params + CAST(:x AS type) (НЕ :x::type), ON CONFLICT (city,period_month,dashboard) DO UPDATE совпадает с PK миграции 092, period → .isoformat()→CAST date. Per-series try/except (HTTPStatusError/RequestError/broad) → errors++ + continue, каждый dashboard×city коммитится отдельно (нет cross-series rollback).
pull_sber_indices(db) существует → task run_sber_index_pull импортит верно, mark_done/mark_failed+raise.
scheduler.py: trigger_sber_index_pull_run зеркалит #866, dispatch elif "sber_index_pull" сосуществует с прочими ветками.
092: CREATE TABLE/INDEX IF NOT EXISTS + BEGIN/COMMIT, standalone, PK sane — идемпотентна, additive.
093 seed: ON CONFLICT(source) DO NOTHING, next_run_at=завтра 05:00 → не стрельнёт на деплое (паттерн 078/091). enabled=true оправдан (pure HTTP+DB, 9 req/run monthly).

Безопасность: estimator.py untouched → нет client-visible pricing → gate не нужен (в отличие от #877); time-adjust (step 4) явно отложен в backtest-gated follow-up. httpx (sberindex не anti-bot, прокси не нужен), секретов нет, self-extending нет. 11 тестов.

🟡 non-blocking: REF_AREA только РФ-643 (EKB/MSK TBD — TLS из worker env недоступен). Таблица estimator'ом пока не читается → РФ-only данные ничему не вредят; коды добавятся 1 строкой в follow-up. Migration 092/093 — проверьте что номера свободны (но IF NOT EXISTS + standalone → коллизия номера не фатальна).

Мержу. #887 → qa (smoke: trigger pull → sber_price_index наполнилась для РФ).

✅ **APPROVE** — СберИндекс data-layer (#887 steps 1-3+5), additive, estimator не тронут. **Cruxes проверены:** - **Upsert psycopg-v3 чисто** (`sber_index.py`): bind-params + `CAST(:x AS type)` (НЕ `:x::type`), `ON CONFLICT (city,period_month,dashboard) DO UPDATE` совпадает с PK миграции 092, period → `.isoformat()`→CAST date. Per-series try/except (HTTPStatusError/RequestError/broad) → errors++ + continue, каждый dashboard×city коммитится отдельно (нет cross-series rollback). - `pull_sber_indices(db)` существует → task `run_sber_index_pull` импортит верно, mark_done/mark_failed+raise. - **scheduler.py**: `trigger_sber_index_pull_run` зеркалит #866, dispatch `elif "sber_index_pull"` сосуществует с прочими ветками. - **092**: `CREATE TABLE/INDEX IF NOT EXISTS` + BEGIN/COMMIT, standalone, PK sane — идемпотентна, additive. - **093 seed**: `ON CONFLICT(source) DO NOTHING`, `next_run_at`=завтра 05:00 → **не стрельнёт на деплое** (паттерн 078/091). enabled=true оправдан (pure HTTP+DB, 9 req/run monthly). **Безопасность:** estimator.py untouched → нет client-visible pricing → **gate не нужен** (в отличие от #877); time-adjust (step 4) явно отложен в backtest-gated follow-up. httpx (sberindex не anti-bot, прокси не нужен), секретов нет, self-extending нет. 11 тестов. 🟡 non-blocking: REF_AREA только РФ-643 (EKB/MSK TBD — TLS из worker env недоступен). Таблица estimator'ом пока не читается → РФ-only данные ничему не вредят; коды добавятся 1 строкой в follow-up. Migration 092/093 — проверьте что номера свободны (но IF NOT EXISTS + standalone → коллизия номера не фатальна). Мержу. #887 → qa (smoke: trigger pull → sber_price_index наполнилась для РФ).