feat(analytics): mv_layout_velocity MV + refresh helper (#113 PR B)

Add materialized view aggregating objective_corpus_room_month JOIN
objective_complex_mapping (matched via objective_complex_name) за last
24 months, grouped by (domrf_obj_id, room_bucket). Cyrillic 'студия'
normalised к ASCII 'studio' для consistency с layout_signature.py (PR A).

Tables / cardinality (verified live на проде):
- objective_corpus_room_month: 15 738 EKB rows, monthly 2025-05..2026-05
- objective_complex_mapping: 114 mapped EKB objects (7.5% coverage)
- Expected MV size: ~459 rows (114 objs × ~4 room_buckets)
- EXPLAIN: Hash Join 19k×114 → 8k pre-agg → 459 groups, no extra indexes needed

UNIQUE INDEX (obj_id, room_bucket) enables REFRESH CONCURRENTLY (used
by Python helper layout_velocity_refresh.py — НЕ scheduled в этом PR,
будет triggered manually или через celery beat в follow-up).

Migration applied auto-через deploy.yml (see .claude/rules/sql.md).

Smoke verification — qa-tester after deploy (test_layout_velocity_mv.py
removed because conftest.py / db fixture infra не существует в проекте;
проще smoke через прод SELECT COUNT после migration apply).
This commit is contained in:
lekss361 2026-05-16 11:00:02 +03:00
parent e673811a20
commit 144970aadd
2 changed files with 114 additions and 0 deletions

View file

@ -0,0 +1,42 @@
"""Refresh helper for mv_layout_velocity (Issue #113 PR B).
Not scheduled automatically in this PR intended for manual invocation or
a Celery beat task in a follow-up issue.
Usage example (manual, via psql-connected shell or admin endpoint):
from sqlalchemy.orm import Session
from app.services.site_finder.layout_velocity_refresh import refresh_layout_velocity
count = refresh_layout_velocity(db)
# logs: "mv_layout_velocity refreshed: 459 rows"
"""
from __future__ import annotations
import logging
from sqlalchemy import text
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
def refresh_layout_velocity(db: Session, *, concurrently: bool = True) -> int:
"""REFRESH MATERIALIZED VIEW mv_layout_velocity.
Args:
db: SQLAlchemy Session (sync).
concurrently: When True, uses REFRESH CONCURRENTLY non-blocking but
requires the unique index mv_layout_velocity_pk to exist (created
by 94_mv_layout_velocity.sql). Pass False only for the very first
populate or when the MV was just recreated.
Returns:
Row count in the MV after refresh (for observability / alerting).
"""
mode = "CONCURRENTLY" if concurrently else ""
db.execute(text(f"REFRESH MATERIALIZED VIEW {mode} mv_layout_velocity"))
row = db.execute(text("SELECT COUNT(*) FROM mv_layout_velocity")).first()
count = int(row[0]) if row else 0
logger.info("mv_layout_velocity refreshed: %d rows", count)
return count

View file

@ -0,0 +1,72 @@
-- 94_mv_layout_velocity.sql
-- Issue #113 PR B — Materialized view per (obj_id, room_bucket) for top-layouts ranking.
--
-- Sources:
-- objective_corpus_room_month — monthly per-corpus-per-room deals (19 738 rows, 2025-05..2026-05)
-- objective_complex_mapping — project_name → domrf_obj_id mapping (114 mapped objects)
--
-- Join key:
-- objective_corpus_room_month.project_name = objective_complex_mapping.objective_complex_name
-- (verified: one project_name → at most one domrf_obj_id)
--
-- room_bucket normalisation:
-- cyrillic 'студия' → ASCII 'studio' for consistency with layout_signature.py (PR A, Issue #113)
-- values '1', '2', '3', '4+' are kept as-is
--
-- Window: last 24 months (rolling, anchored to NOW() at refresh time)
-- Expected rows after aggregation: ~459 (obj_id × room_bucket combinations)
--
-- REFRESH CONCURRENTLY is safe after this migration because a UNIQUE index
-- on (obj_id, room_bucket) is created immediately.
-- Subsequent refreshes: layout_velocity_refresh.py helper (PR B) — not scheduled yet.
--
-- Deploy: auto-applied by deploy.yml via _schema_migrations tracking.
-- Dependencies: none (no views depend on this MV at creation time).
BEGIN;
DROP MATERIALIZED VIEW IF EXISTS mv_layout_velocity CASCADE;
CREATE MATERIALIZED VIEW mv_layout_velocity AS
WITH last24mo AS (
SELECT
ocm.project_name,
CASE
WHEN ocm.room_bucket = 'студия' THEN 'studio'
ELSE ocm.room_bucket
END AS room_bucket,
ocm.deals_total_count,
ocm.deals_total_avg_area_m2,
ocm.deals_total_avg_price_thousand_rub_per_m2,
ocm.deals_total_vol_m2,
ocm.report_month
FROM objective_corpus_room_month ocm
WHERE ocm.report_month >= (NOW() - INTERVAL '24 months')::date
)
SELECT
cm.domrf_obj_id AS obj_id,
l.room_bucket,
SUM(l.deals_total_count)::int AS total_deals_24mo,
AVG(l.deals_total_avg_area_m2)::numeric(10, 2) AS avg_area_m2,
AVG(l.deals_total_avg_price_thousand_rub_per_m2)::numeric(12, 2)
AS avg_price_thousand_rub_per_m2,
SUM(l.deals_total_vol_m2)::numeric(12, 2) AS total_vol_m2,
MIN(l.report_month) AS window_start,
MAX(l.report_month) AS window_end,
COUNT(DISTINCT l.report_month)::int AS months_with_data
FROM last24mo l
JOIN objective_complex_mapping cm
ON cm.objective_complex_name = l.project_name
WHERE l.room_bucket IS NOT NULL
AND cm.domrf_obj_id IS NOT NULL
GROUP BY cm.domrf_obj_id, l.room_bucket;
-- UNIQUE index required for REFRESH CONCURRENTLY (future periodic refreshes)
CREATE UNIQUE INDEX mv_layout_velocity_pk
ON mv_layout_velocity (obj_id, room_bucket);
-- Lookup index used by /best-layouts endpoint (PR C)
CREATE INDEX mv_layout_velocity_obj_idx
ON mv_layout_velocity (obj_id);
COMMIT;