feat(tradein/db): derived price-history trigger (avito/yandex) + last_seen_at deactivate index #1559

Merged

lekss361 merged 1 commit from feat/price-history-trigger-and-lastseen-index into main

2026-06-16 16:48:59 +00:00

lekss361 commented

2026-06-16 16:44:43 +00:00

Owner

Summary

Two DB migrations closing audit gaps in listings data completeness.

116 — derived price-change trigger → `offer_price_history`

The listings upsert (base.py::save_listings, ON CONFLICT … SET price_rub = EXCLUDED.price_rub) overwrites price with no history row. Only cian had offer_price_history data (7125 rows, source-provided from cian's priceChanges JSON). avito/yandex price changes were silently lost on re-scrape.

New AFTER UPDATE OF price_rub trigger listings_price_change_trg records a derived history row when price_rub actually changes.

Critical cian-exclusion: cian_price_history.py::backfill_cian_price_history selects cian listings via LEFT JOIN offer_price_history … WHERE oph.listing_id IS NULL (only listings with zero history rows). A trigger row for a cian listing would permanently exclude it from that backfill, losing the full source-provided timeline. So the trigger fires only for NEW.source <> 'cian' (verified against cian_price_history.py:77).

No-ops by design: deactivate_stale (sets is_active, not price) and geocode backfill (sets geom) never fire it — AFTER UPDATE OF price_rub isn't even evaluated for them.

117 — index for deactivate_stale queries

deactivate_stale_listings filters WHERE source=… AND is_active=true AND last_seen_at < …; there was no supporting index (seq scan ~39k rows). Adds partial listings_active_source_lastseen_idx (source, last_seen_at) WHERE is_active.

Verification (prod, rolled-back transactions)

116: created trigger in txn, then UPDATE avito listing price → 1 new offer_price_history row; UPDATE cian listing price → 0 rows. Proven, then ROLLBACK.
117: EXPLAIN of a deactivate-shaped query → Bitmap Index Scan on listings_active_source_lastseen_idx.

Test plan

Full gate: 1834 passed, 2 deselected
ruff clean (no Python changed)
Post-deploy: confirm trigger exists + first avito/yandex price change writes a row

Refs #753

## Summary Two DB migrations closing audit gaps in listings data completeness. ### 116 — derived price-change trigger → `offer_price_history` The `listings` upsert (`base.py::save_listings`, `ON CONFLICT … SET price_rub = EXCLUDED.price_rub`) overwrites price with **no history row**. Only **cian** had `offer_price_history` data (7125 rows, source-provided from cian's `priceChanges` JSON). avito/yandex price changes were silently lost on re-scrape. New `AFTER UPDATE OF price_rub` trigger `listings_price_change_trg` records a derived history row when `price_rub` actually changes. **Critical cian-exclusion:** `cian_price_history.py::backfill_cian_price_history` selects cian listings via `LEFT JOIN offer_price_history … WHERE oph.listing_id IS NULL` (only listings with **zero** history rows). A trigger row for a cian listing would permanently exclude it from that backfill, losing the full source-provided timeline. So the trigger fires **only for `NEW.source <> 'cian'`** (verified against cian_price_history.py:77). No-ops by design: `deactivate_stale` (sets `is_active`, not price) and geocode backfill (sets `geom`) never fire it — `AFTER UPDATE OF price_rub` isn't even evaluated for them. ### 117 — index for deactivate_stale queries `deactivate_stale_listings` filters `WHERE source=… AND is_active=true AND last_seen_at < …`; there was no supporting index (seq scan ~39k rows). Adds partial `listings_active_source_lastseen_idx (source, last_seen_at) WHERE is_active`. ## Verification (prod, rolled-back transactions) - **116**: created trigger in txn, then UPDATE avito listing price → **1** new `offer_price_history` row; UPDATE cian listing price → **0** rows. Proven, then ROLLBACK. - **117**: `EXPLAIN` of a deactivate-shaped query → **Bitmap Index Scan on listings_active_source_lastseen_idx**. ## Test plan - [x] Full gate: **1834 passed, 2 deselected** - [x] ruff clean (no Python changed) - [ ] Post-deploy: confirm trigger exists + first avito/yandex price change writes a row Refs #753

lekss361 added 1 commit 2026-06-16 16:44:43 +00:00

feat(tradein/db): price-history trigger (116) + last_seen_at index (117)

CI / openapi-codegen-check (push) Has been skipped

Details

CI / changes (push) Successful in 7s

Details

CI / backend-tests (push) Has been skipped

Details

CI / frontend-tests (push) Has been skipped

Details

CI / changes (pull_request) Successful in 7s

Details

CI / backend-tests (pull_request) Has been skipped

Details

CI / frontend-tests (pull_request) Has been skipped

Details

CI / openapi-codegen-check (pull_request) Has been skipped

Details

59d58cf931

116: AFTER UPDATE OF price_rub trigger on listings writes derived
offer_price_history rows for avito/yandex; cian excluded to preserve
source-provided backfill (cian_price_history.py LEFT JOIN oph IS NULL guard).

117: partial composite index (source, last_seen_at) WHERE is_active to
accelerate deactivate_stale_listings seq-scan on ~39k rows.