fix(avito): classify novostroyki by item-development-name marker; map переуступка sale_type #1716

Merged

lekss361 merged 1 commit from fix/avito-novostroyki-segment into main

2026-06-17 19:34:28 +00:00

lekss361 commented

2026-06-17 19:26:07 +00:00

Owner

Summary

Fixes avito novostroyki being 100% mislabeled vtorichka. The producer (_dom_card_to_lot) classified segment by URL pattern: if "/kvartiry/" in href → vtorichka — but EVERY avito apartment URL (incl. novostroyki) contains /kvartiry/, so the /novostroyki/ branch was dead code. Prod: avito novostroyki=0 despite the geo-anchored sweep ingesting them (it hits the generic /kvartiry/prodam-... SERP = both segments mixed).

Verified (home IP DOM, this session)

SERP card discriminator: novostroyki cards carry data-marker="item-development-name" (ЖК name), secondary do not — measured novostroyka SERP 4/4 present, vtorichka 0/12. Example mislabeled row: source_id 8043003287 (ЖК «Федерация», ДДУ переуступка) stored as vtorichka.

Changes (2 edits)

avito.py _dom_card_to_lot: segment now "novostroyki" if card.css_first('[data-marker="item-development-name"]') else "vtorichka"; removed dead href branches.
avito_detail.py _SALE_TYPE_MAP: +"переуступка":"assignment" (was → None; переуступка = DDU assignment, novostroyka signal).

No DB backfill (existing rows self-heal on re-scrape); newbuilding_id/url capture deferred to a follow-up.

Test plan

pytest tests/ -k avito → 212 passed; ruff clean
SERP card with/without item-development-name → novostroyki/vtorichka; detail "Способ продажи: переуступка" → sale_type=assignment
Post-deploy smoke: SELECT listing_segment,count(*) FROM listings WHERE source='avito' AND last_seen_at::date=CURRENT_DATE GROUP BY 1 shows novostroyki>0

Refs #1188

## Summary Fixes avito novostroyki being 100% mislabeled `vtorichka`. The producer (`_dom_card_to_lot`) classified segment by URL pattern: `if "/kvartiry/" in href → vtorichka` — but EVERY avito apartment URL (incl. novostroyki) contains `/kvartiry/`, so the `/novostroyki/` branch was dead code. Prod: avito novostroyki=0 despite the geo-anchored sweep ingesting them (it hits the generic `/kvartiry/prodam-...` SERP = both segments mixed). ## Verified (home IP DOM, this session) SERP card discriminator: novostroyki cards carry `data-marker="item-development-name"` (ЖК name), secondary do not — measured novostroyka SERP 4/4 present, vtorichka 0/12. Example mislabeled row: source_id 8043003287 (ЖК «Федерация», ДДУ переуступка) stored as vtorichka. ## Changes (2 edits) - `avito.py _dom_card_to_lot`: segment now `"novostroyki" if card.css_first('[data-marker="item-development-name"]') else "vtorichka"`; removed dead href branches. - `avito_detail.py _SALE_TYPE_MAP`: `+"переуступка":"assignment"` (was → None; переуступка = DDU assignment, novostroyka signal). No DB backfill (existing rows self-heal on re-scrape); newbuilding_id/url capture deferred to a follow-up. ## Test plan - [x] `pytest tests/ -k avito` → 212 passed; ruff clean - [x] SERP card with/without item-development-name → novostroyki/vtorichka; detail "Способ продажи: переуступка" → sale_type=assignment - [ ] Post-deploy smoke: `SELECT listing_segment,count(*) FROM listings WHERE source='avito' AND last_seen_at::date=CURRENT_DATE GROUP BY 1` shows novostroyki>0 Refs #1188

lekss361 added 1 commit 2026-06-17 19:26:08 +00:00