lekss361/gendesign

Fork 0

feat(tradein): conflict_resolution — register Yandex sources in priority dicts #471

Merged

lekss361 merged 1 commit from feat/tradein-yandex-matching into main

2026-05-23 14:03:36 +00:00

lekss361 commented

2026-05-23 13:57:06 +00:00

Owner

Motivation

Stage 7 of 9 for YandexRealtyScraper v1 (Wave 5). Integrates the 4 Yandex scrapers (yandex / yandex_detail / yandex_realty_nb / yandex_valuation) into cross-source field-priority arbitration. Resolver engine (_resolve()) unchanged — only dict entries + new fields.

Source names recap

Scraper class	`name`
YandexRealtyScraper (SERP)	`yandex`
YandexDetailScraper	`yandex_detail`
YandexNewbuildingScraper (ЖК)	`yandex_realty_nb`
YandexValuationScraper (anon history)	`yandex_valuation`

HOUSE_FIELD_PRIORITY changes

Updated entries:

lat / lon: legacy "yandex" → "yandex_realty_nb" (house geo comes from ЖК landing, not SERP)
year_built: added yandex_valuation, yandex_realty_nb
house_type: added yandex_valuation, yandex_realty_nb
house_class, rating_score, reviews_count: added yandex_realty_nb

New fields:

Field	Sources	Note
`text_reviews_count`	`[yandex_realty_nb]`	The 353 text reviews — Yandex unique
`corpus_count`	`[yandex_realty_nb]`	"три башни" → 3
`total_area_ha`	`[yandex_realty_nb]`	ЖК footprint
`commission_year`	`[cian_serp, yandex_realty_nb]`
`commission_month`	`[yandex_realty_nb]`	RU month name
`developer_name`	`[cian, yandex_realty_nb]`
`has_panorama`	`[yandex_valuation]`	Yandex 3D panorama flag
`yandex_total_listings`	`[yandex_valuation]`	"N объектов" in history
`has_lift`	`[cian_bti, cian_detail, yandex_valuation]`
`ceiling_height`	`[cian_detail, yandex_valuation]`

LISTING_FIELD_PRIORITY changes

Updated: description, house_type — added yandex_detail (NLP fallback for house_type)

New fields (all yandex_detail only — OfferCardAuthorInfo + summary unique):

agency_name, agency_founded_year, agency_objects_count
views_total_yandex (parallel to Cian's views_total, separate column per PR #453 schema)
publish_date_relative (raw "6 часов назад" string)
sale_type_text (raw RU phrase, vs Cian's enum)

Tests

tests/matching/test_conflict_resolution.py: 37 passed, 0 failed (7 pre-existing + 30 new)
tests/ -k matching: 42 passed, 3 skipped (minhash + valuations)
ruff check: All checks passed!

Pre-existing 3 collection errors for test_houses/test_listings/test_normalize are in modules not on forgejo/main (separate stash) — unrelated to this PR.

Files

tradein-mvp/backend/app/services/matching/conflict_resolution.py
tradein-mvp/backend/tests/matching/test_conflict_resolution.py

Reviewer note

Reviewer: deep-code-reviewer — only merge after APPROVE. After merge → Wave 6 (Stage 8 estimator integration — Yandex Valuation as 7th source).

## Motivation Stage 7 of 9 for YandexRealtyScraper v1 (Wave 5). Integrates the 4 Yandex scrapers (`yandex` / `yandex_detail` / `yandex_realty_nb` / `yandex_valuation`) into cross-source field-priority arbitration. Resolver engine (`_resolve()`) unchanged — only dict entries + new fields. ## Source names recap | Scraper class | `name` | |---|---| | YandexRealtyScraper (SERP) | `yandex` | | YandexDetailScraper | `yandex_detail` | | YandexNewbuildingScraper (ЖК) | `yandex_realty_nb` | | YandexValuationScraper (anon history) | `yandex_valuation` | ## HOUSE_FIELD_PRIORITY changes **Updated entries:** - `lat` / `lon`: legacy `"yandex"` → `"yandex_realty_nb"` (house geo comes from ЖК landing, not SERP) - `year_built`: added `yandex_valuation`, `yandex_realty_nb` - `house_type`: added `yandex_valuation`, `yandex_realty_nb` - `house_class`, `rating_score`, `reviews_count`: added `yandex_realty_nb` **New fields:** | Field | Sources | Note | |---|---|---| | `text_reviews_count` | `[yandex_realty_nb]` | The 353 text reviews — Yandex unique | | `corpus_count` | `[yandex_realty_nb]` | "три башни" → 3 | | `total_area_ha` | `[yandex_realty_nb]` | ЖК footprint | | `commission_year` | `[cian_serp, yandex_realty_nb]` | | | `commission_month` | `[yandex_realty_nb]` | RU month name | | `developer_name` | `[cian, yandex_realty_nb]` | | | `has_panorama` | `[yandex_valuation]` | Yandex 3D panorama flag | | `yandex_total_listings` | `[yandex_valuation]` | "N объектов" in history | | `has_lift` | `[cian_bti, cian_detail, yandex_valuation]` | | | `ceiling_height` | `[cian_detail, yandex_valuation]` | | ## LISTING_FIELD_PRIORITY changes **Updated:** `description`, `house_type` — added `yandex_detail` (NLP fallback for house_type) **New fields (all `yandex_detail` only — OfferCardAuthorInfo + summary unique):** - `agency_name`, `agency_founded_year`, `agency_objects_count` - `views_total_yandex` (parallel to Cian's `views_total`, separate column per PR #453 schema) - `publish_date_relative` (raw "6 часов назад" string) - `sale_type_text` (raw RU phrase, vs Cian's enum) ## Tests ``` tests/matching/test_conflict_resolution.py: 37 passed, 0 failed (7 pre-existing + 30 new) tests/ -k matching: 42 passed, 3 skipped (minhash + valuations) ruff check: All checks passed! ``` Pre-existing 3 collection errors for `test_houses/test_listings/test_normalize` are in modules not on `forgejo/main` (separate stash) — unrelated to this PR. ## Files - `tradein-mvp/backend/app/services/matching/conflict_resolution.py` - `tradein-mvp/backend/tests/matching/test_conflict_resolution.py` ## Reviewer note Reviewer: **deep-code-reviewer** — only merge after APPROVE. After merge → Wave 6 (Stage 8 estimator integration — Yandex Valuation as 7th source).

lekss361 added 1 commit 2026-05-23 13:57:07 +00:00

feat(tradein): conflict_resolution — register Yandex sources in priority dicts 64a7eb6824

Stage 7 of YandexRealtyScraper v1 (Wave 5). Integrates the 4 Yandex scrapers
(yandex / yandex_detail / yandex_realty_nb / yandex_valuation) into the
cross-source field-priority arbitration.

HOUSE_FIELD_PRIORITY:
- Renamed legacy 'yandex' → 'yandex_realty_nb' for lat/lon (house geo comes
  from ЖК landing, not SERP)
- Added yandex_valuation + yandex_realty_nb to year_built, house_type
- Added yandex_realty_nb to house_class, rating_score, reviews_count
- NEW fields:
  * text_reviews_count: ['yandex_realty_nb']           (the 353 — Yandex unique)
  * corpus_count:       ['yandex_realty_nb']           ('три башни' → 3)
  * total_area_ha:      ['yandex_realty_nb']
  * commission_year:    ['cian_serp', 'yandex_realty_nb']
  * commission_month:   ['yandex_realty_nb']
  * developer_name:     ['cian', 'yandex_realty_nb']
  * has_panorama:       ['yandex_valuation']
  * yandex_total_listings: ['yandex_valuation']
  * has_lift:           ['cian_bti', 'cian_detail', 'yandex_valuation']
  * ceiling_height:     ['cian_detail', 'yandex_valuation']

LISTING_FIELD_PRIORITY:
- Added yandex_detail to description, house_type (NLP fallback)
- NEW fields:
  * agency_name / agency_founded_year / agency_objects_count: ['yandex_detail']
    (OfferCardAuthorInfo — Yandex unique structured agency block)
  * views_total_yandex: ['yandex_detail']   (parallel to Cian's views_total)
  * publish_date_relative: ['yandex_detail']
  * sale_type_text: ['yandex_detail']        (raw RU phrase, vs Cian enum)

Tests: 30 new cases (37 total), all pre-existing tests preserved. Ruff clean.
Resolver engine unchanged (existing list / first_non_null / union / cross_validate
semantics still apply).

lekss361 merged commit 7d649f9239 into main

2026-05-23 14:03:36 +00:00

lekss361 referenced this pull request from a commit

2026-05-23 14:03:38 +00:00

feat(tradein): conflict_resolution — register Yandex sources in priority dicts (#471)

lekss361 commented

2026-05-23 14:03:58 +00:00

Author

Owner

Merged via deep-code-reviewer — verdict APPROVE (squash, commit 7d649f9).

Highlights:

Source-name verification passed (yandex/yandex_detail/yandex_realty_nb/yandex_valuation все matches scraper.name).
DB columns verified в migrations 030/031 для всех new fields, кроме text_reviews_count — harmless at dict level, нужна column в houses до Stage 8.x writes (suggest add в 031 или новый migration).
Subsource granularity (cian_serp/cian_bti/cian_detail/cian_valuation, avito_houses_catalog/avito_detail/avito_domoteka) — логические, не scraper.name; vault-aligned (Cross_Source_Matching_Strategy.md sec 3.5/4.5). Caller-side mapping (scraper, page_type) -> logical_subsource потребуется в Stage 8.x integration.
37/37 tests pass, ruff clean.

Coordination warning: PR #470 also adds conflict_resolution.py (incompatible 64-line version). #470 needs rebase + drop its own conflict_resolution.py (use this version as superset) перед merge.

Merged via deep-code-reviewer — verdict APPROVE (squash, commit `7d649f9`). Highlights: - Source-name verification passed (`yandex`/`yandex_detail`/`yandex_realty_nb`/`yandex_valuation` все matches scraper.name). - DB columns verified в migrations 030/031 для всех new fields, кроме `text_reviews_count` — harmless at dict level, нужна column в houses до Stage 8.x writes (suggest add в 031 или новый migration). - Subsource granularity (`cian_serp`/`cian_bti`/`cian_detail`/`cian_valuation`, `avito_houses_catalog`/`avito_detail`/`avito_domoteka`) — логические, не scraper.name; vault-aligned (Cross_Source_Matching_Strategy.md sec 3.5/4.5). Caller-side mapping `(scraper, page_type) -> logical_subsource` потребуется в Stage 8.x integration. - 37/37 tests pass, ruff clean. Coordination warning: PR #470 also adds `conflict_resolution.py` (incompatible 64-line version). #470 needs rebase + drop its own conflict_resolution.py (use this version as superset) перед merge.

lekss361 referenced this pull request

2026-05-23 14:04:11 +00:00

feat(tradein): cross-source matching service (3-tier: cadastr / fingerprint / geo / composite) #470

lekss361 referenced this pull request

2026-05-23 14:09:34 +00:00

feat(tradein): cross-source matching service (3-tier: cadastr / fingerprint / geo / composite) #470

lekss361 referenced this pull request

2026-05-23 14:10:03 +00:00

feat(tradein): cross-source matching service (3-tier: cadastr / fingerprint / geo / composite) #470

lekss361 referenced this pull request

2026-05-23 14:12:38 +00:00

feat(tradein): cross-source matching service (3-tier: cadastr / fingerprint / geo / composite) #470