lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/tradein-mvp/backend/tests
History
Light1YT 20b4a195b3 feat(tradein): Phase 2-3 of #582 — backfill houses + canonical reverse audit 
Builds on Phase 1 audit infra (PR #583). Forward-geocode 4141 houses
без coords + reverse audit 4452 already-geocoded houses через Yandex
Geocoder API. Verified mismatch 870м для «Малышева 125» — наш point
в другом квартале от реального здания.

Scripts (new):
- `backfill_house_coords.py` (619 LOC) — driver:
  * Forward mode: UPDATE houses SET lat/lon когда precision ∈ {exact, number}
  * `--audit-only`: reverse геокод + distance check, flag >50м как mismatch
  * Per-row SAVEPOINT, idempotent через UNIQUE (house_id, audit_batch)
  * EKB locality bias (ll=60.6122,56.8389, spn=0.6,0.4)
  * Rate-limit 50ms между запросами, 25k/день квота Yandex покрывает 8.5k houses

- `_yandex_reverse.py` (+123 LOC) — added `forward_via_api()`,
  precision/kind в YandexReverseResult dataclass

- `audit_address_sample.sql` (-7 +47) — FIX: убрана зависимость от FDW
  `gendesign_ekb_districts_geom` (нет на проде). Простой
  `ORDER BY random()` без district stratification. Bind-param контракт
  сохранён (`:limit_per_district` * 8 = 200 default).

Tests:
- `test_backfill_house_coords.py` (510 LOC) — 20 tests: request shape,
  precision filter, UPDATE shape, backfill happy/imprecise/no_match/error,
  audit ok/mismatch/no_match, resumability, missing-key exit, mode routing
- Existing `test_audit_address_mismatch.py` still green
- 39/39 pass, ruff clean

Decisions (см. PR body):
- Precision filter = {exact, number} only (street/range/near → audit-imprecise)
- Mismatch threshold 50м (consistent с Phase 1 report)
- API-only (no Playwright fallback) — exit если key missing

Closes #582 (Phase 2-3 portion; Phase 1 was #583)
2026-05-27 11:30:25 +05:00
..
fixtures feat(tradein): Phase 1 of #582 — address mismatch audit infrastructure 2026-05-25 12:50:30 +05:00
matching feat(tradein): conflict_resolution — register Yandex sources in priority dicts (#471) 2026-05-23 14:03:35 +00:00
scrapers fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487) 2026-05-23 20:08:40 +00:00
services feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) 2026-05-24 08:57:30 +00:00
tasks fix(tradein): geocode backfill + remove Avito exclusion from estimator (#490) 2026-05-23 20:03:48 +00:00
__init__.py feat(tradein): avito_imv.py — Avito IMV evaluation API client (2 requests) (#444) 2026-05-23 12:16:47 +00:00
test_api_avito_stage4a.py feat(tradein): Stage 4a — API endpoints for Avito enrichment + IMV benchmark UI (#460) 2026-05-23 13:34:38 +00:00
test_audit_address_mismatch.py feat(tradein): Phase 1 of #582 — address mismatch audit infrastructure 2026-05-25 12:50:30 +05:00
test_avito_address_clean.py fix(tradein-avito): strip Emotion CSS from listings.address (#502) 2026-05-24 10:52:50 +00:00
test_avito_detail_parse.py feat(tradein): avito_detail.py — detail page parser (30+ fields incl. domoteka) (#443) 2026-05-23 12:17:15 +00:00
test_avito_houses_parse.py feat(tradein): avito_houses.py — Houses Catalog parser (state + reviews + history + recs) (#449) 2026-05-23 12:37:24 +00:00
test_avito_imv_parse.py feat(tradein): avito_imv.py — Avito IMV evaluation API client (2 requests) (#444) 2026-05-23 12:16:47 +00:00
test_avito_relative_date.py feat(tradein-scrapers): parse listing_date from Avito + Yandex (#508) 2026-05-24 11:17:13 +00:00
test_backfill_house_coords.py feat(tradein): Phase 2-3 of #582 — backfill houses + canonical reverse audit 2026-05-27 11:30:25 +05:00
test_base_save_listings.py feat(tradein): extend base.py save_listings to persist Cian-specific columns (PR #450 follow-up) (#455) 2026-05-23 13:05:20 +00:00
test_cian_detail.py feat(tradein): cian_session cookies management + admin endpoints (Wave 4 Worker A) (#457) 2026-05-23 13:22:00 +00:00
test_cian_newbuilding.py feat(tradein): cian_newbuilding.py — ЖК catalog scraper (chart + reliability + offers, Wave 4 Worker C) (#459) 2026-05-23 13:23:11 +00:00
test_cian_serp_scraper.py feat(tradein): cian.py SERP refactor — state-based extraction (137 fields, no jitter) (#450) 2026-05-23 12:52:04 +00:00
test_cian_session.py fix(tradein): CIAN_REQUIRED_COOKIES — add real auth cookies (DMIR_AUTH + Cian session) (#480) 2026-05-23 14:52:24 +00:00
test_cian_state_parser.py feat(tradein): cian_state_parser shared utility + ScrapedLot Cian fields (#447) 2026-05-23 12:35:01 +00:00
test_cian_valuation.py feat(tradein): cian_valuation.py — auth-required Calculator scraper + 24h cache (#465) 2026-05-23 13:43:34 +00:00
test_city_sweep.py feat(tradein): city sweep — auto ЕКБ pipeline (#477) 2026-05-23 14:38:07 +00:00
test_estimator_cian_integration.py fix(tradein): filter ДКП-only in rosreestr importer + re-enable deals (#549) 2026-05-24 19:42:28 +00:00
test_estimator_cohort.py fix(tradein-estimator): extend cohort filter to Tier S + Tier H, fix late_soviet overlap (#522) 2026-05-24 13:14:59 +00:00
test_estimator_floor_optional.py feat(tradein): этаж/этажность optional + best test presets по deal count (#558) 2026-05-24 21:44:44 +00:00
test_estimator_imv_integration.py feat(tradein): integrate Avito IMV as 5th evaluation source (on-demand cached) (#452) 2026-05-23 13:08:22 +00:00
test_estimator_source_quota.py feat(tradein): source-quota in estimator analogs (fix Cian/Yandex starvation) (#491) 2026-05-23 20:59:58 +00:00
test_estimator_yandex_integration.py feat(yandex-valuation): link history rows to houses via match_or_create_house (#531) 2026-05-24 14:28:30 +00:00
test_extract_short_addr.py feat(tradein-estimator): tiered house-match S→H→W (#507) 2026-05-24 12:41:12 +00:00
test_matching.py fix(tradein-matching): pg_advisory_xact_lock to prevent duplicate house INSERTs (#501) 2026-05-24 10:52:10 +00:00
test_rbac.py feat(rbac): pilot scope → только /trade-in/** (#587) 2026-05-26 07:40:44 +00:00
test_scheduler.py feat(tradein): in-app scheduler — UI-managed schedule (replaces SSH crontab) (#482) 2026-05-23 14:54:25 +00:00
test_scrape_pipeline.py feat(tradein): scrape_pipeline.py — Avito full orchestrator (search → houses → detail) (#451) 2026-05-23 12:55:13 +00:00
test_scraper_settings.py feat(tradein): global scraper delay setting (applies across all scrapers) (#485) 2026-05-23 15:43:28 +00:00
test_search_api.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00
test_street_deals_endpoint.py fix(tradein): extract_street_name handles Nominatim reverse format (#557) 2026-05-24 21:03:42 +00:00
test_yandex_detail.py feat(tradein): yandex_detail.py — Product JSON-LD + DOM detail parser (#466) 2026-05-23 13:45:09 +00:00
test_yandex_helpers.py feat(tradein): yandex_helpers.py — JSON-LD + regex + date + NLP + money helpers (#456) 2026-05-23 13:14:11 +00:00
test_yandex_newbuilding.py feat(tradein): yandex_newbuilding.py — ЖК landing parser (149 selectors + NLP) (#467) 2026-05-23 13:46:45 +00:00
test_yandex_realty_serp.py feat(tradein): yandex_realty.py — DOM SERP refactor (vtorichka, 23 cards/page) 2026-05-23 16:21:47 +03:00
test_yandex_scrapers_delay_wiring.py feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484) 2026-05-23 15:28:34 +00:00
test_yandex_valuation.py fix(yandex-valuation): off-by-one cell index in _parse_row_cells (#552) 2026-05-24 19:57:17 +00:00
test_yandex_valuation_save.py feat(yandex-valuation): link history rows to houses via match_or_create_house (#531) 2026-05-24 14:28:30 +00:00