Light1YT
20b4a195b3
Builds on Phase 1 audit infra (PR #583). Forward-geocode 4141 houses без coords + reverse audit 4452 already-geocoded houses через Yandex Geocoder API. Verified mismatch 870м для «Малышева 125» — наш point в другом квартале от реального здания. Scripts (new): - `backfill_house_coords.py` (619 LOC) — driver: * Forward mode: UPDATE houses SET lat/lon когда precision ∈ {exact, number} * `--audit-only`: reverse геокод + distance check, flag >50м как mismatch * Per-row SAVEPOINT, idempotent через UNIQUE (house_id, audit_batch) * EKB locality bias (ll=60.6122,56.8389, spn=0.6,0.4) * Rate-limit 50ms между запросами, 25k/день квота Yandex покрывает 8.5k houses - `_yandex_reverse.py` (+123 LOC) — added `forward_via_api()`, precision/kind в YandexReverseResult dataclass - `audit_address_sample.sql` (-7 +47) — FIX: убрана зависимость от FDW `gendesign_ekb_districts_geom` (нет на проде). Простой `ORDER BY random()` без district stratification. Bind-param контракт сохранён (`:limit_per_district` * 8 = 200 default). Tests: - `test_backfill_house_coords.py` (510 LOC) — 20 tests: request shape, precision filter, UPDATE shape, backfill happy/imprecise/no_match/error, audit ok/mismatch/no_match, resumability, missing-key exit, mode routing - Existing `test_audit_address_mismatch.py` still green - 39/39 pass, ruff clean Decisions (см. PR body): - Precision filter = {exact, number} only (street/range/near → audit-imprecise) - Mismatch threshold 50м (consistent с Phase 1 report) - API-only (no Playwright fallback) — exit если key missing Closes #582 (Phase 2-3 portion; Phase 1 was #583)
47 lines
2 KiB
SQL
47 lines
2 KiB
SQL
-- audit_address_sample.sql
|
|
-- Random sample of EKB houses for the address-mismatch audit (issue #582).
|
|
--
|
|
-- Strategy:
|
|
-- 1. Filter to houses with non-null lat/lon and non-empty address.
|
|
-- 2. Random shuffle via `ORDER BY random()` — repeatable enough for spot
|
|
-- sampling without needing a stable PRNG seed (the audit table dedupes
|
|
-- via UNIQUE (house_id, audit_batch), so re-running gives idempotent
|
|
-- results regardless of which rows land in the sample first).
|
|
-- 3. Cap the result at :limit_per_district * 8 rows — keeps the bind-param
|
|
-- contract compatible with the old stratified sampler (`:limit_per_district`
|
|
-- is still honored, just multiplied by the assumed 8-district count).
|
|
--
|
|
-- Why no spatial stratification anymore:
|
|
-- The previous version JOINed to `gendesign_ekb_districts_geom` (FDW
|
|
-- polygon table) to bucket houses by admin district. That join is fine on
|
|
-- prod where FDW is wired, but it adds a dependency we don't need for
|
|
-- Phase 2-3 (backfill + canonical reverse). Aggregation by district at
|
|
-- report time still works — we re-derive district during the audit via
|
|
-- spatial containment in the report SQL when needed.
|
|
--
|
|
-- Bind param:
|
|
-- :limit_per_district — kept for back-compat with the audit driver.
|
|
-- Effective sample size = :limit_per_district * 8 (e.g. 25 → 200).
|
|
--
|
|
-- Columns returned:
|
|
-- id, address, lat, lon, district
|
|
-- `district` is always NULL here — the audit driver will reverse-derive it
|
|
-- from Yandex Geocoder response (Yandex returns admin component) or leave
|
|
-- it NULL if not present in the response.
|
|
--
|
|
-- NB: uses CAST(:x AS int) per project sql.md rule (psycopg v3 ignores ::type
|
|
-- after bind params).
|
|
|
|
SELECT
|
|
h.id,
|
|
h.address,
|
|
h.lat,
|
|
h.lon,
|
|
NULL::text AS district
|
|
FROM houses h
|
|
WHERE h.lat IS NOT NULL
|
|
AND h.lon IS NOT NULL
|
|
AND h.address IS NOT NULL
|
|
AND length(trim(h.address)) > 0
|
|
ORDER BY random()
|
|
LIMIT CAST(:limit_per_district AS int) * 8;
|