Light1YT
02ad9a1f58
Stratified 200-house Yandex reverse-geocode audit comparing our DB's (address, lat/lon) pair against what Yandex.Карты returns for those coordinates. Лежит фундамент данных для Phases 2-5 (canonical = Yandex per user decision). What ships: - Migration 066: address_mismatch_audit table + FDW foreign table gendesign_ekb_districts_geom для stratified sampling. - Sampling SQL: 8 EKB districts × 25 houses = 200. - Driver scripts/audit_address_mismatch.py с двумя режимами: API (если YANDEX_GEOCODER_API_KEY есть) и Playwright (CAPTCHA-aware, персистентный контекст). Resumable по --batch. - Report SQL: p50/p75/p95 distance, top-20 outliers, per-district breakdown. - 19 unit-тестов: normalize, distance bind, API parser, resume idempotency, captcha → status=blocked. Devops note: после деплоя миграции — выдать tradein_fdw_reader SELECT на gendesign.public.ekb_districts_geom перед первым запуском audit. Refs #582 (Phase 1 of 5). Phase 3 backfill заблокирован до Yandex API key (#402).
53 lines
1.6 KiB
SQL
53 lines
1.6 KiB
SQL
-- audit_address_sample.sql
|
||
-- Stratified sample of EKB houses for the address-mismatch audit (issue #582).
|
||
--
|
||
-- Strategy:
|
||
-- 1. Spatial-join houses → gendesign_ekb_districts_geom via ST_Contains
|
||
-- (only houses with lat/lon falling inside one of the 8 admin polygons).
|
||
-- 2. Partition by district, ROW_NUMBER() with a deterministic ORDER BY id
|
||
-- so repeated runs return the same sample (deterministic = matches the
|
||
-- "resumable per (house_id, audit_batch)" guarantee in 066 migration).
|
||
-- 3. Keep top :limit_per_district per district → ~200 rows total for 8
|
||
-- districts × 25.
|
||
--
|
||
-- Bind param:
|
||
-- :limit_per_district — default 25, expressed as :limit_per_district
|
||
-- (psql variables resolve in :name form for the report; for Python use
|
||
-- SQLAlchemy text() with bindparams).
|
||
--
|
||
-- Columns returned:
|
||
-- id, address, lat, lon, district
|
||
--
|
||
-- NB: uses CAST(:x AS int) per project sql.md rule (psycopg v3 ignores ::type
|
||
-- after bind params).
|
||
|
||
WITH houses_in_districts AS (
|
||
SELECT
|
||
h.id,
|
||
h.address,
|
||
h.lat,
|
||
h.lon,
|
||
d.district_name AS district,
|
||
h.geom
|
||
FROM houses h
|
||
JOIN gendesign_ekb_districts_geom d
|
||
ON ST_Contains(d.geom, h.geom)
|
||
WHERE h.lat IS NOT NULL
|
||
AND h.lon IS NOT NULL
|
||
AND h.address IS NOT NULL
|
||
AND length(trim(h.address)) > 0
|
||
),
|
||
ranked AS (
|
||
SELECT
|
||
id,
|
||
address,
|
||
lat,
|
||
lon,
|
||
district,
|
||
ROW_NUMBER() OVER (PARTITION BY district ORDER BY id) AS rn
|
||
FROM houses_in_districts
|
||
)
|
||
SELECT id, address, lat, lon, district
|
||
FROM ranked
|
||
WHERE rn <= CAST(:limit_per_district AS int)
|
||
ORDER BY district, id;
|