Move JobSettings UI from /admin/jobs/settings into JobSettingsPanel component,
render it alongside new BulkGeoPanel on /admin/scrape/all. BulkGeoPanel has a
parallelism input (1-10, default 5) and POSTs /admin/scrape/geo/bulk. Remove
the standalone /admin/jobs/ route and the nav tab pointing to it.
Объединяю несколько связанных изменений вокруг NSPD geo bulk-fetcher:
Adapt to rosreestr2coord v5 API:
- nspd_lite.fetch_via_rosreestr2coord: drop `delay` kwarg from Area()
(removed upstream in v5); keep it in our function signature for
backward-compat, comment why.
- nspd_geo worker: add explicit time.sleep(rate_ms/1000) after lib-branch
fetch — in v4 the library throttled internally via delay, in v5
rate limiting is the caller's job. Без этого получали Area.__init__()
unexpected kwarg `delay` на каждом target.
Drop use_rosreestr2coord switch:
- Removed urllib-vs-lib choice everywhere. We always use community
rosreestr2coord library — авторы регулярно обновляют WAF-tricks,
наш urllib-fetcher (fetch_geoportal) уже неактуален.
- admin_scrape.py: Pydantic schema, INSERT, SELECT, API response
cleaned of `use_rosreestr2coord`.
- nspd_geo.enqueue_geo_job: param dropped, INSERT shrunk.
- worker process loop: dropped `if use_lib:` branch + import of
fetch_geoportal.
- frontend/geo/page.tsx: removed checkbox + GeoJob.use_rosreestr2coord
field + POST body field.
DB column drop:
- data/sql/78_drop_use_rosreestr2coord.sql (NEW):
DROP COLUMN nspd_geo_jobs.use_rosreestr2coord + CREATE OR REPLACE
VIEW v_scrape_runs_unified (которая depended on the column).
- data/sql/77_nspd_geo_jobs.sql: cleaned historical DDL for fresh setups.
- Migration applied to prod (in-conversation via postgres MCP).
Frontend polish:
- Thematic ID changed from free-form number input to labeled select
(1=parcel / 2=quarter / 4=admin / 5=building / 7=zone / 15=complex).
- Auto-sync thematic_id from Job kind on change (override possible).
- ScrapeLogsPanel: extended union type with "nspd_geo" + fixed
/admin/scrape/geo to pass scraperType="nspd_geo" (was "nspd",
filtering empty legacy nspd_scrape_log table; real logs live in
nspd_geo_log via v_scrape_log_unified).
Verified: ruff ✓, tsc --noEmit ✓, migration ran (BEGIN..COMMIT clean).
Deploy order safe: prod column уже удалена → новый backend код, который
не INSERT'ит use_rosreestr2coord, совпадёт со схемой после deploy.
Two fixes for the NSPD geo bulk-fetcher:
1. rosreestr2coord was listed in pyproject.toml but missing from uv.lock,
so Docker's `uv sync --frozen` didn't install it. NSPD jobs failed every
target with `No module named 'rosreestr2coord'`. Ran `uv lock` to add it
(v5.3.3) plus requests + charset-normalizer transitive deps.
2. /admin/scrape/geo and /admin/scrape/objective lacked the log panel that
/admin/scrape (DomRF) had. Extracted ScrapeLogsPanel component over the
unified /api/v1/admin/scrape/all/logs endpoint with scraper_type filter
and wired into both pages.
Tests: ruff ✓, tsc --noEmit ✓
