Complements the NULL-geom 500 fix. cad_exists_in_db (docstring: "is there
GEOMETRY") checked only row existence, not geom IS NOT NULL — so for the ~964
meta-but-NULL-geom parcels it returned True. Consequence after the 500 fix:
such a parcel fell into the analyze fallback, find_or_enqueue_fetch step 2 saw
cad_exists_in_db=True → returned ("ready", None) → NO NSPD fetch enqueued →
analyze looped to a 202 with job_id=null and the parcel was stuck "fetching"
forever (never pulled real geometry, never resolved).
Fix: add `AND geom IS NOT NULL` to all three EXISTS branches (aligns the
function with its docstring). Now a NULL-geom parcel → cad_exists_in_db=False →
a real NSPD fetch is enqueued (202 + real job_id) → geometry populates →
re-poll → analyze succeeds (or 404 not_in_nspd if NSPD lacks it). No more
stuck-202. Valid-geom parcels unaffected. All 3 callers want geometry-presence
semantics. 37 analyze/fetch/by-bbox tests green. Refs #944.
POST /analyze 500'd (TypeError: float(None)) for ~964 cad_parcels_geom rows
that have a meta row but geom IS NULL: the resolution query returned the row
(no NULL filter), geom_wkt became NULL → ST_Centroid(NULL) → NULL lat/lon →
float(centroid_row["lat"]) crashed (the `if centroid_row else` guard checked
row existence, not NULL values). A user entering any such "random parcel" hit
a 500.
Fix:
- Add `AND geom IS NOT NULL` to all three geometry-resolution queries (initial
UNION, inline-refetch UNION, geom_row WKT subquery) so a NULL-geometry parcel
is treated as "no usable geometry" → falls into the #93 graceful fallback
(enqueue NSPD on-demand fetch → 202 polling), exactly like a parcel absent
from the DB. The fetch then populates real geometry and analyze succeeds.
- Defensive: centroid_lat/lon now check the VALUE is not None (not just row
existence) before float(), falling back to EKB center.
Verified: badformat→400, not-in-DB→202(fetching), valid parcel→200 unaffected.
39 analyze/by-bbox tests green. Refs #944.
The whole backend api/v1 test suite hits `app` via TestClient mimo Caddy, so
rbac_guard (app/main.py) 401'd every authed request (no X-Authenticated-User)
— 112 tests red, the suite effectively dead (which is how #994's district 500
shipped uncaught). Add `settings.testing` (default False — prod RBAC untouched)
+ a strictly-gated bypass at the top of rbac_guard; tests/conftest.py enables it
(env TESTING=1 + settings.testing=True before app import).
Security: bypass fires ONLY when settings.testing is True, which is set NOWHERE
in prod (default False; only tests/conftest.py flips it). RBAC's 401/403 logic
stays covered by tests/test_rbac.py (its own middleware copy, ignores the flag).
Effect: 112 → 42 failed, 1590 → 1660 passed. Remaining 42 (+5 mv_layout env
errors) are stale-mock/assertion drift + env-required tests — fixed in CI-rehab
2/3. Foundation for a real Forgejo pytest gate (3/3).
Refs #944.
LIVE BUG (from #994, already merged): the analysis_runs persist call-site in
analyze_parcel extracted district via result_payload["district"]["district_name"]
— but a district dict without that key (real data: partial district lookup)
raised KeyError OUTSIDE the best-effort SAVEPOINT (extraction is at the call
site, not inside persist_analysis_run) → 500 on the LIVE /analyze endpoint.
Fix: .get("district_name") → None instead of raising. Caught by reviving the
analyze test suite (below).
Test-infra (the suite was un-runnable, which is why the bug shipped):
- Lazy-import WeasyPrint in layout_tz_pdf.py + trade_in_pdf.py (matches the
existing report_pdf.py / snapshot_pdf.py pattern). The eager top-level imports
made `app.main` (→ parcels/trade_in routers) un-importable on hosts without
WeasyPrint native libs (e.g. macOS dev), breaking pytest COLLECTION of the
whole api/v1 suite. WeasyPrint is still imported when a PDF is actually rendered.
- tests/conftest.py: autouse fixture clears app.dependency_overrides after each
test (anti-leak — a leaked get_db override caused real-DB connection attempts).
- test_parcel_by_bbox.py: rewrite get_db mocking from patch("...get_db") (a no-op
— FastAPI Depends holds the original ref) to app.dependency_overrides[get_db],
add explicit X-Authenticated-User header (RBAC gate), patch latest_run_dates,
+ a new test asserting last_analysis_date from a latest run (#994). 5/5 green.
NOTE: reviving collectability exposes PRE-EXISTING rot in other api/v1 suites
(analyze/admin/best_layouts: RBAC-header + stale-assertion/mock drift) — those
are NOT regressions from this PR (they were uncollectable before) and are
tracked separately for a deliberate suite-rehab + CI test-gate effort.
Refs #994.
Persist each successful POST /analyze into analysis_runs (migration 127):
best-effort SAVEPOINT-wrapped INSERT in a thin analysis_runs repository, then
explicit db.commit() (get_db has no commit-on-success; SAVEPOINT RELEASE alone
does not persist — without the commit the row rolls back on request teardown,
a silent no-op caught in code review). A persist failure never breaks the
response or poisons the session. Result serialized via jsonable_encoder→
json.dumps (handles embedded Pydantic models/dates); confidence/status
normalized to satisfy the 127 CHECK constraints (unknown→NULL/'complete').
Populate by-bbox last_analysis_date from v_analysis_runs_latest via a single
batch query (no N+1), replacing the #307 placeholder. The read is best-effort
wrapped too — a view-missing deploy window or future drift must not 500 the
map (falls back to last_analysis_date=None).
Additive only — analyze response shape unchanged. Tests: by-bbox suite patches
latest_run_dates (mock-db returns same rows for any query) + new test asserts
ISO last_analysis_date from a latest run. Analyze mock-suite unaffected
(graceful side_effect overflow + best-effort persist absorb the extra INSERT).
Incidental: ruff-format fixed one pre-existing f-string spacing (line ~2391)
the format hook flags on touch.
Closes#994. Refs #961.
Replaces tradein.cad_buildings snapshot with live postgres_fdw foreign table reading gendesign.v_tradein_cad_buildings. Fixes /trade-in/api/v1/geocode/reverse 500 (Nominatim ban) and address_not_geocoded for cadastre addresses (e.g. Хохрякова 81).
Security (deep-review fixes):
- 100_tradein_fdw_role.sql: passwordless CREATE ROLE; password set by deploy.yml ALTER ROLE bootstrap reading GENDESIGN_FDW_PASSWORD from backend/.env.runtime (via psql :'pw' var → format %L — injection-safe).
- core/fdw.py: regex whitelist [A-Za-z0-9_-]{32,256} on password, ValueError without echoing value, try/rollback on commit.
- 060_postgres_fdw_extension.sql: connect_timeout='3' on FOREIGN SERVER + ALTER ADD/SET fallback.
- geocoder.py: _cadastral_forward_sync / _cadastral_reverse_sync wrapped in asyncio.to_thread.
- 100_*.sql: REVOKE ALL ON ALL TABLES/SEQUENCES/FUNCTIONS IN SCHEMA public; only GRANT SELECT on v_tradein_cad_buildings.
- pg_user_mappings query handles PUBLIC mapping (usename IS NULL).
Tests: 3 SQL-injection guards on ensure_fdw_user_mapping + rewritten cadastral suite.
PR #335 использовал `with db.begin_nested():` SAVEPOINT для per-row UPDATE,
но outer transaction никогда не commit'ился. После цикла Celery task
закрывал session (`db.close()` в finally) → autobegin'd outer tx
rollback → все UPDATE'ы теряются.
Симптом: task завершается с processed=3 succeeded=3, в Glitchtip нет ошибок,
но в БД ни одного catalog_scraped_at != NULL.
Fix: добавлен db.commit() после цикла + rollback на ошибку commit'а.
