Commit graph

159 commits

Author SHA1 Message Date
lekss361
f4e74916be feat(site-finder): P1 geometry suitability score (#45)
Backend (parcels.py):
- _polygon_suitability() — Shapely metrics on parcel WGS84 polygon:
  area_ha/m2 (projected via cos(lat) к UTM-like meters), perimeter_m,
  aspect_ratio (long/short side of MABR), convex_hull_ratio (area / hull area),
  min_inscribed_rect_dim_m.
- Composite suitability score 0..1:
  base = area_subscore (0 при <0.2 ha, linear → 1 при ≥0.5 ha)
  −0.3 если aspect_ratio > 5 (вытянутый)
  −0.3 если convex_hull_ratio < 0.65 (изрезанный)
  −0.5 если min_inscribed_rect_dim_m < 30 (узкий)
- Label: микро / подходящий / сложная форма / слабо подходит
- Recommendation: "для МКД 16+ нужно >0.3 ha + минимум 40м"
- В analyze response → geometry_suitability.

Frontend:
- Новый GeometrySuitabilityBlock.tsx с color-coded badge + метрики grid +
  penalties + recommendation
- Добавлен в LandTab (выше "Зонирование (ПЗЗ)")
- TS типы расширены: GeometrySuitability

Closes #45.
2026-05-12 00:51:08 +03:00
lekss361
a35a3d02ea
feat(site-finder): deep-link tabs via ?tab= URL param (#86)
* feat(site-finder): deep-link tabs via ?tab=env|land|market URL param

Tab state now syncs with URL — можно поделиться ссылкой на конкретный
таб (например /site-finder?tab=market для тренда цен по последнему cad).

- useRouter + useSearchParams (Next 15 app router, client-component)
- router.replace (не push) чтобы tab switches не засоряли history
- useEffect on searchParams → back/forward работают
- overview = default, не пишется в URL (clean)
- Type-guard isTabId(): только valid values из URL переключают state

* fix(site-finder): address PR #86 auto-review minor recommendations

- Use usePathname() for router.replace fallback (avoid dangling "?" in Next 15)
- Remove stale-closure guard in useEffect — React bails Object.is, no eslint-disable
- Wrap page in <Suspense> for useSearchParams (Next 15 App Router req for static)
- Lazy useState initializer — skip wasted CPU on re-renders

Per auto-review on c47e56b.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 00:33:49 +03:00
lekss361
aadf47709b
docs(claude): mandate branch+PR workflow, ban direct push to main (#84)
* docs(claude): mandate branch+PR workflow, ban direct push to main

* docs(claude): sync TL;DR + Don't-do-these with new PR workflow

Address auto-review feedback:
- TL;DR rule 5 → branch+PR mandate (was: "user commits themselves")
- TL;DR rule 6 → no --no-verify/--force/--amend (split from rule 5)
- Don't-do-these → drop conflicting "no auto-commit" line, add
  "no direct push to main" + "no merge without approval"
- Critical workflow rules: add explicit rule 2 — worker-agents don't
  commit themselves; main session commits on feature branch
- Restore "Squash on merge" wording in PR section (was dropped)
- Auto-mode section: clarify that main session does commits on feature
  branch (worker-agents leave changes staged)

* fix(claude): Code-reviewer diagram had stale 'push origin HEAD:main' — replace with branch+PR flow

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 00:16:06 +03:00
lekss361
053ce29a40 docs(claude): add code-reviewer agent — review workflow before push 2026-05-11 23:44:32 +03:00
lekss361
13119f9cb5 feat(docs): /docs/b2b-channels (rendered) + /docs/b2b-channels.md (raw) for sharing 2026-05-11 23:36:02 +03:00
lekss361
59466de811 fix(deploy): aggressive docker image prune (drop until=72h filter to avoid 50GB buildup) 2026-05-11 23:26:31 +03:00
lekss361
a1201aa94e feat(site-finder): tabbed UX - hero KPI band + 4 tabs (Overview/Env/Land/Market) + sticky map 2026-05-11 23:21:39 +03:00
lekss361
199adc31e6 fix(site-finder): UX - sticky map, collapsible sections, score header sticky 2026-05-11 23:10:56 +03:00
lekss361
99ca42e5d0 fix(site-finder): isochrones map re-renders on mode change (key includes coords hash) 2026-05-11 23:03:25 +03:00
lekss361
2cbcea9e73 fix(zoning): NSPD blocks bot access — fallback to deep-links (PKK6 API closed 2024) 2026-05-11 22:56:50 +03:00
lekss361
49eadeb9ce fix(pzz-sync): follow_redirects=True (PKK6 returns 301) 2026-05-11 22:50:46 +03:00
lekss361
1c1ecad8b8 fix(pzz-sync): disable SSL verify for Rosreestr PKK6 (self-signed cert chain) 2026-05-11 22:44:53 +03:00
lekss361
2e83d62237 feat(site-finder): center distance bonus display + success_recommendation block 2026-05-11 22:40:46 +03:00
lekss361
ab0647e4d5 feat(site-finder): distance to EKB center + success quartirography recommendation 2026-05-11 22:35:21 +03:00
lekss361
c31da62e8d feat(analytics): recommend_mix v3.1-v3.4 - noise + 2D competitors + 24m cap + success-driven 2026-05-11 22:19:41 +03:00
lekss361
369b5a4706 feat(db): v_bucket_success_score view (success-driven mix #25) 2026-05-11 22:04:22 +03:00
lekss361
8be539ddc6 feat(site-finder): ПЗЗ territorial zones from Rosreestr PKK6 + zoning in analyze 2026-05-11 21:51:51 +03:00
lekss361
5b03d6d799 feat(site-finder): isochrones UI + networks VKH + OSM substations 2026-05-11 21:48:21 +03:00
lekss361
923f250926 feat(site-finder): /parcels/{cad}/isochrones via OpenRouteService 2026-05-11 21:44:22 +03:00
lekss361
6ce634a9b9 feat(site-finder): utilities (power/pipeline) + fix Внешние факторы layout 2026-05-11 21:38:20 +03:00
lekss361
2855a01ca7 feat(site-finder): v3.5 - seasonal weather + hydrology + geotech risk 2026-05-11 21:32:07 +03:00
lekss361
7bb51aa838 feat(site-finder): v3.4 - weather enrichment (temp/precip/UV) + geology stub 2026-05-11 21:17:57 +03:00
lekss361
7900dc5238 fix(site-finder): WKT LINESTRING needs commas + rosreestr_deals real column names 2026-05-11 20:58:49 +03:00
lekss361
232c81eae9 feat(site-finder): v3.3 - score label + market_trend + multi-thematic bulk 2026-05-11 20:51:59 +03:00
lekss361
1e9d32ee3c feat(site-finder): v3.2 — noise + air quality + wind analytics 2026-05-11 20:42:37 +03:00
lekss361
a2e8a49e6d feat(db): osm_noise_sources_ekb for site-finder noise scoring 2026-05-11 20:35:27 +03:00
lekss361
d08d06d8a8 fix(geo): parcels (thematic_id=1) come from rosreestr2coord in WGS84 — drop ST_Transform 2026-05-11 20:09:58 +03:00
lekss361
81cd7499f6 feat(site-finder): cad_parcels_geom migration + POI markers on map + bulk-job UX 2026-05-11 19:54:56 +03:00
lekss361
f419900968 feat(site-finder): v3.1 — cad_parcels_geom, analyze fallback, POI lat/lon, OSM expand
- nspd_geo: add _save_parcel() for thematic_id=1 → cad_parcels_geom (UPSERT,
  ST_Transform from Web Mercator); _persist_target now handles 1/2/5
- parcels.py: analyze endpoint geom lookup extended with cad_parcels_geom as
  3rd fallback source (after cad_quarters_geom, cad_buildings); both SELECT
  and WKT subqueries updated
- parcels.py: POI score_breakdown items now include lat/lon for map markers
- poi_loader: OSM_CATEGORIES expanded — college+university→school,
  hypermarket→shop_supermarket; coverage +3 tag pairs
2026-05-11 19:52:19 +03:00
lekss361
ea3d58fa7d feat(routing): /sf -> /site-finder permanent redirect (short URL alias) 2026-05-11 19:49:26 +03:00
lekss361
e6c062cc5b feat(home): site-finder no longer WIP + admin/scrape/all + buildings in object card desc 2026-05-11 19:36:43 +03:00
lekss361
a7d7dc9667 fix(site-finder): dedupe competitors — same domrf_kn_objects 3x snapshots bug 2026-05-11 19:04:29 +03:00
lekss361
3aeda297ee fix(poi-sync): split Overpass query per category (504 Gateway Timeout) 2026-05-11 18:54:28 +03:00
lekss361
93f13795bb fix(poi-sync): add User-Agent header — Overpass returns 406 for python-httpx default UA 2026-05-11 18:46:47 +03:00
lekss361
9742219ef6 debug(poi-sync): DB breadcrumbs to nspd_geo_log for silent-fail diagnosis 2026-05-11 18:37:43 +03:00
lekss361
f81bd710c0 feat(api): POST /admin/scrape/poi-sync — manual OSM POI sync trigger 2026-05-11 18:12:37 +03:00
lekss361
a4e8fff5ac feat(site-finder): POST /parcels/{cad}/analyze endpoint + UI
Backend:
- analyze endpoint: cad → geom lookup → POI within 1km → district
  context → competitors within 3km. Tram_stop carries negative weight.

Frontend /site-finder:
- CadInput regex-validated (default 66:41:0204016:10)
- SiteMap (Leaflet via next/dynamic) with parcel polygon + legend
- ScoreCard color-coded (>5 green, 2-5 yellow, <2 red) with tram-warning
- CompetitorTable top-20 with same-district highlight
- useSiteAnalysis hook + TS types
2026-05-11 18:11:58 +03:00
lekss361
6dea34186b feat(site-finder): OSM POI loader + Celery weekly sync
- New poi_loader.py: Overpass QL query for 13 POI categories in EKB bbox,
  UPSERT into osm_poi_ekb with soft 2-year filter tracking (skipped_old counter)
- New poi_sync.py Celery task wrapping sync_poi_to_db()
- celery_app.py: include poi_sync module, add poi-sync-weekly beat entry (Mon 03:00 MSK)
2026-05-11 18:08:34 +03:00
lekss361
688e094844 feat(db): osm_poi_ekb table for site-finder POI analysis 2026-05-11 18:06:28 +03:00
lekss361
ceaf553eb7 fix(analytics): recommend page UI — caveat badge wrap + headline baseline label
Two UX nits:
- Caveats badge had whiteSpace:nowrap on a 250-char data_caveat → overflowed
  visibly outside the section in the screenshot. Drop nowrap, keep small
  font + light grey background. Reads as a compact note.
- Headline KPIs (369 млн ₽ · 19.4 мес · темп 6.0 · ликв 97/100) are
  computed by backend at price_factor=1.0, while live KPI cards below
  recompute with the current slider position. When user moves price to
  +40% the discrepancy is confusing. Append small grey '(базовая цена)'
  marker after headline with a tooltip explaining baseline vs live.
2026-05-11 17:53:35 +03:00
lekss361
0de32cd2ce fix(analytics): dedupe comparables — domrf_kn_objects has 3x snapshots per obj_id
The comparables block on /analytics/recommend was showing the same ЖК
multiple times (ЖК СТАРТ × 3, ЭХО ЛЕСА × 2 etc) because domrf_kn_objects
has ~3 historical snapshots per obj_id. The previous query joined all
rows and LIMIT 5 cherry-picked duplicates by flat_count.

Fix: wrap base in CTE `latest_obj` with `DISTINCT ON (obj_id) ORDER BY
obj_id, snapshot_date DESC` to pick only the latest snapshot per object
before sorting by flat_count.
2026-05-11 17:52:37 +03:00
lekss361
9658baa6b2 fix(deploy): keep last 2 SHA-tagged images per repo, drop older
Previously `docker image prune -af --filter until=72h` only removed
images older than 72h. With multiple deploys/day we accumulate many
fresh (<72h) tagged images — each gendesign-backend / worker / frontend
SHA-tag is 500MB-1GB, filling /var/lib/docker.

New logic: for each ghcr.io/lekss361/gendesign-* repo, list tags sorted
by Created DESC, skip :latest, keep top 2, remove the rest. Still runs
the standard prune after for dangling layers + build cache.
2026-05-11 17:45:20 +03:00
lekss361
03e572330b feat(analytics): recommend page shows cad_quarter, buildings count, district stats 2026-05-11 17:37:57 +03:00
lekss361
2847262953 feat(analytics): extend recommend comparables with cad_quarter, lat/lon, buildings_n 2026-05-11 17:30:09 +03:00
lekss361
b18f44e1ad feat(analytics): add cad_quarter_count to districts endpoint 2026-05-11 17:29:40 +03:00
lekss361
ae86d62e9b fix(worker): worker_ready geo-resume blocked by early return in kn-resume
Root cause of "auto-resume never fired": the kn_scrape_runs resume section
hit `if not rows: return` (and similar `return` in except) before reaching
the nspd_geo_jobs resume section. Whenever there were no zombie kn-runs
(the normal case), the handler bailed out and geo jobs stayed forever
'running' with stale heartbeats — users had to manual cancel/resume after
every deploy.

Fix: don't return early. Initialize `ids = []`, only run UPDATE if rows
exist, drop the inner `return` from exception branch. The for-loop over
ids becomes a no-op when empty, and execution falls through to the geo
section. Same pattern as the breadcrumb above — fail soft, continue.

cleanup_zombies beat task (added in caa467f) stays as belt-and-suspenders
in case worker_ready signal ever misbehaves again.
2026-05-11 17:12:31 +03:00
lekss361
caa467fb7e fix(worker): periodic zombie cleanup via beat instead of worker_ready
worker_ready signal handler was NOT firing in our setup (verified via
DB breadcrumb after 3 deploys — zero rows of stage='worker_ready' in
nspd_geo_log). Root cause of unreliability unknown — possibly Celery
internals, possibly compose recreate timing. Either way, after every
redeploy users had to manually cancel/resume jobs to keep them moving.

Replace signal-based resume with periodic beat task:
- cleanup_zombies runs every minute (* * * * *)
- Finds nspd_geo_jobs in status running/paused with heartbeat >2 min stale
- Sets status='queued' + apply_async with queue=geo
- Idempotent — if no zombies, no-op

worker_ready handler kept (with FK-fix breadcrumb on NULL job_id) for
diagnostic purposes — if signal ever does fire, we'll have evidence.
2026-05-11 17:02:29 +03:00
lekss361
17feaa408e debug(worker): persistent breadcrumb in nspd_geo_log on worker_ready 2026-05-11 16:51:50 +03:00
lekss361
f539e0e414 fix(admin): bump UI stale threshold 60/90s -> 300s
The 'stale' badge in /admin/scrape/* tables was a UI-only label
computed as Date.now() - heartbeat_at > 60s (90s for geo). Heartbeats
naturally lag past 60s during WAF backoff (30s) or NSPD HTTP timeout
(15s) — every page render flashed running jobs as 'stale' even though
they were processing.

300s threshold matches the real auto-resume threshold in the worker
(jobs are zombies only after >5 min of silence). Less false-positive
churn in the UI.
2026-05-11 16:49:37 +03:00
lekss361
fdd5b4e0e2 fix(worker): dispose SQLA engine in worker_process_init + log resume scan
Two related fixes:

1. worker_process_init handler disposes the SQLAlchemy engine in each
   prefork child. Without this, child processes inherit open psycopg
   sockets from the parent. First use in a child raises
   ProgrammingError: can't change 'autocommit' now: connection in
   transaction status INTRANS. This was killing 1 of every 5 parallel
   geo jobs on cold start (job 13 in latest bulk run).

2. Add logger.info at start/end of worker_ready resume handler so we
   can see in worker logs whether it actually fired and how many jobs
   it resumed.
2026-05-11 16:40:21 +03:00