269 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
lekss361	56bf28758c	Merge pull request 'feat(#115): Leaflet layer toggle для connection points (Макс KILLER)' (#230) from feat/connection-points-leaflet-layer into main	2026-05-16 20:00:55 +00:00
lekss361	e7a5ba8f81	Merge pull request 'fix(#112): competitors avg_price_per_m2 всегда None — убран WHERE status='sold'' (#228) from fix/issue112-sold-filter-avg-price into main	2026-05-16 19:52:44 +00:00
lekss361	a7671513c9	feat(#115): Leaflet layer toggle для connection points (Макс KILLER) ... - ConnectionPointsLayer.tsx — per-category CircleMarker слой внутри MapContainer (electricity/gas/water/heat/sewage/telecom/other, классификация по keywords) - CpLayerControlPanel.tsx — toggle panel под картой: checkbox per-category + count, toggle-all, summary badges (ближайший, охранная зона) - SiteMap.tsx — принимает connectionPoints?: ConnectionPointsResponse, управляет visibleCategories state, рендерит оба новых компонента - page.tsx — useConnectionPoints(data?.cad_num) -> передаёт в SiteMap - Empty-state: "0 точек подключения" при dump_available=false или пустом ответе	2026-05-16 22:51:12 +03:00
lekss361	a6e4ff0407	feat(#114): seed 3 default weight presets + include_system API param ... Add SQL migration 100_user_weight_profiles_default_seed.sql with system presets Эконом/Комфорт/Бизнес (user_id='__system__'). Migration is idempotent via ON CONFLICT DO UPDATE. Backend: - weight_profiles.py: add SYSTEM_USER_ID constant + list_profiles_with_system() - admin_weight_profiles.py: add include_system query param to GET list endpoint Tests: 3 new tests covering include_system flag and service sentinel behaviour.	2026-05-16 22:49:56 +03:00
lekss361	31581cedd2	fix(#112): remove broken status='sold' filter from competitors avg_price query ... domrf_kn_flats.status is NULL in ~99.8% of rows, so WHERE status='sold' always returned 0 rows and avg_price_per_m2 was always None. Drop the filter; AVG over all rows with price_per_m2 IS NOT NULL is semantically correct for a complex-level price estimate. Adds regression test test_competitors_avg_price_populated (Issue #227).	2026-05-16 22:46:10 +03:00
lekss361	e4852b0ec9	Merge pull request 'fix(beat): use /tmp/celerybeat-schedule (workdir /app not writable by app user)' (#226) from fix/beat-celerybeat-schedule-permission into main	2026-05-16 19:06:27 +00:00
lekss361	718a70f6e9	fix(beat): use /tmp/celerybeat-schedule (workdir /app not writable by app user) ... Beat был в restart-loop после force-recreate в PR #225 deploy: beat raised exception <class '_gdbm.error'>: error(13, 'Permission denied') shelve.py / dbm/__init__.py — попытка создать 'celerybeat-schedule' в CWD. Root cause - backend/Dockerfile runner stage: WORKDIR /app создаётся под root → owner root:root. - COPY --chown=app:app переносит файлы под app, но саму директорию /app не chown'ит. - USER app → app (uid 1000) не может создавать новые файлы в /app/ (нет write permission на parent dir). - До PR #225 beat container не пересоздавался force-recreate, файл schedule существовал в writable forme из inherited state. Force-recreate сбросил. Fix - --schedule=/tmp/celerybeat-schedule — /tmp всегда writable. - Schedule-файл хранит только last_run_at для periodic tasks; потеря на container restart OK, beat перестраивает из celery_app.conf.beat_schedule. Permanent fix (отдельный PR — runner Dockerfile) - Добавить `chown app:app /app` в runner stage после WORKDIR. Не делаю в этом PR — хотфикс минимальный, runner стейдж шарится с FastAPI image и worker, риск регрессии больше чем у однострочного --schedule override.	2026-05-16 22:03:45 +03:00
lekss361	b5379fb135	Merge pull request 'fix(infra): wire GlitchTip DSNs (build-arg + env_file) + reject sentry.io promote' (#225) from fix/glitchtip-wire-prod-dsns into main	2026-05-16 18:51:41 +00:00
lekss361	e64fbd670b	fix(infra): wire GlitchTip DSNs (build-arg + env_file) + reject sentry.io promote ... Problem - GlitchTip Issues = 0 за всё время, хотя backend + frontend SDK интегрированы (PR #207, #208). - Старый Sentry.io продолжает получать события — user видит уведомления оттуда. Root cause - frontend/Dockerfile не имеет ARG NEXT_PUBLIC_GLITCHTIP_DSN → `npm run build` бьёт с пустым env var → Next.js инлайнит undefined → SDK init guard `if (dsn)` skips. Chrome-devtools check на prod bundle подтвердил: ни в одном из 9 chunks DSN-строка не запечена; `i.env.NEXT_PUBLIC_GLITCHTIP_DSN` evaluates to undefined. - .forgejo/workflows/deploy.yml build-frontend не передавал build-args. - На VPS backend/.env.runtime содержит legacy SENTRY_DSN=...@sentry.io/... config.py:_promote_legacy_sentry_dsn слепо промоутит его в glitchtip_dsn → SDK шлёт в чужой Sentry. GLITCHTIP_DSN там не задан. - deploy.yml SSH-скрипт никогда не редактировал SENTRY_DSN/GLITCHTIP_DSN в .env.runtime. Solution 1. frontend/Dockerfile: ARG NEXT_PUBLIC_GLITCHTIP_DSN + NEXT_PUBLIC_ENVIRONMENT с пустыми defaults, ENV-mirror перед `npm run build`. Локальный build без build-args работает по-прежнему (no-op DSN). 2. .forgejo/workflows/deploy.yml: - build-frontend: `build-args` передаёт `secrets.GLITCHTIP_FRONTEND_DSN` + NEXT_PUBLIC_ENVIRONMENT=production. Инвалидирует cache → frontend image пересоберётся (expected). - deploy step: GLITCHTIP_BACKEND_DSN через secret, в SSH-скрипте: a) `sed -i '/^SENTRY_DSN=/d' backend/.env.runtime` — снести legacy b) upsert GLITCHTIP_DSN (sed/printf) тем же паттерном что SENTRY_RELEASE c) `compose up -d --force-recreate --no-deps backend worker beat` — обычный `up -d` не перечитывает env_file без image change. 3. backend/app/core/config.py: _promote_legacy_sentry_dsn ужесточён — принимает SENTRY_DSN только если host == errors.gendsgn.ru. Для других URLs (sentry.io) выдаёт UserWarning и НЕ промоутит. Anti-regression на случай если SENTRY_DSN снова окажется в .env.runtime после ручного вмешательства. Required Forgejo secrets (Settings → Actions → Secrets) - GLITCHTIP_BACKEND_DSN = https://3d6e291003e142458957490c83559867@errors.gendsgn.ru/1 - GLITCHTIP_FRONTEND_DSN = https://5d7bc85e300c4e80a8554ccc818ff56d@errors.gendsgn.ru/2 DSNs публичны (видны в browser bundle) — secrets ради build-time injection, не для конфиденциальности. Если secrets не заданы → deploy succeeds, SDK no-op, без регрессии. Test plan - Verify Forgejo deploy.yml зелёный после merge - chrome-devtools: открыть gendsgn.ru → search bundle на DSN string → должна быть запечена строка errors.gendsgn.ru/2 - Trigger frontend error → POST к errors.gendsgn.ru/api/2/envelope/ - Backend: curl на endpoint вызывающий 500 → событие в GlitchTip backend project - GlitchTip dashboard https://errors.gendsgn.ru/gendesign/issues — Issues > 0 References - vault: meta/00_credentials.md (DSNs + incident notes 2026-05-16) - vault: decisions/Dec_GlitchTip_Frontend_Sentry_SDK.md (env contract) - vault: fixes/fixes-MOC.md (#204 backend SDK init)	2026-05-16 21:48:28 +03:00
lekss361	ea7a8f4aeb	Merge pull request 'fix(sql): escape apostrophe in migration 99 COMMENT (deploy hotfix)' (#224) from fix/migration-99-apostrophe into main	2026-05-16 18:24:18 +00:00
lekss361	f704785428	fix(sql): escape apostrophe in migration 99 COMMENT (deploy hotfix) ... PR #222 deploy failed на 99_nspd_entities_denorm.sql: ERROR: syntax error at or near "а" LINE 2: '3-сегментный квартал последнего harvest'а. Используется... Unescaped apostrophe в COMMENT string closes literal early. Fix: harvest'а → harvest''а (SQL escape). Других unescaped апострофов в SQL литералах нет (другие 3 — в SQL комментариях '--' , безопасно).	2026-05-16 21:02:00 +03:00
lekss361	4860cc790a	Merge pull request 'feat(nspd): denorm nspd_parcels + nspd_buildings (#94 PR3 of 4)' (#222) from feat/94-pr3-denorm-entities into main	2026-05-16 17:56:59 +00:00
lekss361	813b8bd589	Merge pull request 'fix(infra): GlitchTip EMAIL_URL dummy:// → consolemail:// (crashes Django on start)' (#223) from fix/glitchtip-email-url-consolemail into main	2026-05-16 17:56:53 +00:00
lekss361	1ec0a720fb	fix(infra): GlitchTip EMAIL_URL dummy:// → consolemail:// (django-environ schema)	2026-05-16 20:38:37 +03:00
lekss361	db1ff5b1fc	feat(nspd): denorm nspd_parcels + nspd_buildings tables (#94 PR3 of 4) ... - data/sql/99_nspd_entities_denorm.sql: 2 tables (cad_num PK) + 6 indexes (quarter_cad, GIST geom, partial WHERE purpose ILIKE '%многокв%') - nspd_denorm.py: denorm_parcel_feature / denorm_building_feature / denorm_dump - SAVEPOINT per row via with db.begin_nested() - ST_Transform(ST_SetSRID(ST_GeomFromGeoJSON(:geom), 3857), 4326) - json.dumps(geom, ensure_ascii=False) for psycopg bind - All CAST(:x AS type) — no ❌:type - nspd_sync.py: inline denorm after _upsert_dump in harvest_quarter (non-fatal) - nspd_denorm_backfill.py: Celery one-shot task (per-quarter commit) - admin_etl.py: POST /api/v1/admin/etl/nspd-denorm-backfill (AdminTokenAuth) - 19 tests: coerce helpers, denorm parcel/building, aggregate counts Foundation for downstream features (МКД neighbors lookup, parcel attrs). Part of #94	2026-05-16 20:20:03 +03:00
lekss361	5a7d405c77	Merge pull request 'feat(nspd): TIER 4 opportunity + red lines (#94 PR2 of 4)' (#220) from feat/94-pr2-opportunity-redlines into main	2026-05-16 17:09:52 +00:00
lekss361	572d235d70	Merge pull request 'fix(infra): restore git.gendsgn.ru block in Caddyfile' (#221) from fix/caddyfile-add-git-block into main	2026-05-16 17:02:05 +00:00
lekss361	82931897dd	fix(infra): restore git.gendsgn.ru block in Caddyfile ... Block был добавлен вручную при Forgejo migration (2026-05-16) но не закоммичен в repo. Первый Caddyfile-touching deploy (#205 GlitchTip) сделал git reset --hard origin/main на VPS → блок исчез → TLS handshake internal error 80 на git.gendsgn.ru (prod down ~16:30). Routes git.gendsgn.ru → forgejo:3000 (контейнер из forgejo-migration/ docker-compose.yml, shared gendesign_default network, Forgejo default HTTP port). VPS уже починен hotfix-ом руками; этот PR делает fix permanent — переживёт любой git reset --hard от deploy.yml.	2026-05-16 19:51:54 +03:00
lekss361	fdb54834e5	fix(nspd): rename migration 89→98, fix red lines ST_Area→ST_Length (#220) ... - data/sql/98_*: rename from 89 to avoid collision with existing 89_drop_dead_brin - _get_red_lines: ST_Length(ST_Intersection(planar)::geography) instead of ST_Area(ST_Intersection(::geography, ::geography)) — fixes PostGIS 3.4 tolerance error and returns correct non-zero length for LINESTRING intersections - Rename intersection_area_sqm → intersection_length_m across schema, TS types, frontend component, and tests; add test_get_red_lines_db_exception_returns_empty Addresses review-bot feedback on PR #220.	2026-05-16 19:33:14 +03:00
lekss361	ed3c128528	feat(nspd): TIER 4 opportunity layers + red lines (#94 PR2 of 4) ... - NSPDClient: QUARTER_OPPORTUNITY_LAYERS (auction/scheme/free/future/oopt) + QuarterDump.opportunity field - nspd_sync: harvest_quarter accepts include_opportunity, denorm cols has_auction_parcels + opportunity_count in UPSERT - quarter_dump_lookup: - _get_opportunity_parcels (sort by distance, early-exit on count=0) - _get_red_lines (query existing dump.red_lines core path, layer='red_lines') - SQL 89_*: has_auction_parcels + opportunity_count + partial index - Pydantic: OpportunityParcel + RedLine schemas - /analyze: nspd_opportunity_parcels + nspd_red_lines fields - Frontend: NspdOpportunityBlock + NspdRedLinesBlock + LandTab integration - Tests: 28 total (11 PR1 + 17 PR2), all pass Red lines uses existing core harvest path (layer 879243 already in dump.red_lines from PR1+core) — no duplicate harvest, no red_lines_count_v2 column needed. Part of #94	2026-05-16 19:06:22 +03:00
lekss361	99372a211f	Merge pull request 'fix(recommend): cad_buildings.floors is INTEGER not TEXT (HOTFIX)' (#218) from fix/recommend-mix-floors-integer into main	2026-05-16 16:00:23 +00:00
lekss361	021b0589b2	fix(infra): bootstrap_glitchtip — \gexec instead of DO block (psql var trap) (#219)	2026-05-16 15:54:29 +00:00
lekss361	fbb76812f4	fix(recommend): cad_buildings.floors is INTEGER, not TEXT ... PROD 500 on /api/v1/analytics/recommend/mix: psycopg.errors.UndefinedFunction: operator does not exist: integer ~ unknown LINE 19: AND ((cb.floors ~ '^[0-9]+$' AND cb.f... _district_cadastre_baseline use regex \ on cb.floors. Schema is INTEGER (verified information_schema), so the regex defence against text values ('1-2', '2-3' etc) is misplaced — simplify to floors >= 3 with NULL safety via OR purpose ILIKE '%многокв%' fallback.	2026-05-16 18:29:35 +03:00
lekss361	277499684b	feat(frontend): @sentry/nextjs init для GlitchTip (#204 frontend) (#208)	2026-05-16 15:09:38 +00:00
lekss361	b61f025673	feat(backend): sentry-sdk init для FastAPI + Celery (#204 backend) (#207)	2026-05-16 15:09:16 +00:00
lekss361	d3aa96336b	feat(infra): GlitchTip self-hosted error tracking (#204 devops) (#205)	2026-05-16 15:08:50 +00:00
lekss361	e0055d299c	Merge pull request 'feat(nspd): TIER 3 risk zones (#94 PR1 of 4)' (#217) from feat/94-pr1-risk-zones into main	2026-05-16 15:03:18 +00:00
lekss361	1595157ebf	feat(nspd): enable TIER 3 risk zones harvest + extract in analyze (#94 PR1) ... - harvest_stale_quarters fanout passes include_risks=True to all queued tasks - _get_risk_zones() with ST_Intersects + ST_Area intersection calc - _extract_features_by_layer() generic helper for future PR2+ reuse - RiskZone Pydantic schema; /analyze returns nspd_risk_zones list - NspdRiskZonesBlock component with severity color-coding (red/yellow/orange) - 11 unit tests for extract/intersect/null-area/db-exception paths Part of #94 (TIER 3 risk layers — flooding, landslide, burns, erosion etc)	2026-05-16 16:11:59 +03:00
lekss361	76b1f8673f	test(infra): integration phantom column gate (#197) (#216)	2026-05-16 13:07:13 +00:00
lekss361	ce2c48fff5	feat(site-finder): debounce weight re-analyze (#201 Phase 2) (#215)	2026-05-16 12:54:51 +00:00
lekss361	dbf31cc911	feat(etl): objective_complex_mapping backfill via pg_trgm (#203) (#214)	2026-05-16 12:47:09 +00:00
lekss361	46203ab365	feat(permits): Celery beat monthly + analyze recent_permits (#105 Phase 4+5) (#213)	2026-05-16 12:23:52 +00:00
lekss361	e450546178	feat(site-finder): NSPD frontend integration в LandTab (#202) (#212)	2026-05-16 11:55:31 +00:00
lekss361	adc45f5724	feat(permits): ЕКБ РНС/РВЭ xlsx parser + schema + ETL (#105 PR A) (#211)	2026-05-16 11:41:21 +00:00
lekss361	27a0957bb5	feat(analyze): add market_price из mv_quarter_price_per_m2 (#33 PR B) (#210)	2026-05-16 11:28:16 +00:00
lekss361	efa66a9f68	feat(analytics): mv_quarter_price_per_m2 MV + refresh helper (#33 D1 PR A) (#209)	2026-05-16 11:12:01 +00:00
lekss361	092976f656	feat(site-finder): inline POI weights pass-through в /analyze (#201 Phase 1) (#206)	2026-05-16 11:00:41 +00:00
lekss361	256909d28b	feat(layouts): PDF ТЗ endpoint + BestLayoutsBlock UI (#113 PR D) (#199)	2026-05-16 09:53:05 +00:00
lekss361	ad16fc0e42	feat(parcels): best-layouts endpoint + service (#113 PR C) (#196)	2026-05-16 09:10:06 +00:00
lekss361	6aa22ef8ba	feat(analytics): mv_layout_velocity MV + refresh helper (#113 PR B) (#194)	2026-05-16 08:26:59 +00:00
lekss361	5455d57bed	chore(ci): run CI on push to feature branches (#195)	2026-05-16 08:26:04 +00:00
lekss361	e673811a20	feat(parcels): layout signature schemas + extractor (#113 PR A) (#193)	2026-05-16 07:52:34 +00:00
lekss361	4deb1b0284	feat(parcels): connection-points endpoint (Forgejo #115 Phase 1 backend) (#190)	2026-05-16 05:58:27 +00:00
lekss361	e97f96afa7	fix(ci): switch deploy to .forgejo/workflows + shell GHCR login (#192)	2026-05-16 05:25:00 +00:00
lekss361	e561df1b55	feat(parcels): competitors endpoint (Forgejo #112 Phase 1 backend) (#191)	2026-05-15 22:21:05 +00:00
lekss361	50547dcbb3	test(deploy): validate Forgejo Actions end-to-end flow (#189)	2026-05-15 21:55:04 +00:00
lekss361	ae0fce3528	refactor(claude): split 700-line CLAUDE.md into path-scoped .claude/rules/ (#188) ... Old CLAUDE.md was 700 lines — exceeded Anthropic 200-line recommendation, adherence degraded under context pressure. Refactor per research findings: - CLAUDE.md 700 -> 97 lines (essential only, pointers to rules) - .claude/rules/{backend,frontend,sql,git-pr,deploy}.md (NEW, path-scoped) - ~/.claude/CLAUDE.md (NEW, 21 lines) — personal cross-project prefs - memory/ — deleted 12 feedback files promoted to proper layer (CLAUDE.md rules table, .claude/rules/, ~/.claude/CLAUDE.md). 7 remaining = workflow details. - .gitignore: !.claude/agents/, !.claude/rules/ exceptions Total auto-loaded per session: 97 (CLAUDE.md) + 21 (~/.claude) = 118 lines vs prior 700. Rules in .claude/rules/ load only when matching paths touched. Resolves conflict no_self_review vs pre_push_review: now single rule in git-pr.md Review workflow section (pre-push = local lint; post-push = external Claude window posting as lekss361). Code-reviewer feedback applied: - paths: frontmatter added to git-pr.md - Auto-merge scope blocked-list now includes .claude/rules/*.md changes to prevent self-extending rules without human approval. Co-authored-by: lekss361 <claudestars@proton.me>	2026-05-15 23:06:33 +03:00
lekss361	4d492ebc50	perf(cadastre): freshness skip + grid-walk heartbeat + log noise reduction (#187) ... * perf(cadastre): freshness skip + grid-walk heartbeat + reduce WMS log noise Three operational improvements after ekb_full v3 completed at 99.1%: Fix H — per-quarter freshness skip (scrape_cadastre.py): - enqueue_cadastre_harvest queries cad_quarter_stats.fetched_at per quarter - Skip quarters where fetched_at >= NOW() - skip_fresh_hours (default 1h) - extra_config.skip_fresh_hours=0 disables skip (full re-fetch) - Saves ~30k NSPD calls on repeated runs of same scope Fix I — heartbeat in grid-walk loop (bulk_harvest.py:_grid_walk_category): - 225 cells x 70-300ms = 15-75s heartbeat silence - Pass update_progress callback to _grid_walk_category - Call every heartbeat_every (50) cells with phase progress info - Prevents cleanup_zombies false-positive cancellation Fix J — NSPD WMS HTTP 500 noise to debug (bulk_harvest.py:_grid_walk_category): - NSPD WMS endpoint frequently returns HTTP 500 (server-side issue) - Was: logger.warning each, 200+ warnings per quarter - Now: logger.debug (visible via --log-level=DEBUG) 40 tests passing. * fix(cadastre): parse NSPD DD.MM.YYYY dates (Fix K — job 18 root cause) Job 18 (manual_list recovery 11 quarters) crashed every quarter with: psycopg.errors.DatetimeFieldOverflow: date/time field value out of range: '13.03.2023' NSPD returns registration_date in mixed formats: - DD.MM.YYYY for Сооружения / ОНС / ЕНК / ЗОУИТ - YYYY-MM-DD ISO for ЗУ / Здания - YYYY-MM-DD HH:MM:SS with time PG CAST(:date AS date) accepts ISO only — DD.MM.YYYY crashes. Add _parse_nspd_date helper: handles both formats + None/empty. Apply in upsert_construction, upsert_onc, upsert_enk, upsert_zouit, upsert_parcel for registration_date / build_record_registration_date / legal_act_date. 3 new unit tests for the helper (43 tests total). * fix(cadastre): address bot blockers — pivot Fix H to raw_targets + tests Bot review on PR #187 caught: 1. cadastre_jobs has no extra_config column → Fix H crashed on KeyError 2. Missing tests for Fix H + Fix I 3. _parse_nspd_date didn't validate date ranges Changes: - Pivot Fix H: read skip_fresh_hours from raw_targets JSONB (existing column), no schema migration needed - _parse_nspd_date: validate via date.fromisoformat() — rejects 2023-13-45 - Add 3 tests for Fix H in backend/tests/workers/test_scrape_cadastre.py - Add 2 tests for Fix I heartbeat callbacks in test_cadastre_bulk.py - Add 1 test for parse_nspd_date range validation (was: only format) 49 tests passing. --------- Co-authored-by: lekss361 <claudestars@proton.me>	2026-05-15 23:02:37 +03:00
lekss361	d104222643	fix(cadastre,nspd_sync): SQL syntax bugs surfaced by ekb_full v2 (#186) ... Three distinct SQL syntax errors caught from worker logs: 1. **upsert_zouit** (bulk_harvest.py:940) — trailing comma after 4326 in ST_Transform(..., 3857), 4326,) → 'syntax error at or near ")"'. Hit on 8+ quarters in 0108 block (66:41:0108055/61/69/77 etc). Likely introduced by ruff-format auto-comma on closing paren. 2. **nspd_sync** (nspd_sync.py:120) — `:fetched_at_utc::timestamptz` psycopg double-colon trap → 'syntax error at or near ":"'. Crashed legacy on-demand harvest_quarter called from analyze. 3. **job_settings** (job_settings.py:224) — `:extra_config::jsonb` same double-colon trap in admin settings UPDATE. All converted to CAST(:x AS type) pattern (psycopg v3 safe). 40 tests still passing. Co-authored-by: lekss361 <claudestars@proton.me>	2026-05-15 19:18:43 +03:00
lekss361	5f8df655cc	fix(cadastre): cad_parcels.geom Polygon -> MultiPolygon (migration 93) (#185) ... NSPD returns MultiPolygon for Многоконтурный участок (e.g. 66:41:0105017:4) which crashed upsert_parcel — schema was strict POLYGON. Migration 93: ALTER COLUMN geom Polygon -> MultiPolygon USING ST_Multi(geom), DROP+RECREATE GIST index + back-compat VIEW cad_parcels_geom. upsert_parcel SQL wraps ST_Transform in ST_Multi() — coerces Polygon to MultiPolygon. NSPD returns both types (simple + multi-contour parcels). Co-authored-by: lekss361 <claudestars@proton.me>	2026-05-15 18:52:38 +03:00