1719 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Light1YT	8a3bae0cda	feat(analyze): per-category OSRM routing (foot vs driving) (#39 A3) ... Walk-relevant POIs (school/shop/park/kindergarten/pharmacy/stops) now route via a FOOT OSRM graph (osrm-walk service), car-relevant POIs (mall/hospital + unknown) keep the DRIVING graph. Validation showed driving overstated pedestrian-proximity distance — median 1.6-2.9x straight-line (#39). - config: osrm_walk_local_url + osrm_walk_categories (frozenset, 9 walk cats) - osrm_client_local: base_url override on get_road_distances_m (default unchanged) - _apply_osrm_road_distances: split POIs by category, per-group OSRM call with independent graceful fallback (one server down -> its group keeps straight-line), in-place write-back by original index; never raises; flag-OFF byte-identical - docker-compose: osrm-walk service (foot graph, internal, mem_limit 1.5g) - build_osrm.sh: CAR=0 gate for foot-only refresh (doesn't touch live car graph) - tests: per-category split, per-group fallback, asymmetric intra-group write-back Still flag-gated (use_osrm_distances OFF) — enabling is a product decision. Refs #39	2026-06-27 04:30:25 +05:00
bot-backend	6c8bf0e496	Merge pull request 'feat(site-finder): OSM инж.сети на карту analyze (#1746)' (#1934) from feat/1746-utility-infra-map into main	2026-06-26 22:43:59 +00:00
Light1YT	854fa69a8b	feat(site-finder): render OSM utility-infrastructure layer on analyze map (#1746) ... Section2 "Сети" map now shows the OSM engineering-networks layer (osm_utility_infrastructure_ekb, endpoint /parcels/{cad}/utility-infrastructure) alongside the NSPD connection-points + ЗОУИТ layers from #1751, on the same map. - useUtilityInfrastructure hook (react-query, mirrors useConnectionPoints) - UtilityInfrastructureLayer: Point→CircleMarker, LineString/Polygon(+Multi)→GeoJSON, per-kind colour (power/water/gas/heat/communication/sewage), popups - UtilityLayerControlPanel: collapsible legend, per-kind toggle + count - SiteMap: optional utilityInfrastructure prop + visibleKinds state (CP/ЗОУИТ unchanged) - 12 unit tests (geometry parsing + [lon,lat]→[lat,lon] swap + grouping) Refs #1746	2026-06-27 03:40:51 +05:00
bot-backend	63ac12dc23	feat(tradein): recurring house-dedup merge (schedule dormant) (#1772) (#1933)	2026-06-26 21:39:30 +00:00
bot-backend	f48db87d39	feat(site-finder): OSRM road-distance in /analyze behind flag (#39 A2) (#1932)	2026-06-26 21:25:50 +00:00
bot-backend	7a1d2a4ed6	fix(site-finder): Overpass retry-backoff + mirror fallback for utility loader (#1746) (#1931)	2026-06-26 21:13:06 +00:00
bot-backend	3753079dee	feat(site-finder): OSM engineering-networks loader + endpoint (#1746) (#1930)	2026-06-26 20:01:00 +00:00
bot-backend	4fe20a9cbd	feat(devops): self-hosted OSRM routing engine for site-finder (#39) (#1929)	2026-06-26 19:18:21 +00:00
bot-backend	4846197d54	fix(site-finder): align competitor flats_sold to objective_lots.contract_date (#1926) (#1927)	2026-06-26 18:33:51 +00:00
bot-backend	89d05bbfec	fix(tradein): persist scrape_runs.total_seen/new_count from sweep counters (#1926) (#1928)	2026-06-26 18:09:17 +00:00
bot-backend	87364eebca	fix(test): test_849 expects 3 begin_nested/lot after #1921 upsert-reconcile SAVEPOINT — unblock red main gate (#1925)	2026-06-26 17:23:24 +00:00
bot-backend	d89210200b	perf(tradein): GIST index on (geom::geography) for estimator radius filter (#1189) (#1924)	2026-06-26 17:12:49 +00:00
lekss361	d111573a4b	Merge pull request 'fix(trade-in/ui): расшифровка ДКП/IMV/CV + графики на --viz токены + убрать emoji (#1871)' (#1922) from fix/tradein-redesign-pack into main ... Reviewed-on: #1922	2026-06-26 16:19:23 +00:00
lekss361	74ea6659ea	Merge pull request 'fix(scrapers): avito_full_load краш (source_id-reconcile) + cian/yandex ban-ротация' (#1921) from fix/scraper-resilience-upsert-rotation into main ... Reviewed-on: #1921	2026-06-26 16:19:06 +00:00
lekss361	5d5b2981fc	Merge pull request 'fix(estimator): флаг + тесты для ghost-anchor guard и radius-dedup (#1871)' (#1919) from fix/estimator-trust-ghost-dedup into main ... Reviewed-on: #1919	2026-06-26 16:18:58 +00:00
bot-backend	27766acf38	fix(trade-in/ui): расшифровка ДКП/IMV/CV + графики на --viz токены + убрать emoji из вердиктов (#1871) ... - HeroSummary: все вхождения ДКП/IMV/CV обёрнуты в <abbr title="..."> с расшифровкой - IMVBenchmark: «Avito IMV» → abbr с title - HeroSummary/HeroTransparency: emoji ⚠️ заменены на span-точки var(--warn) + текст - PriceHistoryChart: stroke="#2563eb"/"#ea580c" → var(--viz-1)/var(--viz-4) - PriceRangeBar: "#bfdbfe"/"#1d4ed8" → color-mix(--viz-1)/var(--viz-1) - SellTimeSensitivity: hardcoded bg-цвета → --success-soft/--accent-soft/--warn-soft/--danger-soft; border → var(--viz-1) - MapCard: пины обновлены на hex-значения --viz-1/--viz-3/--viz-4 (Leaflet JS, не CSS-контекст) - MapPicker: комментарий -- viz-1 hex (Leaflet constraint)	2026-06-26 19:16:55 +03:00
bot-backend	c1cdf3969c	fix(scrapers): ban/stuck-ротация для cian+yandex (provider-aware _rotate_proxy_ip)	2026-06-26 19:10:26 +03:00
bot-backend	197068c515	fix(scrapers): save_listings — UPDATE by (source,source_id) при dedup_hash-дрейфе (avito_full_load краш)	2026-06-26 19:10:12 +03:00
bot-backend	2fe34ac785	fix(estimator): ghost-anchor — confidence='low' when radius analogs empty (#1871) (#1920)	2026-06-26 16:07:12 +00:00
bot-backend	9678aface5	fix(estimator): dedup radius-аналогов — не раздувать n_analogs (#1871) ... Добавляет тест test_n_analogs_reflects_deduped_count: 3 дубля одного объекта (один source_id) → 1 кандидат (freshest scraped_at); 3 разных объекта → все 3. Верифицирует ключевое требование #1871 P2: n_analogs отражает дедупнутое число, а не raw кол-во строк в listings с дублями (prod: 797 yandex + 93 cian дублей раздували n_analogs и искажали медиану/перцентили).	2026-06-26 18:52:11 +03:00
bot-backend	68f7c0e015	fix(estimator): запрет confidence=high при n_analogs=0 (#1871 ghost-anchor) ... Добавляет settings-флаг estimate_confidence_floor_no_analogs (дефолт True), гейтящий вызов _enforce_zero_analog_low перед сборкой AggregatedEstimate. При n_analogs==0 и confidence!='low' форсит 'low' + добавляет caveat в explanation («без сопоставимых аналогов рядом»), предотвращая показ выдуманного «высокого» доверия когда оценка построена только на внешних оценщиках (yandex_valuation/cian_valuation) без реальных рыночных аналогов.	2026-06-26 18:50:19 +03:00
bot-backend	e03c810306	feat(tradein-imv): route IMV backfill через BrowserFetcher.fetch_json за dormant-флагом (#915 Stage 3) (#1916)	2026-06-26 09:54:28 +00:00
bot-backend	d479edeb3f	feat(tradein-browser): additive POST /fetch-json + BrowserFetcher.fetch_json (#915 Stage 2) (#1915)	2026-06-26 09:30:35 +00:00
bot-backend	e0cd378d19	fix(migration-133): dedup via cascade-delete, not child re-point (prod hotfix) ... Migration 133 failed on prod (offer_price_history UNIQUE (listing_id,change_time) + multi-loser collisions). Drop data-child re-points, rely on FK ON DELETE CASCADE; keep only merged_into. Dry-run on prod: 9840 groups, 9889 losers deleted, clean. Refs #1773	2026-06-26 08:16:27 +00:00
bot-backend	1d5b081cb4	fix(cian): recover new-build house via cadastral anchor + listings (source,source_id) dedup (#1773) (#1912) ... cian new-build cadastral anchor + listings (source,source_id) dedup (#1773): migration 133 (deep-reviewed data-loss-safe). Refs #1773	2026-06-26 07:45:22 +00:00
bot-backend	3f1925052b	feat(scheduler): wire house_imv_backfill as scheduled source (#854) (#1910) ... wire house_imv_backfill as scheduled source (#854): trigger fn + seed migration 132. Refs #854	2026-06-26 07:45:08 +00:00
bot-backend	f091554942	fix(scrapers): named BrowserSession concurrency const, fix Semaphore(3) doc drift (#1347) (#1913) ... named BrowserSession concurrency const + fix Semaphore(3) doc drift (#1347): keeps 8 (issue-sanctioned doc option). Refs #1347	2026-06-26 07:44:29 +00:00
bot-backend	9c52e0b29f	feat(etl): housing-class normalization fallback via yandex_realty trigram match (#38) (#1911) ... housing-class normalization fallback via yandex_realty trigram (#38): migration 169 + ETL + COALESCE consumers. Refs #38	2026-06-26 07:44:25 +00:00
bot-backend	5ed88c41a4	fix(avito): post-merge polish — unify fetch_mode guard + empty-render guard (#1811) (#1909) ... avito SERP browser post-merge polish (#1811): unify fetch_mode guard, empty-render guard, debug logging. Refs #1811	2026-06-26 07:44:22 +00:00
bot-backend	67d41df880	fix(freshness): repoint nspd source to live nspd_quarter_dumps (data-table mode) ... nspd freshness watched a defunct ledger (nspd_scrape_runs, WAF-banned manual April runs) → false "no successful runs". Add data-table mode (timestamp_col) + repoint nspd to live nspd_quarter_dumps.fetched_at_utc (fresh<14/stale<30). Run-ledger sources untouched. Refs #73	2026-06-26 06:51:00 +00:00
bot-backend	2304e09e37	fix(freshness): calibrate kn staleness thresholds to weekly cadence ... kn (DOM.РФ ЖК) runs weekly (Mon cron) but thresholds were 2/5 → false "stale" every Wed-Sun. Recalibrate to 8/14 (weekly, mirrors objective) + regression test. Surfaced by the freshness monitor's first prod run. Refs #73	2026-06-26 06:33:29 +00:00
bot-backend	66da75681f	feat(financial/dcf): dormant separate-index inflation capability (default OFF) ... smr/sale_inflation_annual in _build_monthly_cashflow, gated by module constants =0.0 → byte-identical zero-diff (deep-review verified IEEE-754 identity + all 57 prior tests unchanged). Land not escalated; marketing/VAT/tax retimed on escalated revenue (total preserved). Capability wired+tested, OFF until calibrated indices + invariant reframe. No schema/codegen. Refs #1881	2026-06-25 12:12:02 +00:00
bot-backend	b82963d12e	feat(financial): ground-floor нежилое (office/commercial) tranche ... Carve a documented office share (comfort/business 5%, econom 0) out of GFA before residential — fixes prior overstatement (all non-parking GFA = residential). total_floor_area unchanged → no double-count; office is additive revenue (+15% premium), нежилое → VAT-able (vat base = parking_va + office_va). Σ invariant holds. Additive schema + api-types regen + PDF/concept display. Deep-review ✅. Refs #1881	2026-06-25 11:40:01 +00:00
bot-backend	55811ff7cc	feat(workers): data-freshness monitor (Celery beat → GlitchTip) ... Extract compute_freshness(db) from the /admin/scrape/freshness handler (endpoint unchanged) + daily beat task scrape_freshness_check that alerts via sentry_sdk.capture_message when a source is stale/failed (error on critical-failed, else warning). Registered in celery include + beat (09:00 MSK). Refs #73	2026-06-25 11:29:34 +00:00
bot-backend	f249212695	fix(site-finder): isolate zone-regulation cache write off the request session (#1850 item 4) ... Cache-miss upsert now runs in a fresh SessionLocal() instead of committing the shared /analyze transaction mid-request. Read-back via shared session unchanged. Refs #1850	2026-06-25 11:16:26 +00:00
bot-backend	39f099bc17	chore(mocks): regenerate parcel-analyze financial_estimate (consistent with current backend) ... Regenerate the drifted dev fixture financial_estimate via compute_financial() from its own teap_synth inputs — parking comfort-priced (#1893), VAT parking-only (#1898), levered_irr refreshed (#1900). Dev-only. Refs #1881	2026-06-25 11:02:36 +00:00
bot-backend	d2773e0c91	fix(site-finder): optimize zone-regulation resolve in /analyze (#1850) ... Memoize coord→zone_index WFS lookup (thread-safe bounded LRU, negative caching), resolve once and thread into IRD block (no double resolve when both flags on), unify timeout into settings.geoportal_timeout_s. Lock guards the threadpool-shared memo (WFS call outside lock). Documented the cache-miss db.commit() smell (separate task). Deep-review ✅. Closes #1850	2026-06-25 09:12:41 +00:00
bot-backend	d589ddf6c4	feat(financial): levered (equity) IRR — capstone финмодели #1881 ... Levered (equity) IRR from the LTC debt-schedule equity cashflow, mirroring the unlevered IRR + roi-proxy fallback. Honest negative-leverage (can be below project IRR). Σ equity_cf == net_profit_after_financing invariant holds. Unlevered untouched. Additive schema fields (api-types regenerated). Cockpit drawer + concept display. Deep-review ✅. Refs #1881	2026-06-25 09:06:46 +00:00
bot-backend	dc659da655	feat(site-finder): geo-radius market-price calibration in /analyze (#1881 follow-up) ... New price tier objective_geo_radius: ST_DWithin median of Objective new-build prices (objective_lots ⋈ complexes) within 3km of parcel centroid, between quarter-MV and district_reference. Closes the name-match gap (5 of 9 EKB districts had no Objective name-match). data/sql/168 functional GIST index (prod EXPLAIN 114ms→28ms). Degenerate-centroid guard + honest RU price_source captions. Deep-review ✅. Refs #1881	2026-06-25 07:59:56 +00:00
bot-backend	3073651812	fix(generative/financial): VAT only on parking, residential ДДУ exempt (ст. 149 НК) ... VAT charged on full gross margin → only on parking value-added; residential (реализация жилья + услуги застройщика по ДДУ) exempt per пп.22/23.1 п.3 ст. 149 НК; input VAT embedded in gross СМР. Σ cashflow == net_profit invariant preserved. PDF + concept-UI caveats synced. +3 tests, deep-review ✅. Refs #1881	2026-06-25 07:22:58 +00:00
bot-backend	89509824c2	chore(agents): bump all worker + analyst agents to opus 4.8 ... All worker + analyst project agents → model: opus (Opus 4.8); reviewers already opus. Guard files (_autonomous_pickup, auto-code-reviewer) untouched. Per Anton request.	2026-06-25 06:54:14 +00:00
bot-backend	4eaf184d0e	fix(avito): graceful sidecar HTTP 500/503 in browser SERP fetch (#1790) ... Classify tradein-browser sidecar HTTP 500/503 in _fetch_serp_html_browser and route into existing rotation/retry machinery: soft-ban → IP rotation, transient → bounded backoff, both exhausted → AvitoRateLimitedError (graceful mark_banned, partial preserved) instead of mark_failed. Fixes run-#356 class where one late 500 nuked a run that had saved 3186 listings. Refs #1790	2026-06-25 06:47:41 +00:00
bot-backend	4bb09911b8	feat(financial): LTC 70%/equity 30% норматив МКД РФ в модели финансирования ... Заменяет 100%-кредитное покрытие разрыва реалистичной LTC-моделью (банк 70%, собственные средства 30%). total_interest снижается ~30%, net_profit_after_financing растёт. Backend 41/41, Frontend 20/20.	2026-06-24 16:49:01 +00:00
bot-backend	5a73f1d0b6	feat(cockpit): поверхностить financial_estimate в Development Scan / Investment Clearance (#1881 rank-5) (#1894) ... feat(cockpit): поверхностить financial_estimate в Development Scan / Investment Clearance (#1881 rank-5)	2026-06-24 16:20:19 +00:00
bot-backend	c13ab32aa2	Merge pull request 'fix(financial): парковка price/cost по классу — econom наземная, business=Excel (#1881)' (#1893) from fix/parking-economics-by-class into main	2026-06-24 12:21:23 +00:00
Light1YT	4eeda46cfa	fix(financial): парковка price/cost по классу (econom наземная, business Excel) ... Финмодель брала ПЛОСКИЕ parking-константы для ВСЕХ классов: price 1.9М/cost 1.8М (Excel «Авангардная 13» premium — подземная, near break-even). Но норма парковки уже class-dependent (econom 0.8/comfort 1.0/business 1.5 м/м на квартиру). Econom (дешёвое жильё) заряжался 1.8М/м/м ПОДЗЕМНОЙ застройки → огромный break-even капитал, раздувающий пиковый долг/проценты. По РФ econom = наземная дешёвая парковка. Фикс: _PARKING_PRICE_PER_SPOT / _PARKING_COST_PER_SPOT → dict[HousingClass]: - econom 800k/350k (наземная), comfort 1.3М/1.0М (частично подземная), business 1.9М/1.8М (подземная, = Excel, unchanged). - маржа/м/м: econom +450k, comfort +300k, business +100k. - НОРМА парковки (_PARKING_PER_APARTMENT) НЕ тронута (возможно ПЗЗ-норматив). - Константы помечены как TUNABLE ASSUMPTIONS (РФ-рынок), правятся в одном месте (паттерн _SALE_PRICE_PER_SQM / _CONSTRUCTION_COST_PER_SQM). ВАЖНО (честно): парковка — ВТОРИЧНЫЙ драйвер. Улучшает econom в основном через снижение peak-debt/процентов, НЕ переворачивает знак. ДОМИНИРУЮЩИЙ рычаг — себестоимость СМР жилья к выручке (econom 72k на GFA vs выручка на sellable = 87%) — отдельное продуктовое решение (не трогаю без согласования). Инвариант Σ cashflow == net_profit сохранён (меняются только величины revenue_parking/construction_parking в том же каскаде). Тесты: +6 (по классам, маржа, инвариант econom+parking) + 3 существующих исправлены (хардкод comfort 1.9/1.8 → 1.3/1.0). tests/services/generative 90 passed. ruff+mypy(strict) чисто. Схема не менялась. Хозяйственное: untrack + gitignore backend/.coverage (артефакт блокировал checkout). Refs #1881 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-24 16:58:21 +05:00
bot-backend	a45dfd4ee3	Merge pull request 'fix(gate): area-gate ЗОУИТ utility-easement blocker (sub-17) — снять over-block can_build (#1881)' (#1892) from fix/zouit-engineering-area-gate into main	2026-06-24 10:09:08 +00:00
Light1YT	bc1da8eac9	fix(gate): area-gate ЗОУИТ utility-easement blocker (sub-17) — снять over-block ... ZOUIT_OVERLAP_SUB17 (единственный subcategory-blocker = охранная зона ЛЭП/газа/трубопровода) был ХАРД-блокером на ЛЮБОЕ пересечение → блокировал 434/574 (75.6%) участков как «нельзя строить МКД». Прод-замер: реальное покрытие участка этими зонами median=6%, p90=47%, только 1/17 >60% — тонкие инж-коридоры. По РФ охранная зона ограничивает застройку ВНУТРИ полосы, а не весь участок (МКД сажается на незанятой части). Хард-блок на 6%-покрытии неверен и (после фикса резолва зон #1891) прятал financial_estimate у ~90% участков. Фикс: area-gate. - quarter_dump_lookup._get_zouit_overlaps + _get_cad_zouit_overlaps: добавлен coverage_pct (доля площади участка под зоной, в geography м², NULLIF-guard). - gate_verdict.compute_gate_verdict: sub-17 (и cad network/blocker) АГРЕГИРУЮТСЯ (sum-capped, консервативно к блоку) → блокер ТОЛЬКО если покрытие > settings.gate_zouit_engineering_blocker_min_coverage (0.6, env-tunable), иначе WARNING ZOUIT_ENGINEERING_PARTIAL с % покрытия. Больше нет 4× дубль-блокеров. СЗЗ/OKN/прочие ЗОУИТ — без изменений (warnings). main_vri/резидентность — без изменений. - coverage_pct отсутствует (legacy/synthetic) → трактуем как 0 → warning (документировано: дамп всегда даёт coverage; ложный хард-блок дороже). Тесты: +помесь gate/aggregation/coverage (131 combined passed); правка _make_zouit_row (7-я колонка coverage_pct) в tests/test_quarter_dump_lookup.py. Полный бэкенд локально: 2 failed → fixed → перепроверка. ruff чисто. Схема не менялась (gate_verdict/overlaps — dict[str,Any], api-types regen не нужен). Refs #1881 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-24 14:45:59 +05:00
bot-backend	41d97aeaf9	Merge pull request 'fix(site-finder): разгейтить geoportal зон-резолвер от gappy NSPD-дампа — починить видимость финмодели (#1881)' (#1891) from fix/zoning-resolution-decouple into main	2026-06-23 23:55:52 +00:00
Light1YT	2e53f0a1c4	test(analyze): авто-стаб geoportal зон-резолвера в api/v1 тестах (CI desync fix) ... PR-A сделал get_or_fetch_zone_regulation БЕЗУСЛОВНЫМ в analyze. Analyze-тесты с позиционным DB-моком (_make_db_for_analyze + call_idx/responses[]) не мокали резолвер → новый безусловный вызов делает db.execute (get_cached) и сдвигает последовательность ответов мока → parcel_meta получает чужой ответ = None. Локально проходило (geoportal недоступен с dev-машины → резолвер возвращал None, без db-вызова), в CI падало (раннер на проде ДОСТАЁТ geoportal → зона резолвится → get_cached → desync): FAILED test_parcel_meta_found_in_cad_parcels. Fix: autouse-фикстура в tests/api/v1/conftest.py глушит резолвер в no-op (return None) для всех v1-тестов — блок 9e-bis ничего не синтезирует и не трогает БД (идентично до-PR-A). test_analyze_zoning_regulation.py переопределяет резолвер per-test (patch поверх autouse), поэтому его проверки сохранены. Verify: затронутые analyze-файлы + zoning-файл = 72 passed, 0 failed. ruff чисто. Refs #1891 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-24 04:33:17 +05:00