lekss361
25b73035a1
sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases
...
- NSPD-skraper переехал в backend/app/services/scrapers/nspd_kn.py +
Celery task scrape_nspd_region (beat: 20-е февраля/мая/авг/нояб).
Redis lock 3h, WAF auto-retry, heartbeat в nspd_scrape_runs.
- Recommend_mix Tier 3: per-bucket elasticity через регрессию по
«доминирующему bucket» каждого ЖК. Weighted-elasticity для inverse-mode.
UI показывает разброс эластичностей и переключение regression/fallback.
- Cadastre vs market cross-check: spatial-join cad_buildings →
ekb_districts_geom; cadastre_vs_market_pct в scope, аномалии
(>+50% / <-30%) подсвечены в UI.
- Sentry release tracking (#4 ): IMAGE_TAG → backend/.env.runtime →
sentry_sdk.init(release=...). Compose v2 env_file optional path.
Schemas: 63_schema_nspd_runs.sql (cad_buildings + nspd_scrape_runs/log
формализуют то, что уже жило в проде через 61_import_nspd_batch.py),
64_v_zk_rosreestr_velocity.sql (refresh с cad_buildings).
2026-04-30 21:51:19 +03:00
lekss361
b3c15bb8ea
fix ui lag
2026-04-27 20:26:55 +03:00
lekss361
5056438fcf
add anal4
2026-04-27 19:45:47 +03:00
lekss361
79070bfe99
add interactive parser of наш.дом.рф ЖК+flats with stealth Playwright
...
- Discovered real endpoints via chrome-devtools: /сервисы/api/kn/object
(offset/limit/place/objStatus) + /portal-kn/api/sales/portal/table.
Server-side filter ignored — local filter by developer.companyGroup.
- Pure Playwright in-context fetch (httpx blocked by ServicePipe TLS-fp).
- Saved fingerprint state in data/playwright_state.json for server reuse.
- Celery beat (configurable cron + jitter) + admin trigger endpoint
+ /admin/scrape UI page.
- Schema migration: UNIQUE(id, snapshot_date) for versioned snapshots,
kn_scrape_runs journal.
- Smoke-tested on PRINZIP: 28 ЖК / 826 квартир as expected.
2026-04-27 17:58:40 +03:00
lekss361
8d3a0874ef
add interactive analytics dashboard for Sverdlovsk market and PRINZIP
...
3 pages (market, PRINZIP drilldown, developers leaderboard) on top of
existing v_developer_full_metrics + domrf_realization views. ECharts on
the frontend, FastAPI router /api/v1/analytics on the backend.
2026-04-27 16:55:30 +03:00
lekss361
d0909e6200
backend: ruff lint fixes
...
- main.py: AsyncIterator from collections.abc (typing.AsyncIterator deprecated, UP035)
- geometry.py: TEAP first in import (isort: ALL_CAPS before MixedCase, I001)
- pyproject.toml: ignore RUF001/002/003 — Russian-language project legitimately
uses Cyrillic lookalikes in comments/docstrings (МСК-66 etc).
Verified: ruff check . — All checks passed!
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 10:53:50 +03:00
lekss361
b95559eac9
init
2026-04-25 13:45:19 +03:00
