lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/tradein-mvp/backend/app/services
History
bot-backend 43b392b809
All checks were successful
CI / changes (pull_request) Successful in 7s
Details
CI / backend-tests (pull_request) Has been skipped
Details
CI / frontend-tests (pull_request) Has been skipped
Details
CI / openapi-codegen-check (pull_request) Has been skipped
Details
feat(tradein/domclick): local-runner + ingest tooling → main (split from parked Layer-B) 
Выделяет МЕРДЖАБЕЛЬНУЮ часть DomClick-тулинга из запаркованного PR #2005
(BLOCKED #2000 — прод Layer-B скрейпер через прокси режет QRATOR-403):

- scripts/domclick_local_runner.py — ручной offline-сборщик с домашнего IP:
  SERP-enumerate (фронт, BFF-JSON режет QRATOR) + card __SSR_STATE__ +
  offer-card v3 (AVM price_prediction + sold_similar). --enum-cache (enumerate
  1 раз → файл с sentinel → следующий запуск грузит кэш и пропускает enumerate),
  wait_for_selector (SERP-карточки догидрируются), resume по id. Self-contained.
- scripts/ingest_domclick_jsonl.py — JSONL → trade-in БД (save_listings +
  house-match в SAVEPOINT + save_detail_enrichment, идемпотентно, --limit/--dry-run).
- app/services/scrapers/domclick_detail.py + domclick_exceptions.py — card-SSR
  parse + save_detail_enrichment (нужны ингесту). Импортятся ТОЛЬКО ингестом
  (operator-run); прод-runtime их не дёргает (dormant в образе).
- tests/scrapers/test_domclick_detail.py + обновлённый scripts/README.md.

НЕ включает прод Layer-B (tasks/domclick_detail_backfill.py, scheduler-расписание,
data/sql/139_*) — остаётся запаркован на PR #2005 до чистого прокси (#2000).
БД-схема не меняется (detail-колонки уже в проде, миграции нет).

Live-validated: раннер качает ЕКБ-вторичку прямо сейчас (600+ карт, 0 блоков),
ингест залил их в прод-БД (house_matched, идемпотентно).
2026-06-28 10:26:52 +03:00
..
exporters fix(tradein): trade-in PDF cover uses estimate.expires_at, not hardcoded +30d (#1530) 2026-06-17 20:27:57 +03:00
matching fix(matching): conflict_resolution picks freshest (last_seen_at) value, deterministic (#1539) 2026-06-17 22:43:59 +03:00
scrapers feat(tradein/domclick): local-runner + ingest tooling → main (split from parked Layer-B) 2026-06-28 10:26:52 +03:00
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
account_quota.py fix(tradein): atomic conditional quota increment — close TOCTOU (#747) (#749) 2026-05-30 15:52:34 +00:00
brand.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
cache.py feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) 2026-05-23 14:44:22 +00:00
cian_price_history.py fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1») 2026-06-15 20:21:11 +05:00
cian_session.py feat(tradein): dedicated Cian mobile proxy — separate egress from Avito (#923) 2026-05-31 19:30:14 +00:00
dadata.py fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1») 2026-06-15 20:21:11 +05:00
estimator.py feat(tradein/estimator): elite-segment manual_review trigger by asking ppm² (#2002) 2026-06-27 20:15:38 +00:00
geocoder.py fix(tradein): drop non-EKB avito detail coords on ingest (audit #1871 P2) 2026-06-23 14:33:10 +05:00
house_dedup_merge.py feat(tradein): recurring house-dedup merge (schedule dormant) (#1772) (#1933) 2026-06-26 21:39:30 +00:00
house_imv_backfill.py feat(tradein-imv): route IMV backfill через BrowserFetcher.fetch_json за dormant-флагом (#915 Stage 3) (#1916) 2026-06-26 09:54:28 +00:00
house_metadata.py feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) 2026-05-24 08:57:30 +00:00
image_sanitizer.py fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1») 2026-06-15 20:21:11 +05:00
rosreestr_poll.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
sber_index.py fix(tradein): correct SberIndex hedonic/asking dataset-paths + 404 warning (#902) 2026-06-17 22:50:52 +03:00
scheduler.py feat(tradein): recurring house-dedup merge (schedule dormant) (#1772) (#1933) 2026-06-26 21:39:30 +00:00
scrape_pipeline.py fix(tradein/domclick): rewrite dead scraper to BFF JSON API + honest status (#1846, #1968) 2026-06-27 13:57:07 +03:00
scrape_runs.py fix(tradein): persist scrape_runs.total_seen/new_count from sweep counters (#1926) (#1928) 2026-06-26 18:09:17 +00:00
scraper_settings.py chore(tradein): деактивировать N1.ru scraper — источник переехал в Cian 2026-06-16 14:50:51 +03:00
search_query.py feat(tradein): segment=vtorichka|novostroyki|all query-param on listings (#1188) 2026-06-17 21:43:01 +03:00
yandex_address_backfill.py fix(tradein): yandex address-enrich — city-agnostic title regex (спутники ЕКБ) (#875) 2026-05-31 09:08:29 +00:00
yandex_price_history.py feat(tradein): populate offer_price_history for yandex from gate price.previous/trend 2026-06-18 14:23:23 +03:00