gendesign/tradein-mvp/backend/app/services/scrapers
bot-backend 03d7884a80
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m37s
fix(tradein/houses): персистить house_class из avito «Класс жилья» и yandex parse (#2003)
houses.house_class (comfort/business/premium/economy) заполнен у 0% из 6958
домов. Два независимых бага:

Bug 2 (quick win): avito EXPAND_PARAMS_MAP не извлекал «Класс жилья»
(developmentPage/новостройки) в колонку — добавлена нормализация RU→канон
+ COALESCE-write в _persist_house (avito = топ-приоритетный источник,
перезаписывает при наличии значения, None не затирает существующее).
Зеркальная правка в scraper-kit копии (parity-тест этого требует).

Bug 1 (частично): resolve_house_field/HOUSE_FIELD_PRIORITY
(conflict_resolution.py) подтверждён dead code (0 прод-вызовов) — не
трогал. Вместо этого починен yandex_newbuilding_sweep.py: парсил
house_class, но нигде не персистил (терялось безвозвратно) — добавлен
fill-only UPDATE (COALESCE(houses.house_class, :new) — yandex_realty_nb
низший приоритет, не перетирает более приоритетный источник).
2026-07-04 03:46:47 +03:00
..
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
avito.py fix(scrapers/avito): unify _unix_to_date timezone across serp/houses/imv (#2129) 2026-07-02 14:03:11 +00:00
avito_detail.py fix(tradein/avito): консолидировать house_type normalizer + актуализировать docstring (#2009) (#2340) 2026-07-03 22:47:05 +00:00
avito_exceptions.py fix(tradein/avito): reap dead 404 listings in detail backfill (don't trip block breaker) 2026-06-28 14:05:31 +03:00
avito_houses.py fix(tradein/houses): персистить house_class из avito «Класс жилья» и yandex parse (#2003) 2026-07-04 03:46:47 +03:00
avito_imv.py fix(scrapers/avito): unify _unix_to_date timezone across serp/houses/imv (#2129) 2026-07-02 14:03:11 +00:00
avito_shared.py fix(scrapers/avito): unify _unix_to_date timezone across serp/houses/imv (#2129) 2026-07-02 14:03:11 +00:00
base.py fix(tradein/scrapers): bump scraped_at при ре-подтверждении живым + бэкфилл (#2206) (#2224) 2026-07-02 20:30:01 +00:00
browser_fetcher.py feat(tradein-imv): route IMV backfill через BrowserFetcher.fetch_json за dormant-флагом (#915 Stage 3) (#1916) 2026-06-26 09:54:28 +00:00
cian.py fix(tradein/pii): удалить телефоны продавцов (NULL) + PII клиента (DROP) (#2217, #1969) 2026-07-03 23:00:42 +03:00
cian_detail.py fix(scrapers): cian_detail — ceiling/repair актуальные пути state (#1791) 2026-06-19 19:57:18 +03:00
cian_newbuilding.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
cian_state_parser.py fix(tradein): cian _cianConfig concat-формат + auth из header-frontend (#639) (#819) 2026-05-30 19:08:01 +00:00
cian_valuation.py fix(tradein): Mera-audit fix-1 — Cian valuation price sanity bounds 2026-06-21 09:02:32 +03:00
domclick.py feat(tradein/domclick): enrich Layer A SERP parser — description, repair-inference, minhash, seller 2026-06-27 16:35:25 +03:00
domclick_detail.py feat(tradein/domclick): local-runner + ingest tooling → main (split from parked Layer-B) 2026-06-28 10:26:52 +03:00
domclick_exceptions.py feat(tradein/domclick): local-runner + ingest tooling → main (split from parked Layer-B) 2026-06-28 10:26:52 +03:00
ekb_geoportal_client.py fix(tradein): verify=False for EKB geoportal WFS (Russian-CA cert) 2026-06-19 18:19:03 +03:00
house_type_normalizer.py fix(tradein/avito): консолидировать house_type normalizer + актуализировать docstring (#2009) (#2340) 2026-07-03 22:47:05 +00:00
price_brackets.py perf(tradein): shared EKB seed price-brackets across avito/cian/yandex (city-dict ready for multi-city) 2026-06-18 17:55:55 +03:00
repair_state_normalizer.py fix(scrapers): cian_detail — ceiling/repair актуальные пути state (#1791) 2026-06-19 19:57:18 +03:00
snapshot_writer.py fix(tradein): implement listings_snapshots + offer_price_history writers (#567) (#624) 2026-05-28 13:49:52 +00:00
yandex_detail.py fix(scrapers): yandex_detail — area/ceiling из структурного блока, покрытие area 63%->~95% (#1792) 2026-06-19 19:37:51 +03:00
yandex_helpers.py fix(tradein): Yandex listing_date — keyword + absolute-no-year (#602) 2026-05-27 13:11:22 +00:00
yandex_newbuilding.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
yandex_realty.py fix(tradein/data): yandex url-дубли — дедуп-миграция + инвариант + корень (#2235) 2026-07-03 09:39:29 +03:00
yandex_valuation.py fix(tradein): route yandex/valuation/imv/newbuilding scrapers через scraper_proxy_url (#860) 2026-05-31 07:04:31 +00:00