Commit graph

6 commits

Author SHA1 Message Date
fc5c4b1e40 Merge pull request 'chore(tradein): полное выключение источника n1 — миграция 165 + вычистка backend/ops/frontend (#2204)' (#2276) from chore/tradein-remove-n1-source into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Successful in 37s
Deploy Trade-In / test (push) Successful in 1m49s
Deploy Trade-In / build-frontend (push) Successful in 2m56s
Deploy Trade-In / build-backend (push) Successful in 1m13s
Deploy Trade-In / deploy (push) Successful in 2m53s
2026-07-03 07:33:28 +00:00
bot-backend
dd050af16d chore(tradein): полное выключение источника n1 — миграция 165 + вычистка backend/ops (#2204) 2026-07-03 10:08:56 +03:00
bot-backend
8a10d943d7 fix(tradein/scraper-kit): scale cian anchor watchdog for proxy-pool browser fetches (#2160)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 11s
CI / changes (pull_request) Successful in 11s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m51s
Fixed ANCHOR_TIMEOUT_SEC=240 guillotined every cian anchor when running via
USE_PROXY_POOL_BROWSER=true: each SERP fetch goes through camoufox with relaunch
on proxy swap (13-45s/page), and an anchor = 4 room-buckets x pages_per_anchor
SERP fetches + detail_top_n details + houses-enrich >> 240s. save_listings never
ran, lots_fetched=0, and after 3 consecutive skips the run was mark_banned.

Introduces _cian_anchor_timeout_s() mirroring the avito/yandex/domclick scaling
pattern: max(ANCHOR_TIMEOUT_SEC, buckets*pages*(delay+per_serp) + details*per_detail
+ houses_budget). Defaults -> 1580s worst-case watchdog (hang guard, not expected
duration). Applied to both the scraper-kit copy (strangler target) and the active
app.services.scrape_pipeline copy that run 581 actually hit.
2026-07-03 10:05:06 +03:00
158a24c25b feat(tradein): schedule cian_full_load (exhaustive secondary) + anchor sweep NB-only
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Подключает prod-проверенный run_cian_full_load (exhaustive региональный сбор
Cian ЕКБ вторички, room×price партиционирование, incremental on_bucket save) в
in-app scheduler как recurring-источник cian_full_load (окно 20-22 UTC — отделено
от cian_city_sweep/history_backfill на 2-5 чтобы не конкурировать на shared
kf-прокси).

run_cian_city_sweep теперь NB-only (newbuilding_only=True default): SERP-фаза
сохраняет только novostroyki-лоты, вторичку отбрасывает — ею авторитетно владеет
full_load. Убирает дубль secondary SERP-сейва. DETAIL/HOUSES-фазы не затронуты.
2026-06-18 13:25:23 +03:00
10b4b02446 fix(tradein): cian sweep houses-phase — JOIN house_sources + graceful (не fatal) (#868)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Has been cancelled
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 30s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 07:59:26 +00:00
f2a123e2aa feat(tradein): cian city-sweep — один прогон SERP+detail(+price-history)+houses (#864)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 29s
Deploy Trade-In / build-backend (push) Successful in 41s
Deploy Trade-In / deploy (push) Successful in 35s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 07:24:34 +00:00