lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/tradein-mvp/backend/tests/scrapers
History
bot-backend dbef242716 feat(tradein/scrapers): domclick вторичка через bff-search-web JSON API (Layer A) 
- DomClickScraper переписан на GET bff-search-web.domclick.ru/api/offers/v1
  с рекурсивным бинарным price-band bucketing (OFFSET_CAP=2000, open-band split
  через _HIGH_ANCHOR=30M); geo-guard по offerRegionName==ЕКБ / bbox
- scrape_pipeline.py: rooms=[0..5] (добавлен 5+), pages default=100,
  _DOMCLICK_PER_FETCH_S=4.0, честный статус mark_failed при 0 лотов + errors
- config.py: bff-search-web.domclick.ru добавлен в scrape_allowed_hosts
- data/sql/132_update_domclick_sweep_params.sql: UPDATE default_params (DORMANT)
2026-06-27 13:51:33 +03:00
..
__init__.py fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487) 2026-05-23 20:08:40 +00:00
test_avito_403_backconnect_reconnect.py fix(avito): gate 403 reconnect-retry on proxy presence only, not rotate_url absence 2026-06-18 21:31:20 +03:00
test_avito_429_bucket_skip.py fix(avito): tolerate 429 per-bucket instead of banning whole sweep 2026-06-18 23:03:28 +03:00
test_avito_anti_bot.py fix(scrapers): avito pipeline — новостройки house graceful (без housePage), AvitoBlockedError per-item не крашит прогон (#1820) 2026-06-20 09:53:46 +03:00
test_avito_detail_403_reconnect.py fix(avito): reconnect-retry on detail 403/firewall under backconnect proxy 2026-06-18 22:12:03 +03:00
test_avito_detail_429_reconnect.py fix(avito): reconnect-retry on detail 429 under backconnect proxy 2026-06-18 23:29:47 +03:00
test_avito_detail_browser_only.py fix(scrapers): avito_detail_backfill — снапшот ЕКБ+active, detail browser-only без curl-fallback на бан (#1814) 2026-06-19 21:58:35 +03:00
test_avito_exhaustive.py fix(tradein): avito _extract_total_count returns 0 on no-results page (stop empty-bracket split waste + fallback-listing scrape) 2026-06-18 20:01:41 +03:00
test_avito_imv_browser_transport.py feat(tradein-imv): route IMV backfill через BrowserFetcher.fetch_json за dormant-флагом (#915 Stage 3) (#1916) 2026-06-26 09:54:28 +00:00
test_avito_incremental_pagination.py feat(avito): incremental SERP pagination with date early-stop (param-gated, default off) 2026-06-18 22:01:46 +03:00
test_avito_sweep_retry.py fix(tradein): avito sweep page-level retry — transient page error no longer aborts run 2026-06-18 14:29:01 +03:00
test_browser_fetcher.py feat(tradein-browser): additive POST /fetch-json + BrowserFetcher.fetch_json (#915 Stage 2) (#1915) 2026-06-26 09:30:35 +00:00
test_cian_exhaustive.py feat(scrapers): cian SERP на BrowserFetcher (camoufox), снят code-side rotate (#1806) 2026-06-19 20:41:44 +03:00
test_domclick.py feat(tradein/scrapers): domclick вторичка через bff-search-web JSON API (Layer A) 2026-06-27 13:51:33 +03:00
test_price_brackets.py perf(tradein): shared EKB seed price-brackets across avito/cian/yandex (city-dict ready for multi-city) 2026-06-18 17:55:55 +03:00