lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/tradein-mvp/backend/app/services/scrapers
History
bot-backend d65a88f704
All checks were successful
CI / changes (pull_request) Successful in 6s
Details
CI / backend-tests (pull_request) Has been skipped
Details
CI / frontend-tests (pull_request) Has been skipped
Details
CI / openapi-codegen-check (pull_request) Has been skipped
Details
fix(avito): reconnect-retry on detail 429 under backconnect proxy 
Detail backfill (run 217) hit 5 consecutive HTTP 429s and aborted at 19 enriched
because the detail fetcher only had a same-session 429 short-retry (no reconnect).
429s are persistent: keep-alive reuses one backconnect exit IP across sequential
detail fetches -> avito rate-limits it. Mirror the merged SERP 429 fix (#1769):
widen short-retry 4->8 and add up to 3 ephemeral-session reconnects (fresh exit IP)
on 429 exhaustion before raising. Caller session never touched (as with 403 path).
2026-06-18 23:29:47 +03:00
..
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
avito.py fix(avito): tolerate 429 per-bucket instead of banning whole sweep 2026-06-18 23:03:28 +03:00
avito_detail.py fix(avito): reconnect-retry on detail 429 under backconnect proxy 2026-06-18 23:29:47 +03:00
avito_exceptions.py fix(tradein): avito content-block (HTTP-200) не маскируется под end-of-pagination (Refs #754) (#779) 2026-05-30 16:28:23 +00:00
avito_houses.py fix(tradein): M1 review — short_address для match_or_create_house (alias keyspace) 2026-06-16 08:35:38 +03:00
avito_imv.py fix(tradein): route yandex/valuation/imv/newbuilding scrapers через scraper_proxy_url (#860) 2026-05-31 07:04:31 +00:00
base.py feat(listings): card_hash column + snapshot-dedup on unchanged cards 2026-06-18 22:01:54 +03:00
browser_fetcher.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
cian.py perf(tradein): shared EKB seed price-brackets across avito/cian/yandex (city-dict ready for multi-city) 2026-06-18 17:55:55 +03:00
cian_detail.py fix(cian): persist detail rich fields (description/kitchen/ceiling) + stamp detail_enriched_at 2026-06-17 22:32:18 +03:00
cian_newbuilding.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
cian_state_parser.py fix(tradein): cian _cianConfig concat-формат + auth из header-frontend (#639) (#819) 2026-05-30 19:08:01 +00:00
cian_valuation.py feat(tradein): dedicated Cian mobile proxy — separate egress from Avito (#923) 2026-05-31 19:30:14 +00:00
domclick.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
domclick_exceptions.py fix(tradein): domklik source name, scraper delay, DataDome block detect (#796 fixup) 2026-05-31 17:21:30 +03:00
price_brackets.py perf(tradein): shared EKB seed price-brackets across avito/cian/yandex (city-dict ready for multi-city) 2026-06-18 17:55:55 +03:00
repair_state_normalizer.py feat(tradein): infer repair_state from listing description text (#622) (#632) 2026-05-28 15:37:34 +00:00
snapshot_writer.py fix(tradein): implement listings_snapshots + offer_price_history writers (#567) (#624) 2026-05-28 13:49:52 +00:00
yandex_detail.py feat(tradein): yandex detail backfill — save_detail_enrichment + nightly task (#1553) 2026-06-16 13:58:20 +03:00
yandex_helpers.py fix(tradein): Yandex listing_date — keyword + absolute-no-year (#602) 2026-05-27 13:11:22 +00:00
yandex_newbuilding.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
yandex_realty.py perf(tradein): shared EKB seed price-brackets across avito/cian/yandex (city-dict ready for multi-city) 2026-06-18 17:55:55 +03:00
yandex_valuation.py fix(tradein): route yandex/valuation/imv/newbuilding scrapers через scraper_proxy_url (#860) 2026-05-31 07:04:31 +00:00