feat(tradein-sweep): --require-house-number filter (default on) #536

Merged
lekss361 merged 2 commits from feat/tradein-sweep-house-number-filter into main 2026-05-24 15:03:41 +00:00

2 commits

Author SHA1 Message Date
lekss361
1d9a8e4ac4 fix(tradein-sweep): paginate until total_objects reached, dedup pages
Previous break condition (items < 13) stopped after page 1 even when Yandex's
header declared total_objects=36 or 259. Live observation: Авиационная 63к2
declared 36 objects, sweep only saved 14 from page=1 (missed 22).

New stop conditions:
  - Cumulative >= total_objects from house meta header (when present)
  - Page yields 0 new items (pure duplicates after dedup → end of feed)
  - max_pages reached

Also dedup items across pages by (publish_date, area_m2, floor, start_price)
to handle Yandex's occasional repeat at page boundaries.
2026-05-24 18:02:38 +03:00
lekss361
5ee8a20732 feat(tradein-sweep): --require-house-number filter (default on)
Yandex valuation endpoint resolves only to specific buildings; bare street
names like 'Академическая улица' return 0 items. Filter to addresses with a
house number (regex ', \d+') cuts the work pool from 583 to ~123 addresses
that actually have history data. Toggle with --no-require-house-number to
sweep everything.
2026-05-24 17:59:09 +03:00