lekss361/gendesign
1
0
Fork 0
Code Issues 141 Pull requests Projects Releases Packages Wiki Activity Actions

feat(tradein): вся Свердловская область — 40 городов в city-sweep #2879

Merged
lekss361 merged 2 commits from feat/tradein-oblast-full into main 2026-08-13 19:08:45 +00:00
Conversation 0 Commits 2 Files changed 6 +25 -6
2 changed files with 25 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit 609768a3bd - Show all commits

10
tradein-mvp/backend/data/sql/262_scrape_schedules_seed_oblast_city_sweeps_wave2.sql
View file

@ -280,7 +280,7 @@ VALUES
'avito_city_sweep_verkhnyaya_tura',
false,
23,
24,
0,
((CURRENT_DATE + INTERVAL '1 day') + make_interval(hours => 23)) AT TIME ZONE 'UTC',
'{"pages_per_anchor": 3, "detail_top_n": 0, "request_delay_sec": 7.0, "enrich_houses": true, "radius_m": 3000, "interval_days": 3, "city": "verkhnyaya_tura"}'::jsonb
),
@ -427,7 +427,7 @@ VALUES
'cian_city_sweep_sysert',
false,
23,
24,
0,
((CURRENT_DATE + INTERVAL '1 day') + make_interval(hours => 23)) AT TIME ZONE 'UTC',
'{"pages_per_anchor": 3, "request_delay_sec": 5, "radius_m": 3000, "detail_top_n": 10, "enrich_houses": true, "interval_days": 3, "city": "sysert"}'::jsonb
),
@ -555,7 +555,7 @@ VALUES
'cian_city_sweep_lesnoy',
false,
23,
24,
0,
((CURRENT_DATE + INTERVAL '1 day') + make_interval(hours => 23)) AT TIME ZONE 'UTC',
'{"pages_per_anchor": 3, "request_delay_sec": 5, "radius_m": 3000, "detail_top_n": 10, "enrich_houses": true, "interval_days": 3, "city": "lesnoy"}'::jsonb
),
@ -766,7 +766,7 @@ VALUES
'yandex_city_sweep_degtyarsk',
false,
23,
24,
0,
((CURRENT_DATE + INTERVAL '1 day') + make_interval(hours => 23)) AT TIME ZONE 'UTC',
'{"pages_per_anchor": 3, "request_delay_sec": 9, "radius_m": 25000, "interval_days": 3, "city": "degtyarsk"}'::jsonb
),
@ -910,7 +910,7 @@ VALUES
'yandex_city_sweep_verkhnyaya_tura',
false,
23,
24,
0,
((CURRENT_DATE + INTERVAL '1 day') + make_interval(hours => 23)) AT TIME ZONE 'UTC',
'{"pages_per_anchor": 3, "request_delay_sec": 9, "radius_m": 25000, "interval_days": 3, "city": "verkhnyaya_tura"}'::jsonb
),

21
tradein-mvp/backend/tests/test_migration_262_oblast_city_sweeps_wave2.py
View file

@ -393,8 +393,27 @@ def _row_windows(path: Path) -> list[tuple[str, str, int, int]]:
def test_migration_262_windows_are_one_hour() -> None:
"""Окно ровно на час. Час 23 заворачивается в 0 (cross-midnight) — scheduler
это поддерживает явно (`window_end_hour <= window_start_hour` → окно через
полночь, см. scraper_kit.orchestration.scheduler)."""
for provider, slug, start, end in _row_windows(_MIGRATION_262):
assert end == start + 1, f"{provider}_{slug}: window [{start},{end}) не 1-часовое"
expected = (start + 1) % 24
assert (
end == expected
), f"{provider}_{slug}: window [{start},{end}) не 1-часовое (ожидали end={expected})"
def test_migration_262_window_hours_satisfy_db_check_constraint() -> None:
"""Оба часа обязаны попадать в 0..23 — иначе миграция не применится вовсе.
Регресс: первая версия давала start=23, end=24 у пяти строк. Проверка
«end == start + 1» это пропускала, а прод-констрейнт `window_end_range`
(052_scrape_schedules.sql: CHECK (window_end_hour BETWEEN 0 AND 23)) —
нет, и весь INSERT падал в CI на реальном postgres.
"""
for provider, slug, start, end in _row_windows(_MIGRATION_262):
assert 0 <= start <= 23, f"{provider}_{slug}: window_start_hour={start} вне 0..23"
assert 0 <= end <= 23, f"{provider}_{slug}: window_end_hour={end} вне 0..23"
def test_migration_262_windows_avoid_ekb_and_wave1_occupied_hours() -> None: