lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions

feat(tradein): global scraper delay setting (applies across all scrapers) #485

Merged
lekss361 merged 2 commits from feat/tradein-global-scraper-delay into main 2026-05-23 15:43:28 +00:00
Conversation 3 Commits 2 Files changed 7 +399 -144

2 commits

Author SHA1 Message Date
lekss361
814530d187 Merge remote-tracking branch 'forgejo/main' into feat/tradein-global-scraper-delay 
# Conflicts:
#	tradein-mvp/backend/app/services/scraper_settings.py
#	tradein-mvp/backend/data/sql/053_scraper_settings.sql
#	tradein-mvp/backend/tests/test_scraper_settings.py
 2026-05-23 18:38:48 +03:00
lekss361
bfe56e44d6 feat(tradein): global scraper delay setting (applies across all scrapers) 
- SQL migrations 053 (scraper_settings table) + 054 (seed global + per-source rows)
- scraper_settings.py: get_scraper_delay() returns max(per_source, global);
  in-memory cache TTL=60s; invalidate_cache() for immediate effect on PUT
- avito/cian/n1 scrapers load delay from DB in __init__ (mirrors yandex pattern)
- Admin API: GET /scraper-settings (list all), PUT /scraper-settings/{source}
  with cache invalidation on update; CAST(:d AS numeric) per psycopg v3 rules
- 10 unit tests: global>per_source, per_source>global, global=0, DB error fallback,
  cache invalidation, API endpoint smoke
 2026-05-23 18:25:40 +03:00