|
|
0084efc12a
|
fix(scrapers): yandex_detail — area/ceiling из структурного блока, покрытие area 63%->~95% (#1792)
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
|
2026-06-19 19:37:51 +03:00 |
|
|
|
03e43f12b5
|
feat(tradein): yandex detail backfill — save_detail_enrichment + nightly task (#1553)
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
- yandex_detail.py: добавить save_detail_enrichment(db, listing_id, enrichment) → bool;
UPDATE listings COALESCE по 17 полям (rooms/area_m2/floor/total_floors/address/
description/repair_state/publish_date/views_total_yandex/publish_date_relative/
agency_name/agency_founded_year/agency_objects_count/metro_stations/photo_urls/
newbuilding_url/newbuilding_id); metro/photos → jsonb; detail_enriched_at = NOW()
- app/tasks/yandex_detail_backfill.py: новый task run_yandex_detail_backfill;
snapshot SELECT source='yandex' AND detail_enriched_at IS NULL; YandexDetailScraper
async context manager; consecutive None abort (max_consecutive_blocks=5); budget guard;
run lifecycle update_heartbeat/mark_done/mark_failed
- scheduler.py: trigger_yandex_detail_backfill_run + elif source=='yandex_detail_backfill'
в scheduler_loop; window 12-15 UTC (после avito_detail 09-12 UTC)
- migration 113: ADD COLUMN IF NOT EXISTS photo_urls jsonb, newbuilding_url text,
newbuilding_id text; seed scrape_schedules yandex_detail_backfill enabled=true
- tests: 10 тестов (8 backfill + 2 save_detail_enrichment); 1819 passed 0 failed
Closes #1553
|
2026-06-16 13:58:20 +03:00 |
|
|
|
fa2bd20ac6
|
refactor(tradein): drop dead schema — sale_type_text column + audit_log (#731) (#893)
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
|
2026-05-31 13:13:43 +00:00 |
|
|
|
312df8536a
|
feat(tradein): infer repair_state from listing description text (#622) (#632)
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 38s
Deploy Trade-In / build-backend (push) Successful in 48s
|
2026-05-28 15:37:34 +00:00 |
|
|
|
b21d7c7c85
|
feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484)
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 39s
Deploy Trade-In / deploy (push) Successful in 32s
|
2026-05-23 15:28:34 +00:00 |
|
|
|
3d90221fa0
|
feat(tradein): yandex_detail.py — Product JSON-LD + DOM detail parser (#466)
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 38s
Deploy Trade-In / deploy (push) Successful in 21s
Stage 4 of YandexRealtyScraper v1. YandexDetailScraper.fetch_detail(offer_url) extracts authoritative price from Product JSON-LD (offers.price exact int) + DOM sections (description, agent, stats, metro, photos 8 sizes, newbuilding link, NLP). 35 unit tests, ruff clean. Photo URLs sanitized downstream via _safe_url CDN allowlist (avatars.mds.yandex.net).
|
2026-05-23 13:45:09 +00:00 |
|
