lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions

fix(tradein-matching): pg_advisory_xact_lock to prevent duplicate house INSERTs

Browse source 
Two scrapers calling match_or_create_house() concurrently for the same address
could both miss Tier 0-3 and each INSERT a new canonical house — splitting
listings across duplicate house rows and breaking aggregations.

Serialize via pg_advisory_xact_lock(42, hashtext(fingerprint)). Lock is
transaction-scoped (released on caller's COMMIT/ROLLBACK). After unblock, the
losing scraper's Tier 2 fingerprint lookup hits the winner's alias and returns
the same house_id.

Closes finding #1 from 2026-05-24 trade-in audit.
This commit is contained in:
lekss361 2026-05-24 13:15:50 +03:00
parent 02b48b8d04
commit 9a6890de6a
1 changed files with 17 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
tradein-mvp/backend/app/services/matching/houses.py
View file

@ -34,11 +34,11 @@ def match_or_create_house(
) -> tuple[int, float, str]: ) -> tuple[int, float, str]:
"""Match existing house or create new canonical record. """Match existing house or create new canonical record.
⚠️ KNOWN LIMITATION: Concurrent calls for the same address can race — Concurrency-safe: serializes concurrent calls for the same address fingerprint
two scrapers calling simultaneously for an unknown address can both via pg_advisory_xact_lock(42, hashtext(fp)). The lock is transaction-scoped,
miss Tier 0-3 and each INSERT a new house row. Mitigation (Stage 8.x): released on the caller's COMMIT/ROLLBACK. Without this, two scrapers calling
gate scraper concurrency to 1 per fingerprint OR add for an unknown address could both miss Tier 0-3 and each INSERT a duplicate
pg_advisory_xact_lock(hashtext(normalized_address)) at function entry. house row. Closes finding #1 from 2026-05-24 audit.
Returns: Returns:
(house_id, confidence ∈ [0.0, 1.0], method ∈ { (house_id, confidence ∈ [0.0, 1.0], method ∈ {
@ -54,6 +54,18 @@ def match_or_create_house(
""" """
cad = building_cadastral_number or cadastral_number cad = building_cadastral_number or cadastral_number
# Compute fingerprint early so we can acquire the advisory lock before any tier reads.
fp = address_fingerprint(address, lat, lon)
# Serialize concurrent calls for the same address fingerprint to prevent
# racing INSERTs into houses (finding #1 from 2026-05-24 audit).
# pg_advisory_xact_lock takes (int4, int4) — namespace 42 = "tradein house matching".
# Lock is released automatically on caller's COMMIT/ROLLBACK.
db.execute(
text('SELECT pg_advisory_xact_lock(42, hashtext(:fp))'),
{'fp': fp},
)
# Tier 0: cadastral number exact match # Tier 0: cadastral number exact match
if cad: if cad:
row = db.execute( row = db.execute(
@ -83,9 +95,6 @@ def match_or_create_house(
ext_id) ext_id)
return (house_id, 1.0, 'source_exact') return (house_id, 1.0, 'source_exact')
# Compute fingerprint once for Tier 2 and reuse for new house registration
fp = address_fingerprint(address, lat, lon)
# Tier 2: address fingerprint lookup # Tier 2: address fingerprint lookup
row = db.execute( row = db.execute(
text( text(