Skip to content

fix(data-store): use REPEATABLE READ isolation for file metadata queries#1894

Merged
Theodus merged 2 commits intomainfrom
theodus/metadata-db
Mar 3, 2026
Merged

fix(data-store): use REPEATABLE READ isolation for file metadata queries#1894
Theodus merged 2 commits intomainfrom
theodus/metadata-db

Conversation

@Theodus
Copy link
Member

@Theodus Theodus commented Mar 3, 2026

Prevents queries from missing rows due to concurrent table modifications
during compaction/GC operations.

  1. Refactor streaming API to Vec (get_revision_files)
    • All callers were immediately collecting the stream into a Vec
    • Simplifies the API and makes transaction handling straightforward
  2. Add REPEATABLE READ transaction isolation
    • Wrap file metadata query in a transaction with REPEATABLE READ
    • Guarantees consistent snapshot from transaction start

Concurrent compaction/GC operations could cause the file metadata query
to miss rows, leading to spurious "missing block" detections that
triggered redundant re-dumping.

Copy link
Contributor

@JohnSwan1503 JohnSwan1503 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@Theodus Theodus merged commit dcf8572 into main Mar 3, 2026
8 of 9 checks passed
@Theodus Theodus deleted the theodus/metadata-db branch March 3, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants