Conversation
- Add shrinkIncomingEdges() to ElementLevelData to reclaim unused capacity - Add getHNSWElementIncomingEdges() to get incoming edge counts per level - Add shrinkAllIncomingEdges() to shrink all incoming edge vectors in index - Add C API exports: VecSimDebug_GetElementIncomingEdgesInHNSWGraph, VecSimDebug_ReleaseElementIncomingEdgesInHNSWGraph, VecSimDebug_ShrinkIncomingEdgesInHNSWGraph - Add tiered index wrappers - Add unit test for shrinkIncomingEdges
🛡️ Jit Security Scan Results
✅ No security findings were detected in this PR
Security scan by Jit
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
| } | ||
| } | ||
| return total_saved; | ||
| } |
There was a problem hiding this comment.
Missing locks in shrinkAllIncomingEdges causes data races
High Severity
shrinkAllIncomingEdges reads curElementCount, iterates all elements, and mutates incoming edge vectors via shrink_to_fit (which reallocates memory) without holding indexDataGuard or per-node locks. In contrast, the read-only getHNSWElementIncomingEdges and getHNSWElementNeighbors both acquire indexDataGuard shared lock and lockNodeLinks. A concurrent insert/delete could modify curElementCount or the same incoming-edges vector, causing undefined behavior (e.g., use-after-free). The tiered wrapper only takes mainIndexGuard shared but still skips the inner indexDataGuard.
Additional Locations (1)
Codecov Report❌ Patch coverage is ❌ Your project check has failed because the head coverage (95.76%) is below the adjusted base coverage (96.14%). You can increase the head coverage or adjust the Removed Code Behavior. Additional details and impacted files@@ Coverage Diff @@
## main #918 +/- ##
==========================================
- Coverage 97.14% 95.76% -1.39%
==========================================
Files 129 129
Lines 7557 7689 +132
==========================================
+ Hits 7341 7363 +22
- Misses 216 326 +110 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
1 participant
Add debug APIs to inspect and optimize incoming unidirectional edges in HNSW graphs.
Changes
shrinkIncomingEdges()- Reclaim unused capacity in incoming edge vectorsgetHNSWElementIncomingEdges()- Get incoming edge counts per level for a labelshrinkAllIncomingEdges()- Shrink all incoming edge vectors in the indexC API Exports
VecSimDebug_GetElementIncomingEdgesInHNSWGraphVecSimDebug_ReleaseElementIncomingEdgesInHNSWGraphVecSimDebug_ShrinkIncomingEdgesInHNSWGraphIncludes tiered index support and unit test.
Pull Request opened by Augment Code with guidance from the PR author
Note
Medium Risk
Adds new debug/maintenance APIs that traverse and mutate HNSW graph edge-storage (
shrink_to_fit) under locks; risk is mainly around memory/perf characteristics and potential contention, not query correctness.Overview
Adds a new HNSW debug capability to inspect incoming unidirectional edges by returning per-level counts for a given label (
getHNSWElementIncomingEdges), with tiered-index support and C API exports (VecSimDebug_GetElementIncomingEdgesInHNSWGraph+ releaser).Introduces a memory reclamation path for incoming-edge storage:
ElementLevelData::shrinkIncomingEdges()and index-wideshrinkAllIncomingEdges()(plus tiered wrapper and C APIVecSimDebug_ShrinkIncomingEdgesInHNSWGraph) to reclaim unused vector capacity that can accumulate after deletion repairs, with a unit test validating allocation decreases and idempotence.Written by Cursor Bugbot for commit 7dc2b4b. This will update automatically on new commits. Configure here.