feat: Add support for multiplexed sessions#1381
Merged
Conversation
.github/workflows/integration-tests-against-emulator-with-multiplexed-session.yaml
Fixed
Show fixed
Hide fixed
f04b0e1 to
e8f813c
Compare
e8f813c to
69f28c8
Compare
fb74975 to
8899249
Compare
currantw
reviewed
May 26, 2025
currantw
reviewed
May 26, 2025
53f0cc5 to
6dc6b4b
Compare
ebdceba to
bbbc41e
Compare
.github/workflows/integration-tests-against-emulator-with-multiplexed-session.yaml
Dismissed
Show dismissed
Hide dismissed
841408a to
146f11a
Compare
e83298f to
aa0e2a5
Compare
4c8a19b to
d7c54cd
Compare
Contributor
Author
graph TD
A[Client Request] --> B{Multiplexed Sessions Enabled?}
B -->|No| C[Regular Session Behavior]
B -->|Yes| D[Multiplexed Session Behavior]
%% Regular Session Path
C --> C1[Read-Only Operations]
C --> C2[Partitioned Operations]
C --> C3[Read-Write Operations]
C1 --> C1A[Get Session from Pool]
C1A --> C1B[Create/Use Regular Session]
C1B --> C1C[Execute Read Operation]
C1C --> C1D[Return Session to Pool]
C2 --> C2A[BatchSnapshot Creation]
C2A --> C2B[Get Session from Pool<br/>TransactionType.PARTITIONED]
C2B --> C2C[Create Regular Session]
C2C --> C2D[Generate Partitions]
C2D --> C2E[Each Partition Uses<br/>SAME Session]
C2E --> C2F[Process All Partitions<br/>Sequentially]
C3 --> C3A[Get Session from Pool]
C3A --> C3B[Create/Use Regular Session]
C3B --> C3C[Execute Transaction]
C3C --> C3D[Return Session to Pool]
%% Multiplexed Session Path
D --> D1[Read-Only Operations]
D --> D2[Partitioned Operations]
D --> D3[Read-Write Operations]
D1 --> D1A[Get Multiplexed Session]
D1A --> D1B{Multiplexed Session Exists?}
D1B -->|No| D1C[Create Multiplexed Session]
D1B -->|Yes| D1D[Reuse Existing Session]
D1C --> D1E[Start Maintenance Thread]
D1E --> D1F[Execute Read Operation]
D1D --> D1F
D1F --> D1G[Session Remains Active<br/>No Return Needed]
D2 --> D2A[BatchSnapshot Creation]
D2A --> D2B[Get Multiplexed Session<br/>TransactionType.PARTITIONED]
D2B --> D2C{Multiplexed Session Exists?}
D2C -->|No| D2D[Create Multiplexed Session]
D2C -->|Yes| D2E[Reuse Existing Session]
D2D --> D2F[Generate Partitions]
D2E --> D2F
D2F --> D2G[Each Partition Uses<br/>SAME Multiplexed Session]
D2G --> D2H[Process All Partitions<br/>Concurrently Possible]
D3 --> D3A[Fallback to Regular Session]
D3A --> D3B[Get Session from Pool]
D3B --> D3C[Create/Use Regular Session]
D3C --> D3D[Execute Transaction]
D3D --> D3E[Return Session to Pool]
%% Environment Variables
E[Environment Variables] --> E1[GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS=true]
E --> E2[GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_PARTITIONED_OPS=true]
E1 --> B
E2 --> D2
%% Key Differences
F[Key Differences] --> F1[Session Reuse]
F --> F2[Concurrency]
F --> F3[Resource Usage]
F1 --> F1A[Regular: New session per operation]
F1 --> F1B[Multiplexed: Same session shared]
F2 --> F2A[Regular: Sequential partition processing]
F2 --> F2B[Multiplexed: Concurrent partition processing]
F3 --> F3A[Regular: Multiple sessions = more resources]
F3 --> F3B[Multiplexed: Single session = fewer resources]
%% Styling
classDef regularPath fill:#ffcccc,stroke:#ff0000,stroke-width:2px
classDef multiplexedPath fill:#ccffcc,stroke:#00ff00,stroke-width:2px
classDef envVar fill:#ffffcc,stroke:#ffaa00,stroke-width:2px
classDef keyDiff fill:#ccccff,stroke:#0000ff,stroke-width:2px
class C,C1,C2,C3,C1A,C1B,C1C,C1D,C2A,C2B,C2C,C2D,C2E,C2F,C3A,C3B,C3C,C3D regularPath
class D,D1,D2,D3,D1A,D1B,D1C,D1D,D1E,D1F,D1G,D2A,D2B,D2C,D2D,D2E,D2F,D2G,D2H multiplexedPath
class E,E1,E2 envVar
class F,F1,F2,F3,F1A,F1B,F2A,F2B,F3A,F3B keyDiff
|
currantw
reviewed
May 27, 2025
currantw
reviewed
May 27, 2025
Comment on lines
+3356
to
+3340
| rows1 = list(snap1.read(sd.TABLE, sd.COLUMNS, sd.ALL)) | ||
| rows2 = list(snap2.read(sd.TABLE, sd.COLUMNS, sd.ALL)) |
Contributor
There was a problem hiding this comment.
Tests seem to be failing here because the snapshot's session has does not exist. This means that Session.create() has not been called. Not sure exactly why that wouldn't be the case, but tough to find out without being able to run locally.
Contributor
Author
There was a problem hiding this comment.
Issue was some tests were calling session.delete which prematurely deletes the multiplex session and hence "Session Not Found" error, Using Emulator calling session.delete() does not actually delete it from Emulator so issue does not occur using Emulator
d7c54cd to
9f2046a
Compare
sakthivelmanii
previously approved these changes
May 27, 2025
4e57db2 to
16954a6
Compare
16954a6 to
94b88dd
Compare
sakthivelmanii
approved these changes
May 28, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕