Conversation
- Create new @cipherstash/drizzle package with Protect.js operators - Move Drizzle integration tests from protect package to drizzle package - Add schema extraction utilities for automatic Protect.js schema generation - Update documentation with new API using createProtectOperators - Export schema types needed by Drizzle package - Add encryptedType helper for Drizzle table definitions
Co-authored-by: Dan Draper <dan@cipherstash.com>
Co-authored-by: Dan Draper <dan@cipherstash.com>
- Implement LazyOperatorPromise class to support deferred encryption - Update all operators (eq, ne, gt, gte, lt, lte, between, notBetween, like, ilike, notIlike) to return LazyOperatorPromise when encryption is needed - Override protectOps.and() to batch collect and encrypt all operator values in a single createSearchTerms call - Update TypeScript types to reflect operators can return Promise<SQL> | SQL - Update documentation and tests to demonstrate batched and() usage pattern This significantly improves performance when using multiple operators in a query by batching all encryption operations into a single call instead of multiple separate calls.
- Add type parameter examples to all encryptedType usage in schema definitions - Add TIP box explaining importance of type parameters for type safety - Update Select + Decrypt example to demonstrate type inference after decryption - Update test file to use type parameters for consistency - Remove outdated WARNING about TypeScript support work in progress - Update TODO list to reflect current state This ensures developers understand that encryptedType<T> maintains type safety throughout the encryption/decryption flow, with TypeScript knowing the correct types of decrypted data.
- Refactor LazyOperatorPromise to implement Promise<SQL> instead of extending it - Add auto-execution logic so operators work when awaited directly - Store encryption context (protectClient, defaultProtectTable, protectTableCache) in LazyOperatorPromise - Implement then/catch/finally to delegate to internal promise - Add _execute() method to handle encryption when awaited outside of and() - Update all operator constructors to pass encryption context This fixes the 'Promise resolve or reject function is not callable' error that occurred when operators were used standalone (not in protectOps.and()). Operators now work both standalone and batched in and() for maximum flexibility.
…ors in and() - Add example showing regular Drizzle operators (eq, ne, gt, etc.) can be mixed with Protect operators in protectOps.and() - Add note explaining that protectOps.and() automatically handles both encrypted and non-encrypted column operators - Update test comments to clarify mixing of operator types - Import eq from drizzle-orm in test file for clarity The implementation already supported this capability - protectOps.and() correctly separates lazy operators (for encrypted columns) from regular SQL conditions (for non-encrypted columns) and combines them appropriately.
auxesis
changed the title
- Add workflow step to create .env file for drizzle package with database URL - Add SQL script to create protect-ci table for integration tests - Update postgres entrypoint to run CI table creation script
- Add SKIP_ORDER_BY_TEST flag to drizzle and supabase test suites - CI database doesn't support order by on encrypted columns - Skip order by tests gracefully with console log message
calvinbrewer
changed the title
| export type EncryptedColumnConfig = { | ||
| /** | ||
| * Data type for the column (default: 'string') | ||
| */ | ||
| dataType?: CastAs | ||
| /** | ||
| * Enable free text search. Can be a boolean for default options, or an object for custom configuration. | ||
| */ | ||
| freeTextSearch?: boolean | MatchIndexOpts | ||
| /** | ||
| * Enable equality index. Can be a boolean for default options, or an array of token filters. | ||
| */ | ||
| equality?: boolean | TokenFilter[] | ||
| /** | ||
| * Enable order and range index for sorting and range queries. | ||
| */ | ||
| orderAndRange?: boolean | ||
| } |
There was a problem hiding this comment.
Is this type different to the ones used in protect itself?
There was a problem hiding this comment.
Those types are the same pulled from @cipherstash/schema package 🚀
| amount: encryptedType<number>('amount', { | ||
| dataType: 'number', | ||
| equality: true, | ||
| orderAndRange: true, | ||
| }), |
There was a problem hiding this comment.
Not a blocker but this is slightly duplicative. Given that the generic type is number, maybe the dataType can be inferred?
There was a problem hiding this comment.
Good call - we can probably infer the dataType from the type interface provided to encrytpedType.
examples/drizzle/README.md
Outdated
| # Database connection | ||
| DATABASE_URL="postgresql://[username]:[password]@[host]:5432/[database]" | ||
|
|
||
| # CipherStash credentials |
There was a problem hiding this comment.
Can we link to some docs or just the sign-up page for how to get these creds?
| # Get all transactions | ||
| curl http://localhost:3000/transactions | ||
|
|
||
| # Filter by account number | ||
| curl "http://localhost:3000/transactions?accountNumber=1234" | ||
|
|
||
| # Filter by amount range | ||
| curl "http://localhost:3000/transactions?minAmount=100&maxAmount=1000" | ||
|
|
||
| # Filter by status | ||
| curl "http://localhost:3000/transactions?status=completed" | ||
| ``` |
There was a problem hiding this comment.
Fine for demo but maybe add a comment that params should usually be done via POST if they are sensitive so that they don't appear in logs.
packages/drizzle/src/pg/operators.ts
Outdated
| isNull, | ||
| isNotNull, | ||
| not, | ||
| and: protectAnd, |
There was a problem hiding this comment.
Move this up to above the comment.
packages/drizzle/README.md
Outdated
| }) | ||
| ``` | ||
|
|
||
| > 💡 **Type Safety Tip**: Always specify the type parameter (`encryptedType<string>`, `encryptedType<number>`, etc.) to maintain type safety after decryption. Without it, decrypted values will be typed as `unknown`. |
There was a problem hiding this comment.
Worth explaining in a note below that this is because the database doesn't know what type that was encrypted.
Suggested (outside of the quote):
This is because the database only sees encrypted data and doesn't know the underlying type. It must be specified in the ORM.
| const results = await db | ||
| .select() | ||
| .from(usersTable) | ||
| .where(await protectOps.eq(usersTable.email, 'jane@example.com')) |
There was a problem hiding this comment.
It's slightly annoying that we need the await here. I wonder if the Drizzle team have any suggestions about how we could avoid it.
There was a problem hiding this comment.
Since it requires a call to ZeroKMS to create the encrypted search term we'd need for the core Drizzle interface to be aware of promises in the filter clauses
There was a problem hiding this comment.
Do the underlying FFI calls have to be async?
Would it be possible to expose an (optional) sync interface for some use cases?
packages/drizzle/README.md
Outdated
| const decrypted = await protectClient.bulkDecryptModels(results) | ||
| ``` | ||
|
|
||
| > ⚠️ **Note**: Sorting with ORE on Supabase and other databases that don't support operator families may not work as expected. |
There was a problem hiding this comment.
Sorting won't work on Supabase. Period.
packages/drizzle/README.md
Outdated
| ) | ||
| ``` | ||
|
|
||
| > 💡 **Performance Tip**: Using `protectOps.and()` batches all encryption operations into a single `createSearchTerms` call, which is more efficient than awaiting each operator individually. |
| @@ -0,0 +1,426 @@ | |||
| # Drizzle ORM Integration with Protect.js | |||
There was a problem hiding this comment.
Is this doc superseded by the README? Seems like its saying the same stuff
There was a problem hiding this comment.
Yeah it's pretty much a duplicate. I'll just remove it and we can add better how tos later
3 participants
Change Summary:
Added: The Cipherstash Drizzle package along with its initial interface utilizing protect operators.
Enhanced: Enabled users to extend the Encrypted type, a custom type from Drizzle, allowing for the definition of custom protect schemas.
This update aims to improve data protection capabilities and flexibility when working with encrypted data types in Drizzle.