-
Notifications
You must be signed in to change notification settings - Fork 3.2k
feat(bedrock): added aws bedrock as a model provider #2722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Greptile Summary
This PR adds AWS Bedrock as a new model provider, enabling access to 36+ foundation models including Claude (Anthropic), Nova (Amazon), Llama (Meta), and Mistral variants. The implementation follows the established provider pattern with comprehensive support for tool execution, streaming, structured outputs, and inference profiles.
Key Changes
- Core Provider Implementation: Created
providers/bedrock/with full Bedrock Converse API integration supporting tool calls, streaming, and forced tool usage patterns - Model Registry: Added 36 Bedrock models with pricing and capabilities (temperature 0-1, tool usage control, native structured outputs for Claude)
- Credential Management: Integrated AWS credential flow (Access Key ID, Secret Access Key, Region) across all LLM blocks
- Code Quality Improvement: Refactored duplicate provider credential configurations into reusable
getProviderCredentialSubBlocks()utility inblocks/utils.ts - Integration Points: Updated Agent, Router, Evaluator, Guardrails, and Translate blocks with Bedrock credential fields
- Infrastructure: Added
@aws-sdk/client-bedrock-runtimev3.940.0 dependency
Technical Highlights
The Bedrock provider implements advanced features:
- Inference profile ID transformation for regional routing (e.g.,
us.anthropic.claude-...) - Tool execution with parallel promise handling and iteration tracking
- Hybrid streaming support (stream after tool execution completes)
- Structured output via tool use approach for models without native support
- Comprehensive timing/cost tracking with time segments
Code Quality Note
One minor style issue identified: @ts-ignore usage on line 895 for adding timing to error objects (see inline comment).
Confidence Score: 4/5
- This PR is safe to merge with minimal risk - the implementation follows established patterns and is well-isolated
- Score reflects solid implementation quality with comprehensive provider integration, proper type safety, and established patterns. One minor style issue with
@ts-ignoreprevents a perfect score. The refactoring of credential handling into shared utilities actually improves overall code quality. - Pay close attention to
apps/sim/providers/bedrock/index.tsfor the@ts-ignoreon line 895
Important Files Changed
File Analysis
| Filename | Score | Overview |
|---|---|---|
| apps/sim/providers/bedrock/index.ts | 4/5 | implemented comprehensive AWS Bedrock provider with tool execution, streaming support, and structured outputs; includes one style issue with @ts-ignore |
| apps/sim/providers/bedrock/utils.ts | 5/5 | added utility functions for Bedrock streaming, forced tool tracking, ID generation, and inference profile resolution; clean implementation |
| apps/sim/providers/models.ts | 5/5 | added Bedrock provider definition with 36 models (Claude, Nova, Llama, Mistral variants) including pricing and capabilities |
| apps/sim/providers/registry.ts | 5/5 | registered Bedrock provider in the provider registry for server-side execution |
| apps/sim/lib/api-key/byok.ts | 5/5 | added special handling for Bedrock models to bypass standard API key requirements since it uses AWS credentials |
| apps/sim/blocks/utils.ts | 5/5 | refactored provider credential logic into reusable getProviderCredentialSubBlocks() function, added Bedrock credential fields |
| apps/sim/blocks/blocks/agent.ts | 4/5 | added Bedrock credential fields (accessKeyId, secretKey, region) and updated API key visibility logic; removed searchable property from model field |
Sequence Diagram
sequenceDiagram
participant User
participant AgentBlock
participant BlockExecutor
participant AgentHandler
participant ProvidersAPI
participant BedrockProvider
participant BedrockClient
participant ToolExecutor
User->>AgentBlock: Configure Bedrock model + AWS credentials
AgentBlock->>BlockExecutor: Execute block with inputs
BlockExecutor->>AgentHandler: Handle agent execution
AgentHandler->>ProvidersAPI: POST /api/providers
Note over ProvidersAPI: Extract bedrockAccessKeyId,<br/>bedrockSecretKey, bedrockRegion
ProvidersAPI->>BedrockProvider: executeRequest()
BedrockProvider->>BedrockProvider: Create BedrockRuntimeClient<br/>with AWS credentials
BedrockProvider->>BedrockProvider: Transform messages to<br/>Bedrock format
BedrockProvider->>BedrockProvider: Prepare tools/structured outputs
alt Streaming (no tools)
BedrockProvider->>BedrockClient: ConverseStreamCommand
BedrockClient-->>BedrockProvider: Stream chunks
BedrockProvider-->>ProvidersAPI: StreamingExecution
ProvidersAPI-->>AgentHandler: Text stream
AgentHandler-->>User: Streaming response
else Tool execution
BedrockProvider->>BedrockClient: ConverseCommand
BedrockClient-->>BedrockProvider: Response with tool calls
loop Tool iterations (max 10)
BedrockProvider->>ToolExecutor: Execute tools in parallel
ToolExecutor-->>BedrockProvider: Tool results
BedrockProvider->>BedrockClient: ConverseCommand with results
BedrockClient-->>BedrockProvider: Next response
end
alt Stream final response
BedrockProvider->>BedrockClient: ConverseStreamCommand
BedrockClient-->>BedrockProvider: Final stream
end
BedrockProvider-->>ProvidersAPI: ProviderResponse
ProvidersAPI-->>AgentHandler: Response with tool calls
AgentHandler-->>User: Complete response
end
Summary
Type of Change
Testing
Tested manually
Checklist