Add VLLMChatModel support to chat API

[jardinetsouffleton]

Description by Korbit AI

What change is being made?

Add support for the VLLMChatModel to the chat API.

Why are these changes being made?

To extend the chat API functionality to accommodate the VLLM backend, allowing users to leverage the new model configuration parameters like model_name, temperature, and max_tokens. This enhancement makes the API more versatile and capable of supporting different model backends, addressing user needs for more customizable interaction with chat models.

Is this description stale? Ask me to generate a new description by commenting /korbit-generate-pr-description

[korbit-ai]

Review by Korbit AI

Korbit automatically attempts to detect when you fix issues in new commits.

Category	Issue	Fix Detected
	Inconsistent Retry Parameter Naming ▹ view

Files scanned

​

File Path	Reviewed
src/agentlab/llm/chat_api.py	✅

Explore our documentation to understand the languages and file types we support and the files we ignore.

Need a new review? Comment /korbit-review on this PR and I'll review your latest changes.

Korbit Guide: Usage and Customization

Interacting with Korbit

• You can manually ask Korbit to review your PR using the /korbit-review command in a comment at the root of your PR.
• You can ask Korbit to generate a new PR description using the /korbit-generate-pr-description command in any comment on your PR.
• Too many Korbit comments? I can resolve all my comment threads if you use the /korbit-resolve command in any comment on your PR.
• Chat with Korbit on issues we post by tagging @korbit-ai in your reply.
• Help train Korbit to improve your reviews by giving a 👍 or 👎 on the comments Korbit posts.

Customizing Korbit

• Check out our docs on how you can make Korbit work best for you and your team.
• Customize Korbit for your organization through the Korbit Console.

Current Korbit Configuration

General Settings

​

Setting	Value
Review Schedule	Automatic excluding drafts
Max Issue Count	10
Automatic PR Descriptions	✅

Issue Categories

​

Category	Enabled
Documentation	✅
Logging	✅
Error Handling	✅
Readability	✅
Design	✅
Performance	✅
Security	✅
Functionality	✅

Feedback and Support

• Tell us what you think of Korbit
• Schedule a call with our team
• Email us @ support@korbit.ai

Note

Korbit Pro is free for open source projects 🎉

Looking to add Korbit to your team? Get started with a free 2 week trial here

[korbit-ai]

Inconsistent Retry Parameter Naming

Tell me more

What is the issue?

The 'n_retry_server' parameter name in VLLMChatModel is inconsistent with the parent class's 'max_retry' parameter, while they serve the same purpose.

Why this matters

This inconsistency could cause confusion and maintenance issues as the parameter serves the same purpose but has a different name than its parent class.

Suggested change ∙ Feature Preview

Rename the parameter to match the parent class:

def __init__(
    self,
    model_name,
    api_key=None,
    temperature=0.5,
    max_tokens=100,
    max_retry=4,  # Changed from n_retry_server
    min_retry_wait_time=60,
):
    super().__init__(
        model_name=model_name,
        api_key=api_key,
        temperature=temperature,
        max_tokens=max_tokens,
        max_retry=max_retry,  # Use consistent parameter name
        min_retry_wait_time=min_retry_wait_time,
        api_key_env_var="VLLM_API_KEY",
        client_class=OpenAI,
        client_args={"base_url": base_url},
        pricing_func=None,
    )

💬 Chat with Korbit by mentioning @korbit-ai.