Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "meta-llama/Llama-2-7b-chat-hf"
summarize: false
summaryModel: "meta-llama/Llama-2-7b-chat-hf"
modelDisplayLabel: "Anyscale"
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,6 @@ endpoints:
fetch: false
titleConvo: true
titleModel: "claude-3-haiku"
summarize: false
summaryModel: "claude-3-haiku"
dropParams: ["stream"]
modelDisplayLabel: "APIpie"
```
Expand Down
41 changes: 1 addition & 40 deletions content/docs/configuration/librechat_yaml/ai_endpoints/azure.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ endpoints:
azureOpenAI:
# Endpoint-level configuration
titleModel: "llama-70b-chat"
plugins: true
assistants: true
groups:
# Group-level configuration
Expand Down Expand Up @@ -94,15 +93,11 @@ To integrate Azure OpenAI with LibreChat, specific fields must be configured in
<OptionTable
options={[
['titleModel', 'string', 'Specifies the model to use for generating conversation titles. If not provided, the default model is set as `gpt-3.5-turbo`, which will result in no titles if lacking this model. You can also set this to dynamically use the current model by setting it to `current_model`.', 'titleModel:'],
['plugins', 'boolean', 'Enables the use of plugins through Azure. Set to `true` to activate Plugins endpoint support through your Azure config. Default: `false`.', 'plugins:false'],
['assistants', 'boolean', 'Enables the use of assistants through Azure. Set to `true` to activate Assistants endpoint through your Azure config. Default: `false`. Note: this requires an assistants-compatible region.', 'assistants:false'],
['summarize', 'boolean', 'Enables conversation summarization for all Azure models. Set to `true` to activate summarization. Default: `false`.', 'summarize:false'],
['summaryModel', 'string', 'Specifies the model to use for generating conversation summaries. If not provided, the default behavior is to use the first model in the `default` array of the first group.', 'summaryModel:'],
['titleConvo', 'boolean', 'Enables conversation title generation for all Azure models. Set to `true` to activate title generation. Default: `false`.', 'titleConvo:false'],
['titleMethod', 'string', 'Controls the method used for generating conversation titles. Valid values: "completion" (default), "structured", "functions" (legacy alias for "structured").', 'titleMethod:completion'],
['titlePrompt', 'string', 'Custom prompt for title generation. Must include {convo} placeholder for the conversation content.', 'See documentation for default prompt'],
['titlePromptTemplate', 'string', 'Template for formatting conversation content. Must include {input} and {output} placeholders. Default: "User: {input}\\nAI: {output}"', 'titlePromptTemplate:'],
['titleEndpoint', 'string', 'Alternative endpoint to use for title generation. Accepted values: openAI, azureOpenAI, google, anthropic, bedrock, or custom endpoint names.', 'titleEndpoint:'],
]}
/>

Expand All @@ -113,25 +108,13 @@ To integrate Azure OpenAI with LibreChat, specific fields must be configured in
]}
/>

**Custom Order (Optional):**
<OptionTable
options={[
['customOrder', 'number', 'Allows you to specify a custom order for the Azure endpoint in the user interface. Higher numbers will appear lower in the list. If not provided, the default order is determined by the order in which the endpoints are defined in the `librechat.yaml` file.', 'customOrder:'],
]}
/>

The `customOrder` option is commented out, as it is optional.

Example of these endpoint-level settings in your `librechat.yaml` file:

```yaml filename="librechat.yaml"
endpoints:
azureOpenAI:
titleModel: "gpt-3.5-turbo-1106"
plugins: true
assistants: true
summarize: true
summaryModel: "gpt-3.5-turbo-1106"
titleConvo: true
titleMethod: "functions"
groups:
Expand Down Expand Up @@ -358,8 +341,7 @@ endpoints:
- For this reason, it's recommended you use only one region or resource group for Azure OpenAI Assistants, or you will experience an error.
- Uploading to "OpenAI" is the default behavior for official `code_interpreter` and `retrieval` capabilities.
- Downloading files that assistants generate will soon be supported.
- As of May 19th 2024, retrieval and streaming are not yet supported through Azure OpenAI.
- To avoid any errors with retrieval while it's not supported, it's recommended to disable the capability altogether through the `azureAssistants` endpoint config:
- The default `azureAssistants` capabilities include `retrieval`. To turn it off, list the capabilities you want explicitly and omit it:

```yaml filename="librechat.yaml"
endpoints:
Expand All @@ -371,27 +353,6 @@ endpoints:
capabilities: ["code_interpreter", "actions", "tools"]
```

- By default, all capabilities, except retrieval, are enabled.

### Using Plugins with Azure

To use the Plugins endpoint with Azure OpenAI, you need a deployment supporting **[function calling](https://techcommunity.microsoft.com/t5/azure-ai-services-blog/function-calling-is-now-available-in-azure-openai-service/ba-p/3879241)**. Otherwise, set "Functions" off in the Agent settings. When you are not using "functions" mode, it's recommended to have "skip completion" off as well, which is a review step of what the agent generated.

To use Azure with the Plugins endpoint, make sure the field `plugins` is set to `true` in your Azure OpenAI endpoint config:

```yaml filename="librechat.yaml"
# Example Azure OpenAI Object Structure
endpoints:
azureOpenAI:
plugins: true # <------- Set this
groups:
# omitted for brevity
```

Configuring the `plugins` field will configure Plugins to use Azure models.

**NOTE**: The current configuration through `librechat.yaml` uses the primary model you select from the frontend for Plugin use, which is not usually how it works without Azure, where instead the "Agent" model is used. The Agent model setting can be ignored when using Plugins through Azure.

### Using a Specified Base URL with Azure

The base URL for Azure OpenAI API requests can be dynamically configured. This is useful for proxying services such as [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/providers/azureopenai/), or if you wish to explicitly override the baseURL handling of the app.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "accounts/fireworks/models/llama-v2-7b-chat"
summarize: false
summaryModel: "accounts/fireworks/models/llama-v2-7b-chat"
modelDisplayLabel: "Fireworks"
dropParams: ["user"]
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ endpoints:
"mistralai/Mixtral-8x7B-Instruct-v0.1",
"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO",
]
fetch: true
fetch: false
titleConvo: true
titleModel: "NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO"
dropParams: ["top_p"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "gpt-3.5-turbo"
summarize: false
summaryModel: "gpt-3.5-turbo"
modelDisplayLabel: "LiteLLM"
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@ endpoints:
fetch: false # fetching list of models is not supported
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
modelDisplayLabel: "Apple MLX"
addParams:
max_tokens: 2000
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
modelDisplayLabel: "NeurochainAI"
iconURL: "https://raw.githubusercontent.com/LibreChat-AI/librechat-config-yaml/refs/heads/main/icons/NeurochainAI.png"
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
modelDisplayLabel: "Ollama"
```

Expand Down Expand Up @@ -59,8 +57,6 @@ endpoints:
fetch: false # pinned to the list above; set true to discover models from the server
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
modelDisplayLabel: "Ollama"
addParams:
"stop": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ endpoints:
fetch: false
titleConvo: true
titleModel: "llama-3-sonar-small-32k-chat"
summarize: false
summaryModel: "llama-3-sonar-small-32k-chat"
dropParams: ["stop", "frequency_penalty"]
modelDisplayLabel: "Perplexity"
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
modelDisplayLabel: "Portkey:OpenAI"
iconURL: https://images.crunchbase.com/image/upload/c_pad,f_auto,q_auto:eco,dpr_1/rjqy7ghvjoiu4cd1xjbf
```
Expand All @@ -57,8 +55,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
modelDisplayLabel: "Portkey:Llama"
iconURL: https://images.crunchbase.com/image/upload/c_pad,f_auto,q_auto:eco,dpr_1/rjqy7ghvjoiu4cd1xjbf
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "shuttle-2.5-mini"
summarize: false
summaryModel: "shuttle-2.5-mini"
modelDisplayLabel: "ShuttleAI"
dropParams: ["user", "stop"]
```
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,6 @@ endpoints:
fetch: false # fetching list of models is not supported
titleConvo: true
titleModel: "togethercomputer/llama-2-7b-chat"
summarize: false
summaryModel: "togethercomputer/llama-2-7b-chat"
modelDisplayLabel: "together.ai"
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,6 @@ endpoints:
fetch: true
titleConvo: true
titleModel: "current_model"
summarize: false
summaryModel: "current_model"
modelDisplayLabel: "TrueFoundry:OpenAI"
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ endpoints:
titleConvo: true
titleModel: "current_model"
titleMessageRole: "user"
summarize: false
summaryModel: "current_model"
```

## Notes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ endpoints:
custom:
- name: 'Vultr Cloud Inference'
apiKey: '${VULTRINFERENCE_TOKEN}'
baseURL: 'https://api.vultrinference.com/v1/chat/completions'
baseURL: 'https://api.vultrinference.com/v1'
models:
default: [
"llama2-7b-chat-Q5_K_M.gguf",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,6 @@ endpoints:
titleConvo: true
titleMethod: "completion"
titleModel: "grok-beta"
summarize: false
summaryModel: "grok-beta"
modelDisplayLabel: "Grok"
```

Expand Down
Loading
Loading