> ## Documentation Index
> Fetch the complete documentation index at: https://docs.obiguard.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Bring Your Own LLM

> Integrate your privately hosted LLMs with Obiguard for unified management, observability, and reliability.

Obiguard's Bring Your Own LLM feature allows you to seamlessly integrate privately hosted language models into your AI infrastructure.
This powerful capability enables unified management of both private and commercial LLMs through a consistent interface while
leveraging Obiguard's comprehensive suite of observability and reliability features.

## Key Benefits

* **Unified API Access**: Manage private and commercial LLMs through a single, consistent interface
* **Comprehensive Monitoring**: Track performance, usage, and costs alongside your commercial LLM usage
* **Simplified Access Control**: Manage team-specific permissions and usage limits
* **Secure Credential Management**: Protect sensitive authentication details through Obiguard's secure vault

## Integration Options

<Note>
  **Prerequisites**

  Your private LLM must implement an API specification compatible with one of Obiguard's [supported providers](/integrations/llms)
  (e.g., OpenAI's `/chat/completions`, Anthropic's `/messages`, etc.).
</Note>

Obiguard offers two primary methods to integrate your private LLMs:

1. **Using Virtual Keys**: Store your deployment details securely in Obiguard's vault
2. **Direct Integration**: Pass deployment details in your requests without storing them

### Option 1: Using Virtual Keys

#### Step 1: Add Your Deployment Details

Navigate to the [Virtual Keys](/virtual-keys) section in your Obiguard dashboard and create a new Virtual Key.

1. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle
2. Configure your deployment:
   * Select the matching provider API specification (typically `OpenAI`)
   * Enter your model's base URL in the `Custom Host` field
   * Add required authentication headers and their values
3. Click **"Create"** to generate your virtual key

#### Step 2: Use Your Virtual Key in Requests

After creating your virtual key, you can use it in your applications:

<Tabs>
  <Tab title="Python SDK">
    ```py Python theme={null}
    from obiguard import Obiguard

    client = Obiguard(
      obiguard_api_key="sk-obg***",  # Your Obiguard API key
      virtual_key="YOUR_PRIVATE_LLM_VIRTUAL_KEY"
    )

    response = client.chat.completions.create(
      model="YOUR_MODEL_NAME",  # The model name your private deployment expects
      messages=[
        {"role": "user", "content": "Explain quantum computing in simple terms"}
      ]
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="cURL">
    ```sh cURL theme={null}
    curl https://gateway.obiguard.ai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \
      -H "x-obiguard-virtual-key: $YOUR_PRIVATE_LLM_VIRTUAL_KEY" \
      -d '{
        "model": "YOUR_MODEL_NAME",
        "messages": [
          { "role": "user", "content": "Explain quantum computing in simple terms" }
        ]
      }'
    ```
  </Tab>
</Tabs>

### Option 2: Direct Integration Without Virtual Keys

If you prefer not to store your private LLM details in Obiguard's vault, you can pass them directly in your API requests:

<Tabs>
  <Tab title="Python SDK">
    ```py Python theme={null}
    from obiguard import Obiguard

    client = Obiguard(
      obiguard_api_key="sk-obg***",  # Your Obiguard API key
      provider="openai",  # The API spec your LLM implements
      custom_host="https://your-llm-server.com/v1/",  # Include the API version
      Authorization="Bearer YOUR_AUTH_TOKEN",  # Optional: Any auth headers needed
      forward_headers=["Authorization"]  # Headers to forward without processing
    )

    response = client.chat.completions.create(
      model="YOUR_MODEL_NAME",
      messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="cURL">
    ```sh cURL theme={null}
    curl https://gateway.obiguard.ai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \
      -H "x-obiguard-provider: openai" \
      -H "x-obiguard-custom-host: https://your-llm-server.com/v1" \
      -H "Authorization: Bearer YOUR_AUTH_TOKEN" \
      -H "x-obiguard-forward-headers: Authorization" \
      -d '{
        "model": "YOUR_MODEL_NAME",
        "messages": [
          { "role": "user", "content": "Explain quantum computing in simple terms" }
        ]
      }'
    ```
  </Tab>
</Tabs>

<Note>
  The `custom_host` must include the API version path (e.g., `/v1/`). Obiguard will automatically append the endpoint path (`/chat/completions`, `/completions`, or `/embeddings`).
</Note>

## Advanced Features

## Monitoring and Analytics

Obiguard provides comprehensive observability for your private LLM deployments, just like it does for commercial providers:

* **Log Analysis**: View detailed request and response logs
* **Performance Metrics**: Track latency, token usage, and error rates
* **User Attribution**: Associate requests with specific users via metadata

## Troubleshooting

| Issue                   | Possible Causes                                     | Solutions                                                                      |
| ----------------------- | --------------------------------------------------- | ------------------------------------------------------------------------------ |
| Connection Errors       | Incorrect URL, network issues, firewall rules       | Verify URL format, check network connectivity, confirm firewall allows traffic |
| Authentication Failures | Invalid credentials, incorrect header format        | Check credentials, ensure headers are correctly formatted and forwarded        |
| Timeout Errors          | LLM server overloaded, request too complex          | Adjust timeout settings, implement load balancing, simplify requests           |
| Inconsistent Responses  | Different model versions, configuration differences | Standardize model versions, document expected behavior differences             |

## FAQs

<AccordionGroup>
  <Accordion title="Can I use any private LLM with Obiguard?">
    Yes, provided it adheres to an API specification supported by Obiguard (e.g., OpenAI, Anthropic, etc.).
    The model must handle requests and responses in the expected format and be publicly accessible.
  </Accordion>

  <Accordion title="How do I handle multiple deployment endpoints?">
    Create separate virtual keys for each endpoint.
  </Accordion>

  <Accordion title="Are there any request volume limitations?">
    Obiguard itself doesn't impose specific request volume limitations for private LLMs.
    Your throughput will be limited only by your private LLM deployment's capabilities and any rate limits you configure in Obiguard.
  </Accordion>

  <Accordion title="Can I use different models with the same private deployment?">
    Yes, you can specify different model names in your requests as long as your private LLM deployment supports them. The model name is passed through to your deployment.
  </Accordion>

  <Accordion title="Can I mix private and commercial LLMs in the same application?">
    Absolutely! One of Obiguard's key benefits is the ability to manage both private and commercial LLMs through a unified interface.
  </Accordion>
</AccordionGroup>

## Next Steps

Explore these related resources to get the most out of your private LLM integration:

<CardGroup cols={2}>
  <Card title="Universal API" icon="globe" href="/gateway/universal-api" />

  <Card title="Request Tracing" icon="route" href="/observability/tracing" />
</CardGroup>