> ## Documentation Index
> Fetch the complete documentation index at: https://docs.obiguard.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# vLLM

> Integrate vLLM-hosted custom models with Obiguard and take them to production

Obiguard provides a robust and secure platform to observe, govern, and manage your **locally** or **privately** hosted custom models using vLLM.

<Info>
  Here's a [list](https://docs.vllm.ai/en/latest/models/supported_models.html) of all model architectures supported on vLLM.
</Info>

## Integrating Custom Models with Obiguard SDK

<Steps>
  <Step title="Expose your vLLM Server">
    Expose your vLLM server by using a tunneling service like [ngrok](https://ngrok.com/) or any other way you prefer. You can skip this step if you’re self-hosting the Gateway.

    ```sh theme={null}
    ngrok http 11434 --host-header="localhost:8080"
    ```
  </Step>

  <Step title="Install the Obiguard SDK">
    <Tabs>
      <Tab title="Python SDK">
        ```sh theme={null}
        pip install obiguard
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Initialize Obiguard with vLLM custom URL">
    1. Pass your publicly-exposed vLLM server URL to Obiguard with `customHost` (by default, vLLM is on `http://localhost:8000/v1`)
    2. Set target `provider` as `openai` since the server follows OpenAI API schema.

    <Tabs>
      <Tab title="Python SDK">
        ```python theme={null}
        from obiguard import Obiguard

        client = Obiguard(
            obiguard_api_key="sk-obg***",  # Your Obiguard API key
            provider="openai",
            custom_host="https://7cc4-3-235-157-146.ngrok-free.app" # Your vLLM ngrok URL
            Authorization="AUTH_KEY", # If you need to pass auth
        )
        ```
      </Tab>
    </Tabs>

    More on `custom_host` [here](/gateway/universal-api#integrating-local-or-private-models).
  </Step>

  <Step title="Invoke Chat Completions">
    Use the Obiguard SDK to invoke chat completions from your model, just as you would with any other provider:

    <Tabs>
      <Tab title="Python SDK">
        ```python theme={null}
        completion = client.chat.completions.create(
            messages= [{ "role": 'user', "content": 'Say this is a test' }]
        )

        print(completion)
        ```
      </Tab>
    </Tabs>
  </Step>
</Steps>

## [Using Virtual Keys](/virtual-keys)

Virtual Keys serve as Obiguard's unified authentication system for all LLM interactions, simplifying the use of multiple providers and Obiguard features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model.

1. Navigate to [Virtual Keys](/virtual-keys) in your Obiguard dashboard
2. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle
3. Configure your deployment:
   * Select the matching provider API specification (typically `OpenAI`)
   * Enter your model's base URL in the `Custom Host` field
   * Add required authentication headers and their values
4. Click **"Create"** to generate your virtual key

You can now use this virtual key in your requests:

<Tabs>
  <Tab title="Python SDK">
    ```python theme={null}
    client = Obiguard(
      obiguard_api_key="vk-obg***",  # Your Obiguard virtual key
      virtual_key="YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY"
    )

    response = client.chat.completions.create(
      model="your-self-hosted-model-name",
      messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"}
      ]
    )
    ```
  </Tab>
</Tabs>

For more information about managing self-hosted LLMs with Obigaurd, see [Bring Your Own LLM](/integrations/llms/byollm).

## Next Steps

Explore the complete list of features supported in the SDK:

<Card title="SDK" href="/api-reference/sdk/python" />