vLLM

Obiguard provides a robust and secure platform to observe, govern, and manage your locally or privately hosted custom models using vLLM.

Here’s a list of all model architectures supported on vLLM.

Integrating Custom Models with Obiguard SDK

Expose your vLLM Server

Expose your vLLM server by using a tunneling service like ngrok or any other way you prefer. You can skip this step if you’re self-hosting the Gateway.

ngrok http 11434 --host-header="localhost:8080"

Install the Obiguard SDK

Python SDK

pip install obiguard

Initialize Obiguard with vLLM custom URL

Pass your publicly-exposed vLLM server URL to Obiguard with customHost (by default, vLLM is on http://localhost:8000/v1)
Set target provider as openai since the server follows OpenAI API schema.

Python SDK

from obiguard import Obiguard

client = Obiguard(
    obiguard_api_key="ak***",  # Your Obiguard access key
    provider="openai",
    custom_host="https://7cc4-3-235-157-146.ngrok-free.app" # Your vLLM ngrok URL
    Authorization="AUTH_KEY", # If you need to pass auth
)

Using Provider Credentials and Access Keys

Obiguard stores the credentials for your provider in its vault as a provider credential, and your application authenticates to the gateway with an access key. This simplifies the use of multiple providers and Obiguard features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model.

Add your self-hosted model to Obiguard as a provider credential
Configure your deployment:
- Select the matching provider API specification (typically OpenAI)
- Enter your model’s base URL as the custom host
- Add required authentication headers and their values
Create an access key for the agent or project that will call it

You can now use the access key in your requests:

Python SDK

client = Obiguard(
  obiguard_api_key="ak***",  # Your Obiguard access key
)

response = client.chat.completions.create(
  model="your-self-hosted-model-name",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

For more information about managing self-hosted LLMs with Obigaurd, see Bring Your Own LLM.

Next Steps

Explore the complete list of features supported in the SDK:

Ecosystem

AI Apps

Agents

LLMs

Vector Databases

Integrating Custom Models with Obiguard SDK

Using Provider Credentials and Access Keys

Next Steps

SDK

​Integrating Custom Models with Obiguard SDK

​Using Provider Credentials and Access Keys

​Next Steps

SDK

Integrating Custom Models with Obiguard SDK

Using Provider Credentials and Access Keys

Next Steps