Obiguard provides a robust and secure platform to observe, govern, and manage your locally or privately hosted custom models using Triton.

Here’s the official Triton Inference Server documentation for more details.

​
Integrating Custom Models with Obiguard SDK

1

Expose your Triton Server

Expose your Triton server by using a tunneling service like ngrok or any other way you prefer. You can skip this step if you’re self-hosting the Gateway.

ngrok http 11434 --host-header="localhost:8080"
2

Install the Obiguard SDK

pip install obiguard
3

Initialize Obiguard with Triton custom URL

  1. Pass your publicly-exposed Triton server URL to Obiguard with customHost
  2. Set target provider as triton.
from obiguard import Obiguard

client = Obiguard(
    obiguard_api_key="sk-obg***",  # Your Obiguard API key
    provider="triton",
    custom_host="http://localhost:8000/v2/models/mymodel" # Your Triton Hosted URL
    Authorization="AUTH_KEY", # If you need to pass auth
)

More on custom_host here.

4

Invoke Chat Completions

Use the Obiguard SDK to invoke chat completions (generate) from your model, just as you would with any other provider:

completion = client.chat.completions.create(
    messages= [{ "role": 'user', "content": 'Say this is a test' }]
)

​
Next Steps

Explore the complete list of features supported in the SDK:

SDK