Bring Your Own LLM

Obiguard’s Bring Your Own LLM feature allows you to seamlessly integrate privately hosted language models into your AI infrastructure. This powerful capability enables unified management of both private and commercial LLMs through a consistent interface while leveraging Obiguard’s comprehensive suite of observability and reliability features.

Key Benefits

Unified API Access: Manage private and commercial LLMs through a single, consistent interface
Comprehensive Monitoring: Track performance, usage, and costs alongside your commercial LLM usage
Simplified Access Control: Manage team-specific permissions and usage limits
Secure Credential Management: Protect sensitive authentication details through Obiguard’s secure vault

Integration Options

PrerequisitesYour private LLM must implement an API specification compatible with one of Obiguard’s supported providers (e.g., OpenAI’s /chat/completions, Anthropic’s /messages, etc.).

Obiguard offers two primary methods to integrate your private LLMs:

Using Provider Credentials: Store your deployment details securely in Obiguard’s vault
Direct Integration: Pass deployment details in your requests without storing them

Option 1: Using Provider Credentials

Step 1: Add Your Deployment Details

Add your private deployment to Obiguard as a provider credential.

Enable the “Local/Privately hosted provider” option
Configure your deployment:
- Select the matching provider API specification (typically OpenAI)
- Enter your model’s base URL in the Custom Host field
- Add required authentication headers and their values
Create an access key for the agent or project that will call this deployment

Step 2: Use Your Access Key in Requests

After creating your access key, you can use it in your applications:

Python SDK
cURL

Python

from obiguard import Obiguard

client = Obiguard(
  obiguard_api_key="ak***",  # Your Obiguard access key
)

response = client.chat.completions.create(
  model="YOUR_MODEL_NAME",  # The model name your private deployment expects
  messages=[
    {"role": "user", "content": "Explain quantum computing in simple terms"}
  ]
)

print(response.choices[0].message.content)

cURL

curl https://gateway.obiguard.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \
  -d '{
    "model": "YOUR_MODEL_NAME",
    "messages": [
      { "role": "user", "content": "Explain quantum computing in simple terms" }
    ]
  }'

Option 2: Direct Integration Without Stored Provider Credentials

If you prefer not to store your private LLM details in Obiguard’s vault, you can pass them directly in your API requests:

Python SDK
cURL

Python

from obiguard import Obiguard

client = Obiguard(
  obiguard_api_key="ak***",  # Your Obiguard access key
  provider="openai",  # The API spec your LLM implements
  custom_host="https://your-llm-server.com/v1/",  # Include the API version
  Authorization="Bearer YOUR_AUTH_TOKEN",  # Optional: Any auth headers needed
  forward_headers=["Authorization"]  # Headers to forward without processing
)

response = client.chat.completions.create(
  model="YOUR_MODEL_NAME",
  messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
)

print(response.choices[0].message.content)

cURL

curl https://gateway.obiguard.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \
  -H "x-obiguard-provider: openai" \
  -H "x-obiguard-custom-host: https://your-llm-server.com/v1" \
  -H "Authorization: Bearer YOUR_AUTH_TOKEN" \
  -H "x-obiguard-forward-headers: Authorization" \
  -d '{
    "model": "YOUR_MODEL_NAME",
    "messages": [
      { "role": "user", "content": "Explain quantum computing in simple terms" }
    ]
  }'

The custom_host must include the API version path (e.g., /v1/). Obiguard will automatically append the endpoint path (/chat/completions, /completions, or /embeddings).

Advanced Features

Monitoring and Analytics

Obiguard provides comprehensive observability for your private LLM deployments, just like it does for commercial providers:

Log Analysis: View detailed request and response logs
Performance Metrics: Track latency, token usage, and error rates
User Attribution: Associate requests with specific users via metadata

Troubleshooting

Issue	Possible Causes	Solutions
Connection Errors	Incorrect URL, network issues, firewall rules	Verify URL format, check network connectivity, confirm firewall allows traffic
Authentication Failures	Invalid credentials, incorrect header format	Check credentials, ensure headers are correctly formatted and forwarded
Timeout Errors	LLM server overloaded, request too complex	Adjust timeout settings, implement load balancing, simplify requests
Inconsistent Responses	Different model versions, configuration differences	Standardize model versions, document expected behavior differences

FAQs

Can I use any private LLM with Obiguard?

Yes, provided it adheres to an API specification supported by Obiguard (e.g., OpenAI, Anthropic, etc.). The model must handle requests and responses in the expected format and be publicly accessible.

How do I handle multiple deployment endpoints?

Add a separate provider credential for each endpoint.

Are there any request volume limitations?

Obiguard itself doesn’t impose specific request volume limitations for private LLMs. Your throughput will be limited only by your private LLM deployment’s capabilities and any rate limits you configure in Obiguard.

Can I use different models with the same private deployment?

Yes, you can specify different model names in your requests as long as your private LLM deployment supports them. The model name is passed through to your deployment.

Can I mix private and commercial LLMs in the same application?

Absolutely! One of Obiguard’s key benefits is the ability to manage both private and commercial LLMs through a unified interface.

Next Steps

Explore these related resources to get the most out of your private LLM integration:

Ecosystem

AI Apps

Agents

LLMs

Vector Databases

Bring Your Own LLM

Key Benefits

Integration Options

Option 1: Using Provider Credentials

Step 1: Add Your Deployment Details

Step 2: Use Your Access Key in Requests

Option 2: Direct Integration Without Stored Provider Credentials

Advanced Features

Monitoring and Analytics

Troubleshooting

FAQs

Next Steps

Universal API

Request Tracing