# Create Speech Source: https://docs.obiguard.ai/api-reference/inference-api/audio/create-speech post /audio/speech # Create Transcription Source: https://docs.obiguard.ai/api-reference/inference-api/audio/create-transcription post /audio/transcriptions # Create Translation Source: https://docs.obiguard.ai/api-reference/inference-api/audio/create-translation post /audio/translations # Authentication Source: https://docs.obiguard.ai/api-reference/inference-api/authentication To ensure secure access to Obiguard's APIs, authentication is required for all requests. This guide provides the necessary steps to authenticate your requests using the Obiguard API key, regardless of whether you are using the SDKs for Python, the OpenAI SDK, or making REST API calls directly. ## Obtaining Your API Key [Create](https://app.obiguard.ai/sign-up) or [log in](https://app.obiguard.ai/sign-in) to your Obiguard account. Grab your account's API key from a guardrail policy, or create a virtual key. ## Authentication with SDKs ### Obiguard SDKs ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="**-obg***", # Your Obiguard API or virtual key ) chat_completion = client.chat.completions.create( messages=[{"role": "user", "content": "Say this is a test"}], model='gpt-4o' ) print(chat_completion.choices[0].message["content"]) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $Obiguard_API_KEY" \ -d '{ "model": "gpt-4o", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "Hello!" } ] }' ``` ### OpenAI SDK When integrating Obiguard through the OpenAI SDK, modify the base URL and add the `x-obiguard-api-key` header for authentication. Here's an example of how to do it: We use the `createHeaders` helper function from the Obiguard SDK here to easily create Obiguard headers. You can pass the raw headers (`x-obiguard-api-key`, `x-obiguard-provider`) directly in the `defaultHeaders` param as well. ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai_client = OpenAI( base_url=Obiguard_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key provider="openai" ) ) response = openai_client.chat.completions.create( messages=[{'role': 'user', 'content': 'Say this is a test'}], model='gpt-4bo' ) ``` Read more [here](/integrations/llms/openai). # Chat Source: https://docs.obiguard.ai/api-reference/inference-api/chat post /chat/completions # Completions Source: https://docs.obiguard.ai/api-reference/inference-api/completions post /completions # Embeddings Source: https://docs.obiguard.ai/api-reference/inference-api/embeddings post /embeddings # Errors Source: https://docs.obiguard.ai/api-reference/inference-api/error-codes Obiguard uses conventional response codes to indicate the success or failure of an API request. In general: Codes in the `2xx` range indicate success. Codes in the `4xx` range indicate an error that failed given the information provided. Codes in the `5xx` range indicate an error with Obiguard’s servers (these are rare). ## Common Errors During request processing, you may encounter error codes from either Obiguard or the LLM provider: | Status Code | Description | Source | | ----------- | -------------------------------------------- | -------------------- | | `408` | Request timed out | Obiguard OR Provider | | `412` | Budget exhausted | Obiguard OR Provider | | `429` | Request rate limited | Obiguard OR Provider | | `446` | Guardrail checks failed (request denied) | Obiguard | | `246` | Guardrail checks failed (request successful) | Obiguard | Provider-specific error codes are passed through by Obiguard. # Gateway to Other APIs Source: https://docs.obiguard.ai/api-reference/inference-api/gateway-for-other-apis Access any custom provider endpoint through Obiguard API Obiguard API has first-class support for monitoring and routing your requests to 10+ provider endpoints, like `/chat/completions`, `/audio`, `/embeddings`, etc. We also make these endpoints work across 250+ different LLMs. **However**, there are still many endpoints like Cohere's `/rerank` or Deepgram's `/listen` that are uncommon or have niche use cases. With the **Gateway to Other APIs** feature, you can route to any custom provider endpoint using Obiguard (including the ones hosted on your private setups) and get **complete logging & monitoring** for all your requests. ## Supported HTTP Methods Both the REST API and Obiguard SDKs (Python, NodeJS) support all of these HTTP methods. # How to Integrate 1. Get your Obiguard API key 2. Add your provider details to Obiguard 3. Make your request using Obiguard's API or SDK ### 1. Get Obiguard API Key Create or log in to your Obiguard account. Grab your account's API key from the ["Virtual Keys" page](https://app.obiguard.ai). ### 2. Add Provider Details Choose one of these authentication methods: Obiguard integrates with 40+ LLM providers. Add your provider credentials (such as API key) to Obiguard, and get a virtual key that you can use to authenticate and send your requests. ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) ``` ```sh cURL theme={null} curl https://gateway.obiguard.ai/v1/rerank \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" ``` Creating virtual keys lets you: * Manage all credentials in one place * Rotate between different provider keys Set the provider name from one of Obiguard's 40+ supported providers list and use your provider credentials directly with each request. ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) ``` ```sh cURL theme={null} curl https://gateway.obiguard.ai/v1/rerank \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: cohere" \ -H "Authorization: Bearer $COHERE_API_KEY" \ ``` Route to your privately hosted model endpoints. * Choose a compatible provider type (e.g., `openai`, `cohere`) * Provide your endpoint URL with `customHost` * Include `Authentication` if needed ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "cohere", custom_host = "https://182.145.24.5:8080/v1", Authorization = "Bearer COHERE_API_KEY" ) ``` ```sh cURL theme={null} curl https://gateway.obiguard.ai/v1/rerank \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: cohere" \ -H "x-obiguard-custom-host: https://182.145.24.5:8080/v1" \ -H "Authorization: Bearer $COHERE_API_KEY" \ ``` ### 3. Make Requests Construct your request URL: 1. Obiguard Gateway base URL remains same: `https://gateway.obiguard.ai/v1` 2. Append your custom endpoint at the end of the URL: `https://gateway.obiguard.ai/v1/{provider - endpoint}` ```bash POST theme={null} curl --request POST \ --url https://gateway.obiguard.ai/v1/rerank \ -H 'Content-Type: application/json' \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ --data '{ "model": "rerank-english-v2.0", "query": "What is machine learning?", "documents": [ "Machine learning is a branch of AI focused on building systems that learn from data.", "Data science involves analyzing and interpreting complex data sets." ] }' ``` ```bash GET theme={null} curl --request GET \ --url https://gateway.obiguard.ai/v1/collections \ -H 'Content-Type: application/json' \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" ``` ```bash PUT theme={null} curl --request PUT \ --url https://gateway.obiguard.ai/v1/collections/my-collection \ -H 'Content-Type: application/json' \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ --data '{ "metadata": { "description": "Updated collection description" } }' ``` ```bash DELETE theme={null} curl --request DELETE \ --url https://gateway.obiguard.ai/v1/collections/my-collection \ -H 'Content-Type: application/json' \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" ``` The SDK fully supports all HTTP methods: `POST`, `GET`, `PUT`, and `DELETE`. ```python POST theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) response = client.post( '/rerank', model="rerank-english-v2.0", query="What is machine learning?", documents=[ "Machine learning is a branch of AI focused on building systems that learn from data.", "Data science involves analyzing and interpreting complex data sets." ] ) ``` ```python GET theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) response = client.get('/collections') ``` ```python PUT theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) response = client.put( '/collections/my-collection', metadata={ "description": "Updated collection description" } ) ``` ```python DELETE theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) response = client.delete('/collections/my-collection') ``` ## End-to-end Example A complete example showing document reranking with Cohere: ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) response = client.post( '/rerank', return_documents=False, max_chunks_per_doc=10, model="rerank-english-v2.0", query="What is the capital of the United States?", documents=[ "Carson City is the capital city of the American state of Nevada.", "Washington, D.C. is the capital of the United States.", "Capital punishment has existed in the United States since before its founding." ] ) ``` # Caveats & Considerations * Response objects are returned exactly as received from the provider, without Obiguard transformations * REST API supports all HTTP methods: `POST`, `GET`, `PUT`, and `DELETE` * SDK supports all HTTP methods: `POST`, `GET`, `PUT`, and `DELETE` * There are no limitations on which provider endpoints can be proxied * All requests are logged and monitored through your Obiguard dashboard ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Headers Source: https://docs.obiguard.ai/api-reference/inference-api/headers Header requirements and options for the Obiguard API Obiguard API accepts 4 kinds of headers for your requests: | | | | | :-------------------------------------------------------- | :--------- | :------------------------------------ | | Obiguard Authentication Header | `Required` | For Obiguard auth | | Provider Authentication Headers OR Cloud-Specific Headers | `Required` | For provider auth | | Custom Headers | `Optional` | To forward any other headers directly | ## Obiguard Authentication ### Obiguard API Key Authenticate your requests with your Obiguard API key. Obtain API key from the [Obiguard dashboard](https://app.obiguard.ai/api-keys).
Environment variable: `OBIGUARD_API_KEY` ```py Python {4} theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key = "OBIGUARD_API_KEY" # defaults to os.environ.get("OBIGUARD_API_KEY") ) ``` ```sh cURL {2} theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ ``` ## Provider Authentication In addition to the Obiguard API key, you must provide information about the AI provider you're using. There are **4** ways to do this: ### 1. Provider Slug + Auth Useful if you do not want to save your API keys to Obiguard vault and make direct requests. Specifies the provider you're using (e.g., `openai`, `anthropic`, `vertex-ai`).
List of [Obiguard supported providers here](/integrations/llms). Pass the auth details for the specified provider as a `"Bearer $TOKEN"`.

If your provider expects their auth with headers such as `x-api-key` or `api-key`, you can pass the token with the `Authorization` header directly and Obiguard will convert it into the provider-specific format. ```sh cURL {3,4} theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ ``` ```py Python {5,6} theme={null} from obiguard import Obiguard client = Obiguard( api_key = "OBIGUARD_API_KEY", # defaults to os.environ.get("OBIGUARD_API_KEY") provider = "openai", Authorization = "Bearer OPENAI_API_KEY" ) ``` ### 2. Virtual Key Save your provider auth on Obiguard and use a virtual key to directly make a call. This value is pass in the same field as Obiguard API key. [Docs](/virtual-keys) ```py Python {4} theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ```sh cURL {2} theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ ``` ### 4. Custom Host Specifies the base URL where you want to send your request Target provider that's availabe on your base URL. If you are unsure of which target provider to set, you can set `openai`. Pass the auth details for the specified provider as a `"Bearer $TOKEN"`.

If your provider expects their auth with headers such as `x-api-key` or `api-key`, you can pass the token with the `Authorization` header directly and Obiguard will convert it into the provider-specific format. ```py Python {5-7} theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key custom_host = "http://124.124.124.124/v1", provider = "openai", Authorization = "Bearer TOKEN" ) ``` ```sh cURL {3-5} theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-custom-host: http://124.124.124.124/v1" \ -H "x-obiguard-provider: openai" \ -H "Authorization: Bearer $TOKEN" \ ``` ## Custom Headers You can pass any other headers your API expects by directly forwarding them without any processing by Obiguard.
This is especially useful if you want to pass send sensitive headers. ### Forward Headers Pass all the headers you want to forward directly in this array. ([Docs](/integrations/byollm#forward-sensitive-headers-securely)) ```sh cURL {4-6} theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "X-Custom-Header: ...."\ -H "Another-Header: ....."\ -H "x-obiguard-forward-headers: ['X-Custom-Header', 'Another-Header']" \ ``` ```py Python {6} theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key = "OBIGUARD_API_KEY", # defaults to os.environ.get("OBIGUARD_API_KEY") X_Custom_Header = "....", Another_Header = "....", forward_headers = ['X_Custom_Header', 'Another_Header'] ) ``` #### Python Usage With the Python SDK, you need to transform your headers to **Snake Case** and then include them while initializing the Obiguard client. Example: If you have a header of the format `X-My-Custom-Header`, it should be sent as `X_My_Custom_Header` in the SDK #### JavaScript Usage With the JS SDK, you need to transform your headers to **Camel Case** and then include them while initializing the Obiguard client. Example: If you have a header of the format `X-My-Custom-Header`, it should be sent as `xMyCustomHeader` in the SDK ## Cloud-Specific Headers (`Azure`, `Google`, `AWS`) Pass more configuration headers for `Azure OpenAI`, `Google Vertex AI`, or `AWS Bedrock` ### Azure * `x-obiguard-azure-resource-name`, `x-obiguard-azure-deployment-id`, `x-obiguard-azure-api-version`, `Authorization`, `x-obiguard-azure-model-name` ### Google Vertex AI * `x-obiguard-vertex-project-id`, `x-obiguard-vertex-region`, `X-Vertex-AI-LLM-Request-Type` ### AWS Bedrock * `x-obiguard-aws-session-token`, `x-obiguard-aws-secret-access-key`, `x-obiguard-aws-region`, `x-obiguard-aws-session-token` *** ## List of All Headers The following is a comprehensive list of headers that can be used when initializing the Obiguard client. Obiguard adheres to language-specific naming conventions: * **camelCase** for **JavaScript/Node.js** parameters * **snake\_case** for **Python** parameters * **hyphenated-keys** for **HTTP headers** | Parameter | Type | Key | | :---------------------------------------------------------------------------------------------------------------------- | :-------------- | :--------------------------------------------------------------------------------- | | **API Key** Your Obiguard account's API Key. | stringrequired | `obiguard_api_key` | | **Provider** The AI provider to use for your calls. ([supported providers](/integrations/llms#supported-ai-providers)). | string | `provider` | | **Base URL** You can edit the URL of the gateway to use. | string | `base_url` | | **Custom Host** Route to locally or privately hosted model by configuring the API URL with custom host | string | `custom_host` | | **Forward Headers** Forward sensitive headers directly to your model's API without any processing from Obiguard. | array of string | `forward_headers` | | **Azure OpenAI Headers** Configuration headers for Azure OpenAI that you can send separately | string | `azure_resource_name` `azure_deployment_id` `azure_api_version` `azure_model_name` | | **Google Vertex AI Headers** Configuration headers for Vertex AI that you can send separately | string | `vertex_project_id` `vertex_region` | | **AWS Bedrock Headers** Configuration headers for Bedrock that you can send separately | string | `aws_access_key_id` `aws_secret_access_key` `aws_region` `aws_session_token` | | Parameter | Type | Header Key | | :---------------------------------------------------------------------------------------------------------------------- | :-------------- | :-------------------------------------------------------------------------------------------------------------------------------- | | **API Key** Your Obiguard account's API Key. | stringrequired | `x-obiguard-api-key` | | **Provider** The AI provider to use for your calls. ([supported providers](/integrations/llms#supported-ai-providers)). | string | `x-obiguard-provider` | | **Base URL** You can edit the URL of the gateway to use. | string | Change the request URL | | **Forward Headers** Forward sensitive headers directly to your model's API without any processing from Obiguard. | array of string | `x-obiguard-forward-headers` | | **Azure OpenAI Headers** Configuration headers for Azure OpenAI that you can send separately | string | `x-obiguard-azure-resource-name`, `x-obiguard-azure-deployment-id`, `x-obiguard-azure-api-version`, `x-obiguard-azure-model-name` | | **Google Vertex AI Headers** Configuration headers for Vertex AI that you can send separately | string | `x-obiguard-vertex-project-id`, `x-obiguard-vertex-region` | | **AWS Bedrock Headers** Configuration headers for Bedrock that you can send separately | string | `x-obiguard-aws-session-token`, `x-obiguard-aws-secret-access-key`, `x-obiguard-aws-region`, `x-obiguard-aws-session-token` | ### Using Headers You can send these headers in multiple ways: ```python Obiguard SDK theme={null} # Python - At initialization from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) # Python - At runtime completion = client.with_options( trace_id="your_trace_id", metadata={"_user": "user_12345"} ).chat.completions.create( messages=[{"role": "user", "content": "Hello!"}], model="gpt-4o" ) ``` ```python OpenAI SDK theme={null} # Python from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders # At initialization client = OpenAI( api_key="OPENAI_API_KEY", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders({ obiguard_api_key="vk-obg***", # Your Obiguard virtual key }) ) # At runtime response = client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Hello!"}] ) ``` ```sh REST API theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -d '{ "model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}] }' ``` # Create Image Source: https://docs.obiguard.ai/api-reference/inference-api/images/create-image post /images/generations # Create Image Edit Source: https://docs.obiguard.ai/api-reference/inference-api/images/create-image-edit post /images/edits # Create Image Variation Source: https://docs.obiguard.ai/api-reference/inference-api/images/create-image-variation post /images/variations # Introduction Source: https://docs.obiguard.ai/api-reference/inference-api/introduction This documentation provides detailed information about the various ways you can access and interact with Obiguard - **a robust AI gateway** designed to simplify and enhance your experience with Large Language Models (LLMs) like OpenAI's GPT models. Whether you're integrating directly with OpenAI, using a framework like Langchain or LlamaIndex, or building standalone applications, Obiguard offers a flexible, secure, and efficient way to manage and deploy AI-powered features. ## 3 Ways to Integrate Obiguard Obiguard can be accessed through three primary methods, each catering to different use cases and integration requirements: ### 1. Obiguard SDKs (Python) **Ideal for:** standalone applications or when you're not already using the OpenAI integration. The SDKs are also highly recommended for seamless integration with frameworks like Langchain and LlamaIndex. The Obiguard SDKs are available in Python and JavaScript, designed to provide a streamlined, code-first approach to integrating LLMs into your applications. #### Installing the SDK Choose the SDK that matches your development environment: ```sh theme={null} pip install obiguard ``` #### Usage Once installed, you can use the SDK to make calls to LLMs, manage prompts, handle keys, and more, all through a simple and intuitive API. ### 2. OpenAI SDK through the Obiguard Gateway **Ideal for:** if you're currently utilizing OpenAI's Python or Node.js SDKs. By changing the base URL and adding Obiguard-specific headers, you can quickly integrate Obiguard's features into your existing setup. Learn more [here](/integrations/llms/openai). ### 3. REST API **Ideal for:** applications that prefer RESTful services. The base URL for all REST API requests is `https://gatewawy.Obiguard.ai/v1`, with an [authentication](/api-reference/inference-api/authentication) header. Learn more [here](/api-reference/inference-api/chat). # Supported Providers Source: https://docs.obiguard.ai/api-reference/inference-api/supported-providers | Provider | Chat | Chat - Vision | Embeddings | Images | Audio | Fine-tuning | Batch | Files | Moderations | Assistants | Completions | Obiguard Prompts | Chat - Tools | | :--------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | :--------------------------- | | AI21 | | | | | | | | | | | | | | | Anthropic | | | | | | | | | | | | | | | Azure OpenAI | | | | | | | | | | | | | | | AWS Bedrock | | | | | | | | | | | | | | | Cohere | | | | | | | | | | | | | | | Google Vertex AI | | | | | | | | | | | | | | | Google Gemini | | | | | | | | | | | | | | | Groq | | | | | | | | | | | | | | | Mistral AI | | | | | | | | | | | | | | | Nomic AI | | | | | | | | | | | | | | | Ollama | | | | | | | | | | | | | | | OpenAI | | | | | | | | | | | | | | | Openrouter | | | | | | | | | | | | | | | Perplexity AI | | | | | | | | | | | | | | | Stability AI | | | | | | | | | | | | | | | Together AI | | | | | | | | | | | | | | # Supported SDKs Source: https://docs.obiguard.ai/api-reference/sdk/list Find the best way to use Obiguard in your preferred language. Set up your development environment to use Obiguard in Python, or any OpenAI-compatible SDK. ## Official SDKs Easy, modern integration for Python apps. ## Compatible & Community Libraries You can use Obiguard with many OpenAI-compatible and community SDKs. Here are some popular options: Use Obiguard as a drop-in for OpenAI methods. ## FAQs [Email us](mailto:support@obiguard.com). Yes! Any OpenAI-compatible SDK or HTTP client can be used with Obiguard by simply pointing to the Obiguard API endpoint and using your Obiguard API key. Check out our exhaustive list of integrations [here](/integrations/ecosystem). # Obiguard Python SDK Source: https://docs.obiguard.ai/api-reference/sdk/python Official Obiguard Python SDK to help take your AI apps to production The official Python SDK makes it easy to integrate Obiguard into any Python application. Enjoy unified access to 250+ LLMs, advanced observability, routing, governance, and enterprise features with just a few lines of code. Official ![](https://img.shields.io/pypi/v/portkey.svg) ## Installation Install the Obiguard SDK from PyPI: ```bash theme={null} pip install obiguard ``` ## API Key Setup 1. [Create a Obiguard API key](https://app.obiguard.ai) in your dashboard. 2. Store your API key securely as an environment variable: ```sh macOS/Linux theme={null} export OBIGUARD_API_KEY="your_api_key_here" ``` ```sh Windows(PowerShell) theme={null} setx OBIGUARD_API_KEY "your_api_key_here" ``` The SDK automatically detects your API key from the environment. ## Quickstart Here’s a minimal example to get you started: ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API or virtual key ) response = client.chat.completions.create( messages=[{"role": "user", "content": "Hello, world!"}], model="gpt-4o" # Example provider/model ) ``` You can use either a Virtual Key or a Config object to select your AI provider. Find more info on different authentication mechanisms [here](/api-reference/inference-api/headers#provider-authentication). ## Authentication & Configuration The SDK requires: * **Obiguard API Key**: Your Obiguard API key (env var `OBIGUARD_API_KEY` recommended) * **Provider Authentication**: * **Virtual Key**: The [Virtual Key](/virtual-keys) of your chosen AI provider * **Provider Slug + Auth Headers**: Useful if you do not want to save your API keys to Obiguard and make direct requests. ```python theme={null} # With Virtual Key client = Obiguard(obiguard_api_key="...") # With Provider Slug + Auth Headers client = Obiguard(obiguard_api_key="...", provider="openai", Authorization = "Bearer OPENAI_API_KEY") ``` ## Async Usage Obiguard supports **Async** usage - just use `AsyncObiguard` client instead of `Obiguard` with `await`: ```py Python theme={null} import asyncio from obiguard import AsyncObiguard client = AsyncObiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) async def main(): chat_completion = await client.chat.completions.create( messages=[{'role': 'user', 'content': 'Say this is a test'}], model='gpt-4' ) print(chat_completion) asyncio.run(main()) ``` ## Using a Custom httpx Client If you need to customize HTTP networking—for example, to disable SSL verification due to VPNs like Zscaler or to use custom proxies—you can pass your own `httpx.Client` to the Obiguard SDK. Disabling SSL certificate verification is insecure and should only be used for debugging or in trusted internal environments. Never use this in production. **Example: Disable SSL Verification** ```python theme={null} import httpx from obiguard import Obiguard # Create an httpx client with SSL verification disabled custom_client = httpx.Client(verify=False) client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key http_client=custom_client ) response = client.chat.completions.create( messages=[{"role": "user", "content": "Hello!"}], model="gpt-4o" ) print(response) ``` * You can use any `httpx.Client` options (e.g., for proxies, timeouts, custom headers). * For async usage, pass an `httpx.AsyncClient` to `AsyncObiguard`. * See [OpenAI Python SDK: Configuring the HTTP client](https://github.com/openai/openai-python#configuring-the-http-client) for more examples and best practices. ## Parameters View the complete list of headers that can be used with Obiguard API requests, including authentication, configuration, and custom headers. Here's how you can use these headers with the Python SDK: ```python theme={null} portkey = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key # Add any other headers from the reference ) # Or at runtime completion = portkey.with_options( metadata={"_user": "user_id"}, # Add any other headers as needed ).chat.completions.create( messages=[{"role": "user", "content": "Hello!"}], model="gpt-4o" ) ``` ## Troubleshooting & Support * Having trouble? [Email support](mailto:support@obiguard.com). ## FAQs Yes! Obiguard’s Python SDK is OpenAI-compatible. You can also use any OpenAI-compatible library by pointing it to the Obiguard API endpoint and using your Obiguard API key. If you are behind a VPN (like Zscaler) or a corporate proxy and see SSL/certificate errors, you can pass a custom `httpx.Client` to the Obiguard SDK with SSL verification disabled. See the docs above for an example. **Warning:** Disabling SSL verification is insecure—only use this as a last resort or for debugging. Check out our integration docs [here](/integrations/ecosystem). # Obiguard Features Source: https://docs.obiguard.ai/features-overview Explore the powerful features of Obiguard. ## AI Gateway Connect to 250+ AI models using a single consistent API. Set up load balancers, automated fallbacks, caching, conditional routing, and more, seamlessly. Use any of the supported models with a universal API (REST and SDKs) Leverage OpenAI’s Realtime API for insights Connect to Remote MCP severs, allowing you to connect external tools and data sources. Ensure responses strictly adhere to the OpenAI specification Use vision, audio, image generation, and more models Manage provider access with virtual API keys A collection of guardrail policies that you can oversee ## Observability & Logs Gain real-time insights, track key metrics, and streamline debugging with our OpenTelemetry-compliant system. Access and analyze detailed logs Implement distributed tracing for request flows Gain insights through comprehensive analytics ## Guardrails Enforce Real-Time LLM Behavior with 50+ state-of-the-art AI guardrails, so that you can synchronously run guardrails on your requests. Implement rule-based safety checks Leverage AI for advanced content filtering ## Agents Natively integrate Obiguard's gateway, guardrails, and observability suite with leading agent frameworks and take them to production. ## More Resources Have questions or need help? Contact the Obiguard team for support. Common errors encountered when using the Obiguard Gateway and their resolutions. # AI Gateway Source: https://docs.obiguard.ai/gateway/AI-gateway A unified interface to interact with 250+ AI models. ## Features Use any of the supported models with a universal API (REST and SDKs) Leverage OpenAI’s Realtime API for insights Connect to Remote MCP severs, allowing you to connect external tools and data sources. Ensure responses strictly adhere to the OpenAI specification Use vision, audio, image generation, and more models # Function Calling Source: https://docs.obiguard.ai/gateway/multimodal-capabilities/function-calling Obiguard’s AI Gateway enables function calling, allowing you to define functions in API requests. Supported models can respond with either plain text or a function name and its parameters. ## Using Functions Obiguard allows you to define functions in API requests using the OpenAI function signature. The `tools` parameter can include functions and is supported by models that enable function or tool calling. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key='vk-obg***', # Your Obiguard virtual key here ) tools = [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA", }, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, }, "required": ["location"], }, } } messages = [{"role": "user", "content": "What's the weather like in Boston today?"}] completion = client.chat.completions.create( model="gpt-3.5-turbo", messages=messages, tools=tools, tool_choice="auto" ) print(completion) ``` ## Managing Functions and Tools in Prompts With Obiguard’s Prompt Library, you can create prompt templates that include function or tool definitions and specify the tool choice parameter. Obiguard also validates your tool definitions in real time to help prevent syntax errors. ## Supported Providers and Models Function calling is supported by the following providers, with more being added regularly. | Provider | Supported Models | | ------------------------- | ----------------------------------------------------------------------------------------------------------------- | | OpenAI | gpt-4 series, gpt-3.5-turbo series | | Azure OpenAI | gpt-4 series, gpt-3.5-turbo series | | Anyscale | mistralai/Mistral-7B-Instruct-v0.1, mistralai/Mixtral-8x7B-Instruct-v0.1 | | Together AI | mistralai/Mixtral-8x7B-Instruct-v0.1, mistralai/Mistral-7B-Instruct-v0.1, togethercomputer/CodeLlama-34b-Instruct | | Fireworks AI | firefunction-v1, fw-function-call-34b-v0 | | Google Gemini / Vertex AI | gemini-1.0-pro, gemini-1.0-pro-001, gemini-1.5-pro-latest | # Image Generation Source: https://docs.obiguard.ai/gateway/multimodal-capabilities/image-generation Obiguard AI gateway enables image generation features provided by various foundational model vendors. A typical scenario is text-to-image generation, where a user provides a prompt that the image model interprets to produce an image. You can find the vision models [guide here](vision). ### Text-to-Image Generation Obiguard allows you to make text-to-image requests using the OpenAI-compatible API signature. ```python theme={null} from obiguard import Obiguard client = Obiguard( provider='openai', base_url='https://gateway.obiguard.ai/v1', obiguard_api_key='vk-obg***', # Your Obiguard virtual key here ) response = client.images.generate( prompt="A beautiful sunset over the mountains", n=1, size="1024x1024" ) ``` ```python theme={null} from openai import OpenAI from obiguard import createHeaders client = OpenAI( base_url='https://gateway.obiguard.ai/v1', api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="vk-obg***", # Your Obiguard virtual key here ) ) response = client.images.generate( prompt="A beautiful sunset over the mountains", n=1, size="1024x1024" ) ``` ```curl theme={null} curl "https://gateway.obiguard.ai/v1/images/generations" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_VIRTUAL_KEY" \ -d '{ "model": "dall-e-3", "prompt": "Lucy in the sky with diamonds" }' ``` # Speech-to-Text Source: https://docs.obiguard.ai/gateway/multimodal-capabilities/speech-to-text Obiguard's AI gateway enables speech-to-text using models such as OpenAI Whisper. ## Using Transcription and Translation Obiguard provides both `Transcription` and `Translation` options for speech-to-text (STT) models, following the OpenAI API format. You can submit audio files in formats such as `flac`, `mp3`, `mp4`, `mpeg`, `mpga`, `m4a`, `ogg`, `wav`, or `webm` as part of your API request. Example: ``` from pathlib import Path from obiguard import Obiguard # Initialize the Obiguard client client = Obiguard( virtual_key="vk-obg***" # Your Obiguard virtual key here ) audio_file= open("/path/to/file.mp3", "rb") # Transcription transcription = client.audio.transcriptions.create( model="whisper-1", file=audio_file ) print(transcription.text) # Translation translation = client.audio.translations.create( model="whisper-1", file=audio_file ) print(translation.text) ``` ``` from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="dummy" #We are using Virtual Key from Obiguard base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( api_key="vk-obg***", # Your Obiguard virtual key here ) ) audio_file= open("/path/to/file.mp3", "rb") # Transcription transcription = client.audio.transcriptions.create( model="whisper-1", file=audio_file ) print(transcription.text) # Translation translation = client.audio.translations.create( model="whisper-1", file=audio_file ) print(translation.text) ``` For Transcriptions: ``` curl "https://gateway.obiguard.ai/v1/audio/transcriptions" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H 'Content-Type: multipart/form-data' \ --form file=@/path/to/file/audio.mp3 \ --form model=whisper-1 ``` For Translations: ``` curl "https://gateway.obiguard.ai/v1/audio/translations" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H 'Content-Type: multipart/form-data' \ --form file=@/path/to/file/audio.mp3 \ --form model=whisper-1 ``` After completion, your request will appear in the logs UI, displaying the transcribed or translated text, as well as the associated cost and latency. ## Supported Providers and Models Currently, the following providers are supported for speech-to-text. More providers will be added soon. | Provider | Model | Functions | | -------- | --------- | -------------------------- | | OpenAI | whisper-1 | Transcription, Translation | # Text-to-Speech Source: https://docs.obiguard.ai/gateway/multimodal-capabilities/text-to-speech Obiguard’s AI gateway offers text-to-speech support for both `OpenAI` and `Azure OpenAI` models. ## Usage The API follows the OpenAI format, allowing you to send input text and specify a voice option in your request. Supported output formats include `mp3`, `opus`, `aac`, `flac`, and `pcm`. Obiguard also enables real-time audio streaming for TTS models. Example: ```python theme={null} from obiguard import Obiguard client = Obiguard( provider='openai', obiguard_api_key='vk-obg***', # Your Obiguard virtual key here ) speech_file_path = Path(__file__).parent / "speech.mp3" response = client.audio.speech.create( model="tts-1", voice="alloy", input="Today is a wonderful day to build something people love!" ) f = open(speech_file_path, "wb") f.write(response.content) f.close() ``` ```python theme={null} from openai import OpenAI from obiguard import createHeaders client = OpenAI( base_url='https://gateway.obiguard.ai/v1', api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="vk-obg***", # Your Obiguard virtual key here ) ) speech_file_path = Path(__file__).parent / "speech.mp3" response = client.audio.speech.create( model="tts-1", voice="alloy", input="Today is a wonderful day to build something people love!" ) f = open(speech_file_path, "wb") f.write(response.content) f.close() ``` ``` curl https://gateway.obiguard.ai/v1/audio/speech \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: vk-obg***" \ -d '{ "model": "tts-1", "voice": "alloy", "input": "Today is a wonderful day to build something people love!" }' --output speech.mp3 ``` After completion, the request will appear in the logs UI, displaying the associated cost and latency. ## Supported Providers and Models The following providers currently support text-to-speech. More providers will be added soon. | Provider | Models | | ------------ | --------------- | | OpenAI | tts-1, tts-1-hd | | Azure OpenAI | tts-1, tts-1-hd | | Deepgram | Coming Soon | | ElevenLabs | Coming Soon | # Thinking Mode Source: https://docs.obiguard.ai/gateway/multimodal-capabilities/thinking-mode Enable Thinking Mode to enhance your AI's reasoning capabilities. Thinking/Reasoning models are a new class of LLMs designed to make their internal reasoning visible. Unlike standard LLMs that provide only final answers, thinking models such as Claude 3.7 Sonnet, OpenAI o1/o3, and Deepseek R1 “think out loud” by generating a detailed chain of thought before presenting their conclusions. These models are optimized for tasks that demand complex analysis, multi-step reasoning, and structured logic. Obiguard offers access to these advanced models through a unified API that works seamlessly across different providers. ## Supported Thinking Models Obiguard currently supports these thinking-enabled models: * **Anthropic**: claude-3-7-sonnet-latest * **Google Vertex AI**: anthropic.claude-3-7-sonnet\@20250219 * **Amazon Bedrock**: claude-3-7-sonnet Additional thinking models will be supported as they become available. ## Using Thinking Mode 1. Set `strict_open_ai_compliance=False` in your headers or client configuration. 2. Thinking responses are returned in a format different from standard completions. 3. For streaming, the thinking content appears in `response_chunk.choices[0].delta.content_blocks`. ### Basic Example ```python theme={null} from obiguard import Obiguard # Initialize the Obiguard client client = Obiguard( virtual_key="vk-obg***", # Add your provider's virtual key strict_open_ai_compliance=False # Required for thinking mode ) # Create the request response = client.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 # Maximum tokens to use for thinking }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) # For streaming responses, handle content_blocks differently # response = client.chat.completions.create( # ...same config as above but with stream=True # ) # for chunk in response: # if chunk.choices[0].delta: # content_blocks = chunk.choices[0].delta.get("content_blocks") # if content_blocks is not None: # for content_block in content_blocks: # print(content_block) ``` ```python theme={null} from openai import OpenAI from obiguard import createHeaders # Initialize the OpenAI client with Obiguard headers client = OpenAI( base_url='https://gateway.obiguard.ai/v1', api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="vk-obg***", # Your Obiguard virtual key here strict_open_ai_compliance=False # Required for thinking mode ) ) # Create the request response = client.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 # Maximum tokens to use for thinking }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) # For streaming responses, handle content_blocks differently # response = client.chat.completions.create( # ...same config as above but with stream=True # ) # for chunk in response: # if chunk.choices[0].delta: # content_blocks = chunk.choices[0].delta.get("content_blocks") # if content_blocks is not None: # for content_block in content_blocks: # print(content_block) ``` ```curl theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anthropic" \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "claude-3-7-sonnet-latest", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": false, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] }' ``` ## Multi-Turn Conversations For multi-turn conversations, include the previous thinking content in the conversation history: ```python theme={null} from obiguard import Obiguard # Initialize the Obiguard client client = Obiguard( virtual_key="vk-obg***", # Your Obiguard virtual key here strict_open_ai_compliance=False ) # Create the request response = client.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='Anthropic_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anthropic", obiguard_api_key="vk-obg***", # Your Obiguard virtual key here strict_open_ai_compliance=False ) ) response = openai.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```curl theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anthropic" \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "claude-3-7-sonnet-latest", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": false, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] } ``` ## Understanding the Response Structure When working with thinking-enabled models, note that their responses use a special format: The assistant’s thinking output is found in the `response_chunk.choices[0].delta.content_blocks` array, not in the `response.choices[0].message.content` string. This distinction is crucial for streaming responses, where you must extract the thinking content from the content blocks. ## FAQs No, thinking mode is only available on specific reasoning-optimized models. Currently, this includes Claude 3.7 Sonnet and will expand to other models as they become available. Yes, enabling thinking mode will increase your token usage since the model is generating additional content for its reasoning process. The `budget_tokens` parameter lets you control the maximum tokens allocated to thinking. Yes, particularly for streaming responses. The thinking content is returned in the `content_blocks` array rather than the standard content field, so you’ll need to adapt your response parsing logic. The thinking mode response format extends beyond the standard OpenAI completion schema. Setting `strict_open_ai_compliance` to false allows Obiguard to return this extended format with the thinking content. # Vision Source: https://docs.obiguard.ai/gateway/multimodal-capabilities/vision Obiguard’s AI gateway enables integration with leading vision models such as OpenAI’s GPT-4V, Google’s Gemini, and others. What are vision models? Vision models are AI systems that integrate visual and language understanding, enabling them to interpret images alongside natural language text. They are trained on extensive datasets containing both images and text, with training methods varying based on their specific objectives. ## Using Vision Chat Completion Obiguard implements the OpenAI message format, allowing you to include images in API requests. You can provide images to the model either by supplying a URL or by embedding the image as a base64-encoded string. Below is an example with OpenAI’s `gpt-4o` model: ``` from obiguard import Obiguard client = Obiguard( obiguard_api_key='vk-obg***', # Your Obiguard virtual key here ) response = client.chat.completions.create( model="gpt-4o", messages=[{ "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, { "type": "image_url", "image_url": { "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", }, }, ], }], ) print(response) ``` ``` from openai import OpenAI from obiguard import createHeaders client = OpenAI( api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="vk-obg***", # Your Obiguard virtual key here ) ) response = client.chat.completions.create( model="gpt-4o", messages=[{ "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, { "type": "image_url", "image_url": { "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", }, }, ], }], ) print(response) ``` ``` curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-virtual-key: $OPENAI_VIRTUAL_KEY" \ -d '{ "model": "gpt-4o", "messages": [{ "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, { "type": "image_url", "image_url": { "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" } } ] }] }' ``` ## Supported Providers and Models Obiguard integrates with a wide range of vision models from leading providers. The table below lists some of the supported models. | Provider | Models | Functions | | ------------ | -------------------------------------------------------------------------------------------------- | ---------------------- | | OpenAI | gpt-4-vision-preview, gpt-4o, gpt-4o-mini | Create Chat Completion | | Azure OpenAI | gpt-4-vision-preview, gpt-4o, gpt-4o-mini | Create Chat Completion | | Gemini | gemini-1.0-pro-vision, gemini-1.5-flash, gemini-1.5-flash-8b, gemini-1.5-pro | Create Chat Completion | | Anthropic | claude-3-sonnet, claude-3-haiku, claude-3-opus, claude-3.5-sonnet, claude-3.5-haiku | Create Chat Completion | | AWS Bedrock | anthropic.claude-3-5-sonnet, anthropic.claude-3-5-haiku, anthropic.claude-3-5-sonnet-20240620-v1:0 | Create Chat Completion | # Realtime API Source: https://docs.obiguard.ai/gateway/realtime-api Leverage OpenAI's Realtime API with logging, cost tracking, and additional features! OpenAI’s Realtime API, while being the fastest way to utilize multi-modal generation, comes with challenges such as logging, cost tracking, and implementing guardrails. Obiguard’s AI Gateway addresses these challenges through seamless integration. It's logging feature uniquely captures the complete request and response, including the model’s output, cost, and any guardrail violations. ## Getting Started: ```python theme={null} from obiguard import AsyncObiguard as Obiguard, OBIGUARD_GATEWAY_URL import asyncio async def main(): client = Obiguard( virtual_key="vk-obg-***", # Your Obiguard virtual key here base_url=OBIGUARD_GATEWAY_URL, ) async with client.beta.realtime.connect( model="gpt-4o-realtime-preview-2024-10-01" # Replace with the model you want to use ) as connection: await connection.session.update(session={'modalities': ['text']}) await connection.conversation.item.create( item={ "type": "message", "role": "user", "content": [{"type": "input_text", "text": "Say hello!"}], } ) await connection.response.create() async for event in connection: if event.type == 'response.text.delta': print(event.delta, flush=True, end="") elif event.type == 'response.text.done': print() elif event.type == "response.done": break asyncio.run(main()) ``` ```python theme={null} import asyncio from openai import AsyncOpenAI from obiguard import createHeaders, OBIGUARD_GATEWAY_URL async def main(): headers = createHeaders( provider="openai", virtual_key="vk-obg-***" # Your Obiguard virtual key here ) client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, ) async with client.beta.realtime.connect( model="gpt-4o-realtime-preview", extra_headers=headers # Replace with the model you want to use ) as connection: await connection.session.update(session={'modalities': ['text']}) await connection.conversation.item.create( item={ "type": "message", "role": "user", "content": [{"type": "input_text", "text": "Say hello!"}], } ) await connection.response.create() async for event in connection: if event.type == 'response.text.delta': print(event.delta, flush=True, end="") elif event.type == 'response.text.done': print() elif event.type == "response.done": break asyncio.run(main()) ``` ```curl theme={null} # we're using websocat for this example, but you can use any websocket client websocat "wss://gateway.obiguard.ai/v1/realtime?model=gpt-4o-realtime-preview-2024-10-01" \ -H "x-obiguard-provider: openai" \ -H "x-obiguard-api-key: sk-obg***" \ -H "x-obiguard-OpenAI-Beta: realtime=v1" # once connected, you can send your messages as you would with OpenAI's Realtime API ``` If you prefer not to store your API keys with Obiguard, you can include your OpenAI key in the `Authorization` header instead. ## Fire Away! View your logs in real-time with clear visualized traces and detailed cost tracking. # Remote MCP Source: https://docs.obiguard.ai/gateway/remote-mcp Obiguard's AI gateway has MCP server support that many foundational model providers offer. [Model Context Protocol (MCP)](https://modelcontextprotocol.io/introduction) is an open standard that defines how applications can provide tools and context to LLMs. The MCP tool in the Responses API enables developers to grant models access to tools hosted on Remote MCP servers. These servers are managed by developers and organizations across the internet and make tools available to MCP clients, such as the Responses API. Obiguard supports using MCP servers through the Responses API. Making a request to a remote MCP server with the Responses API is simple. For instance, you can use the DeepWiki MCP server to query nearly any public GitHub repository. ## Example MCP Request A Responses API request to OpenAI with MCP tools enabled. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg-***", # Your Obiguard API key here ) resp = client.responses.create( model="gpt-4.1", tools=[ { "type": "mcp", "server_label": "deepwiki", "server_url": "https://mcp.deepwiki.com/mcp", "require_approval": "never", }, ], input="What transport protocols are supported in the 2025-03-26 version of the MCP spec?", ) print(resp.output_text) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="dummy", # Not used when using virtual keys base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk_obg-***", # Your Obiguard virtual key here ) ) resp = client.responses.create( model="gpt-4.1", tools=[ { "type": "mcp", "server_label": "deepwiki", "server_url": "https://mcp.deepwiki.com/mcp", "require_approval": "never", }, ], input="What transport protocols are supported in the 2025-03-26 version of the MCP spec?", ) print(resp.output_text) ``` ```curl theme={null} curl https://gateway.obiguard.ai/v1/responses \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: vk-obg***" \ # Your Obiguard API key here -d '{ "model": "gpt-4.1", "tools": [ { "type": "mcp", "server_label": "deepwiki", "server_url": "https://mcp.deepwiki.com/mcp", "require_approval": "never" } ], "input": "What transport protocols are supported in the 2025-03-26 version of the MCP spec?" }' ``` ## MCP Server Authentication While the DeepWiki MCP server does not require authentication, most other MCP servers do. The MCP tool in the Responses API allows you to specify custom headers for requests to remote MCP servers. These headers can include API keys, OAuth tokens, or any other authentication method supported by the server. The most commonly used header for authentication is the Authorization header. Here is an example of how to include it: Using the Stripe MCP tool ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key here ) resp = client.responses.create( model="gpt-4.1", input="Create a payment link for $20", tools=[ { "type": "mcp", "server_label": "stripe", "server_url": "https://mcp.stripe.com", "headers": { "Authorization": "Bearer $STRIPE_API_KEY" } } ] ) print(resp.output_text) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="dummy", # Not used when using virtual keys base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk_obg-***", # Your Obiguard virtual key here ) ) resp = client.responses.create( model="gpt-4.1", tools=[ { "type": "mcp", "server_label": "stripe", "server_url": "https://mcp.stripe.com", "headers": { "Authorization": "Bearer $STRIPE_API_KEY" } }, ], input="What transport protocols are supported in the 2025-03-26 version of the MCP spec?", ) print(resp.output_text) ``` ```curl theme={null} curl https://gateway.obiguard.ai/v1/responses \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: vk-obg***" \ # Your Obiguard API key here -d '{ "model": "gpt-4.1", "input": "Create a payment link for $20", "tools": [ { "type": "mcp", "server_label": "stripe", "server_url": "https://mcp.stripe.com", "headers": { "Authorization": "Bearer $STRIPE_API_KEY" } } ] }' ``` To ensure sensitive keys remain secure, the Responses API does not store any string values provided in the `headers` object. These values are also excluded from the created Response object. Additionally, since some remote MCP servers generate authenticated URLs, the path portion of the `server_url` is removed in responses (e.g., `example.com/mcp` becomes `example.com`). As a result, you must include the full `server_url` path and any necessary `headers` in every Responses API creation request. # Strict OpenAI Compliance Source: https://docs.obiguard.ai/gateway/strict-open-ai-compliance Obiguard ensures that all responses strictly adhere to the [OpenAI specification](https://platform.openai.com/docs/api-reference/chat/create) by default. In certain scenarios, responses from providers like Perplexity may include additional fields that do not directly correspond to OpenAI fields. To retrieve these fields, use one of the following methods: * **Python SDK**: Set the parameter `strict_open_ai_compliance=false` when initializing the Obiguard client. * **Node SDK**: Set the parameter `strictOpenAiCompliance: false` when initializing the Obiguard client. * **HTTP Requests**: Include the header `x-obiguard-strict-open-ai-compliance: false` in your request. By default, Obiguard Python and Node SDKs have `strict_open_ai_compliance` set to `false`. ## Examples ```python theme={null} from obiguard import Obiguard client = Obiguard( provider='openai', base_url='https://gateway.obiguard.ai/v1', obiguard_api_key='vk-obg***', # Your Obiguard virtual key here strict_open_ai_compliance=False, ) response = client.chat.completions.create( messages = [{ "role": 'user', "content": 'Say this is a test' }], model = "Qwen/Qwen2.5-32B-Instruct" # The LLM model tied to the virtual key ) ``` ```python theme={null} from openai import OpenAI from obiguard import createHeaders client = OpenAI( base_url='https://gateway.obiguard.ai/v1', api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="sk-obg***", # Your Obiguard API key here strict_open_ai_compliance=False, ) ) response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}] ) ``` ```curl theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "gpt-4o-mini", "messages": [{ "role": "user", "content": "Hello!" }] }' ``` # Universal API Source: https://docs.obiguard.ai/gateway/universal-api Obiguard Universal API provides a unified interface for accessing multiple LLM providers, simplifying integration and management. Instead of building and maintaining separate integrations for each multimodal LLM, you can use a single, unified API to access models from OpenAI, Anthropic, Meta, Cohere, Mistral, and over 100 models across 15+ providers. ## Obiguard Uses the OpenAI API Standard The Obiguard API leverages its robust open-source AI Gateway to translate all incoming requests into the OpenAI API format, ensuring responses are fully OpenAI-compatible. ## Seamlessly Switch Between Providers ```python theme={null} from obiguard import Obiguard # Calling OpenAI client = Obiguard( provider = "openai", Authorization = "sk-xxxxx" ) response = obiguard.chat.completions.create( messages = [{ "role": 'user', "content": 'Hello' }], model = 'gpt-4' ) # Switching to Anthropic client = Obiguard( provider = "anthropic", Authorization = "sk-ant-xxxxx" ) response = obiguard.chat.completions.create( messages = [{ "role": 'user', "content": 'Hello' }], model = 'claude-3-opus-20240229' ) ``` ## Integrating Local or Private Models Obiguard can connect to and monitor your locally or privately hosted LLMs, provided the model is compatible with one of the 15+ providers supported by Obiguard and its URL is accessible. Just set the `custom_host` parameter along with the provider name, and Obiguard will manage communication with your local model. When using `custom_host`, make sure to include the version segment (e.g., /v1) in the URL. Obibuard will automatically add the correct endpoint path (`/chat/completions`, `/completions`, or `/embeddings`). For Ollama models, this process is different. See documentation for details. ### Multimodality Obiguard offers seamless integration with multimodal models using the same unified API, enabling support for vision, audio, image generation, and additional advanced capabilities across multiple providers. Multimodal Capabilities # Guardrail Policies Source: https://docs.obiguard.ai/guardrail-AI/guardrail-policies Obiguard enables the screening of LLM interactions to address various threats. Validator checks, flagging logic, and strictness levels are all centrally managed through an Obiguard guardrail policy. A [project](../projects) may contain multiple guardrail policies. A policy allows you to define specific configurations for each application, LLM-based feature, environment, or end user. You can also adjust validators and strictness levels for individual apps and integrations dynamically to address threats, improve user experience, or align with your risk tolerance. ## Validators Obiguard validators are categorized into four types of defenses: 1. Prompt defense - Safeguards against user prompts, documents, or other LLM inputs containing instructions that could override intended behavior, manipulate the LLM into malicious actions, or leak sensitive data. These harmful instructions, known as prompt attacks, include jailbreaks, prompt injections, and other manipulative or malicious inputs. 2. Content moderation - Prevent your applications from generating inappropriate or harmful content, and identify attempts by users to produce offensive material or engage in dangerous or illegal activities. Content moderation includes: * Criminal activity * Hate speech * Profanity * Explicit content * Violence * Weapon-related content * Custom detectors to flag specific trigger words or phrases 3. Data Leakage Prevention - Safeguard Personally Identifiable Information, prevent system prompt leakage, and avoid costly leakage of sensitive data, ensuring compliance with data protection and privacy regulations. PII coverage inclues: * Full names * United States mailing addresses * Phone numbers * Email addresses * Internet Protocol (IP) addresses * Credit card numbers * International Bank Account Numbers (IBANs) * United States Social Security Numbers (SSNs) 4. Unknown links - Prevent attackers from tricking the LLM into presenting malicious or phishing links to users. A policy group specifies the defenses to apply for securing LLM interactions and the paranoia level, allowing you to tailor configurations to your specific use cases and risk tolerance. ## How Obiguard policies work Each [project](../projects) in Obiguard includes a set of policy configurations. These policies determine the guardrail checks applied to every API request associated with the project's API key. For example, a policy might: * Inspect user inputs for prompt attacks or sensitive information (PII). * Analyze LLM outputs for violations of content moderation rules or suspicious links from untrusted domains. ## Flagging logic When the validators in the policy group screen a request, any detection will result in the request being marked as ‘flagged’. If one or more validators flag the request, the `guard` response will indicate flagged equals `failed`. If no validators flag the request, the response will indicate flagged equals `passed`. You can configure your application to handle flagged responses in various ways. Options include blocking the flagged inputs or outputs, prompting the user for confirmation before proceeding, logging the flagged response for analysis and monitoring, or taking no action. The choice is entirely yours. Optionally, the response can include a detailed breakdown of the flagging decision. This will show the detectors that were executed, as specified in the policy, and indicate whether each detector identified an issue. ## Threshold levels Certain validators allows adjusting the flagging thresholds by setting the confidence level for each defense category. This allows you to customize the strictness and risk tolerance for your specific use cases. For instance, if a use case has a high tolerance for risk, you can configure a detector to flag only detections with very high confidence, minimizing false positives. Conversely, for scenarios requiring strict protection, even at the expense of user experience, you can set the detector to flag any potential detection. This is achieved by adjusting the confidence level threshold for each defense. The threshold determines the minimum confidence level required for a detector to flag a detection. Obiguard defines the following confidence levels: L1 - Lenient, minimizing false positives. L2 - Balanced, allowing some false positives. L3 - Strict, with very low false negatives but more false positives. L4 - Paranoid, prioritizing minimal false negatives at the cost of higher false positives. This is the default confidence level. These levels align with OWASP’s paranoia level definitions for web application firewalls (WAFs). The Unknown Links detector cannot be adjusted and only provides a binary result: `passed` or `failed`. ## Latency impact The response time of the guard API depends on the content length being screened and the detectors specified in the policy. Changes to a policy may affect the latency experienced by your application and its users. If you need help aligning your policies to meet strict latency requirements please reach out to [support@obiguard.ai](mailto:support@obiguard.ai). ## A simple policy example Here’s an example of how you could configure a policy for your customer support chatbot: **Input settings:** * **Detect Jailbreak validator**: Set the threshold level to 0.8, ensuring that any request highly likely to be a prompt attack is flagged. * **Detect PII validator**: Enable detection for credit card numbers, IBAN codes, and US Social Security Numbers to prevent malicious data extraction and accidental user data leaks to the app or third-party LLM provider. * **Profanity Check validator**: Flag any content that is likely to be inappropriate or offensive. **Output settings:** * **Toxic Language validator**: Set the threshold level to 0.8 to flag content that is likely to violate moderation rules, ensuring the chatbot isn’t manipulated into generating inappropriate responses. * **Competitor Guard validator**: Add your competitors’ names to flag any mention of their brand or products. * **NSFW Text validator**: Set the threshold level to 0.8 to prevent the chatbot from sharing phishing or malicious links due to indirect prompt attacks. For example, if one of your products is jerk seasoning and Obiguard flags mentions of “jerk” as potential profanity due to strict content moderation settings, you can resolve this by editing the policy. Adjust the profanity threshold level to L1 while keeping other content moderation detectors at L3. This change takes effect immediately, allowing your customers to freely ask questions about jerk seasoning. # Guardrail Validators Source: https://docs.obiguard.ai/guardrail-AI/guardrail-validators Confidently deploy to production with Obiguard guardrails protecting your requests and responses LLMs can be unpredictable — not just in terms of API reliability or unexpected 400/500 errors, but also in their fundamental behavior. A response with a 200 status code might still break your application’s workflow due to unexpected or malformed output. Obiguard’s AI Guardrails help you enforce consistent LLM behavior in real time, using a guardrail on the Gateway approach. Leverage Obiguard’s Guardrails to validate both your LLM inputs and outputs according to your specified checks. Built on our Gateway, Guardrails let you orchestrate requests with actions such as denying requests, logging results, falling back to alternative LLMs or prompts, retrying requests, and more. Here are some examples of guardrails offered by Obiguard: * **Regex Match**: Checks if the request or response text matches a specific regex pattern. * **JSON Schema**: Verifies that the response JSON adheres to a defined schema. * **Code Detection**: Detects code snippets in formats such as SQL, Python, or TypeScript. * ...and more. Obiguard includes over 20 deterministic guardrails, along with LLM-based options like gibberish detection and prompt injection scanning. These guardrails provide robust protection, enabling organizations to deploy Gen AI securely and responsibly. ## How to use Guardrails AI Putting Obiguard guardrails in production is just a 4-step process: 1. Create a policy group 2. Add validators to the policy group 3. Generate an API key for the policy group 4. Attach the API key to a request This flowchart shows how Obiguard processes a guardrail request: Let's see in detail how to set up guardrails in Obiguard. ### 1. Create a Policy Group Navigate to the [Project](../projects) page, select guardrail Policies, and click Create to set up a new policy group. Give your policy group a name and, if desired, add a description for easier identification later. ### 2. Add Validators to the Policy Group In Obiguard, you can assign a guardrail validator to either the `INPUT` (PROMPT) or the `OUTPUT`. Ensure each validator is configured to check only one: either the input or the output. Each guardrail Check provides a specific input field tailored to its purpose—simply fill in the required details and save your check. You can include multiple checks within a single validator. Each check returns a simple boolean (passed/failed) result. ### 3. Generate an API Key for the Policy Group Once your policy group is set up with the desired validators, generate an API key for it. This key will be used to authenticate requests that require the guardrails defined in this policy group. You can generate multiple API keys for the same policy group if needed. ### 4. Attach the API Key to a Request To use the guardrails in your application, attach the generated API key to your requests. This is where Obiguard’s magic comes into play. The guardrail you created above is yet not an Active guardrail because it is not attached to any request. Provide the API key whenever you make a request to Obiguard, and it will apply the guardrails defined in your policy group to that request. ```python theme={null} from obiguard import Obiguard client = Obiguard( provider='openai', base_url='https://gateway.obiguard.ai/v1', obiguard_api_key='vk-obg***', # Your Obiguard virtual key here ) response = client.chat.completions.create( messages = [{ "role": 'user', "content": 'Say this is a test' }], model = "Qwen/Qwen2.5-32B-Instruct" # The LLM model tied to the virtual key ) ``` ```python theme={null} from openai import OpenAI from obiguard import createHeaders client = OpenAI( base_url='https://gateway.obiguard.ai/v1', api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="sk-obg***", # Your Obiguard API key here ) ) response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}] ) ``` ```curl theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -d '{ "model": "gpt-4o-mini", "messages": [{ "role": "user", "content": "Hello!" }] }' ``` You can monitor all requests and responses processed by the guardrails using Obiguard Logs. # Overview Source: https://docs.obiguard.ai/integrations/agents Obiguard helps bring your agents to production OpenAI Agents Pydantic AI Agents Autogen CrewAI Swarm Bring Your Agent ## Integrate Obiguard with your agents with just 2 lines of code ```py Langchain theme={null} from langchain_openai import ChatOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders llm = ChatOpenAI( api_key="OpenAI_API_Key", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", #choose your provider obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) ``` ## Key Production Features By routing your agent's requests through Obiguard, you make your agents production-grade with the following features. ### 1. [Interoperability](/gateway/universal-api) Easily switch between LLM providers. Call various LLMs such as Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, AWS Bedrock and much more by simply changing the `provider ` and `Obiguard API key` in the LLM object. ### 2. [Observability](/product/observability) Obiguard automatically logs key details about your agent runs, including cost, tokens used, response time, etc. For agent-specific observability, add Trace IDs to the request headers for each agent. This enables filtering analytics by Trace IDs, ensuring deeper monitoring and analysis. ### 3. [Logs](/product/observability/logs) Access a dedicated section to view records of action executions, including parameters, outcomes, and errors. Filter logs of your agent run based on multiple parameters such as trace ID, model, tokens used, metadata, etc. # Autogen Source: https://docs.obiguard.ai/integrations/agents/autogen Use Obiguard with Autogen to take your AI Agents to production ## Getting Started ### 1. Install the required packages: ```sh theme={null} pip install -qU pyautogen obiguard ``` ### **2.** Configure your Autogen configs: ```py theme={null} from autogen import AssistantAgent, UserProxyAgent, config_list_from_json from obiguard import OBIGUARD_GATEWAY_URL, createHeaders config = [ { "api_key": "OPENAI_API_KEY", "model": "gpt-3.5-turbo", "base_url": OBIGUARD_GATEWAY_URL, "api_type": "openai", "default_headers": createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key provider = "openai", ) } ] ``` ## Make your agents Production-ready with Obiguard Obiguard makes your Autogen agents reliable, robust, and production-grade with its observability suite and AI Gateway. Seamlessly integrate 200+ LLMs with your Autogen agents using Obiguard. Gain granular insights into agent performance and costs, and continuously optimize your AI operations—all with just 2 lines of code. Let's dive deep! Let's go through each of the use cases! ### 1.[ Interoperability](/gateway/universal-api) Easily switch between 200+ LLMs. Call various LLMs such as Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, AWS Bedrock, and many more by simply changing the `provider ` and `obiguard_api_key` in the `ChatOpenAI` object. If you are using OpenAI with autogen, your code would look like this: ```py theme={null} config = [ { "api_key": "OPENAI_API_KEY", "model": "gpt-3.5-turbo", "base_url": OBIGUARD_GATEWAY_URL, "api_type": "openai", "api_type": "openai", # Obiguard conforms to the openai api_type "default_headers": createHeaders( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "openai", ) } ] ``` To switch to Azure as your provider, add your Azure details to Obiguard vault ([here's how](/integrations/llms/azure-openai)) and use Azure OpenAI using virtual keys. ```py theme={null} config = [ { "api_key": "api-key", "model": "gpt-3.5-turbo", "base_url": OBIGUARD_GATEWAY_URL, "api_type": "openai", # Obiguard conforms to the openai api_type "default_headers": createHeaders( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "azure-openai", ) } ] ``` If you are using Anthropic with CrewAI, your code would look like this: ```py theme={null} config = [ { "api_key": "ANTHROPIC_VIRTUAL_KEY", "model": "gpt-3.5-turbo", "api_type": "openai", # Obiguard conforms to the openai api_type "base_url": OBIGUARD_GATEWAY_URL, "default_headers": createHeaders( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "anthropic", ) } ] ``` To switch to AWS Bedrock as your provider, add your AWS Bedrock details to Obiguard vault ([here's how](/integrations/llms/aws-bedrock)) and use AWS Bedrock using virtual keys, ```py theme={null} config = [ { "api_key": "api-key", #We are using Virtual Key "model": "gpt-3.5-turbo", "api_type": "openai", # Obiguard conforms to the openai api_type "base_url": OBIGUARD_GATEWAY_URL, "default_headers": createHeaders( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "bedrock", virtual_key="AWS_VIRTUAL_API_KEY" # Replace with Virtual Key ) } ] ``` ### 3. [Metrics](/product/observability) Agent runs can be costly. Tracking agent metrics is crucial for understanding the performance and reliability of your AI agents. Metrics help identify issues, optimize runs, and ensure that your agents meet their intended goals. Obiguard automatically logs comprehensive metrics for your AI agents, including **cost**, **tokens used**, **latency**, etc. Whether you need a broad overview or granular insights into your agent runs, Obiguard's customizable filters provide the metrics you need. ```py theme={null} config = [ { "api_key": "OPENAI_API_KEY", "model": "gpt-3.5-turbo", "base_url": OBIGUARD_GATEWAY_URL, "api_type": "openai", "default_headers": createHeaders( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "openai", ) } ] ``` ### 4. [Logs](/observability/logs) Agent runs are complex. Logs are essential for diagnosing issues, understanding agent behavior, and improving performance. They provide a detailed record of agent activities and tool use, which is crucial for debugging and optimizing processes. Obiguard offers comprehensive logging features that capture detailed information about every action and decision made by your AI agents. Access a dedicated section to view records of agent executions, including parameters, outcomes, function calls, and errors. Filter logs based on multiple parameters such as trace ID, model, tokens used, and metadata. # Bring Your own Agents Source: https://docs.obiguard.ai/integrations/agents/bring-your-own-agents You can also use Obiguard if you are doing custom agent orchestration! ## Getting Started ### 1. Install the required packages: ```sh theme={null} pip install obiguard openai ``` ### 2. Configure your OpenAI object: ```py theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="OPENAI_API_KEY", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg***", # Your Obiguard API key ) ) ``` ## Make your agents Production-ready with Obiguard Obiguard makes your agents reliable, robust, and production-grade with its observability suite and AI Gateway. Seamlessly integrate 200+ LLMs with your custom agents using Obiguard. Implement fallbacks, gain granular insights into agent performance and costs, and continuously optimize your AI operations—all with just 2 lines of code. Let's dive deep! Let's go through each of the use cases! ### 1. [Interoperability](/product/ai-gateway/universal-api) Easily switch between 200+ LLMs. Call various LLMs such as Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, AWS Bedrock, and many more by simply changing the `provider ` and `obiguard_api_key` in the `ChatOpenAI` object. If you are using OpenAI with CrewAI, your code would look like this: ```py theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="OPENAI_API_KEY", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg***", # Your Obiguard API key ) ) ``` To switch to Azure as your provider, add your Azure details to Obiguard vault ([here's how](/integrations/llms/azure-openai)) and use Azure OpenAI using virtual keys ```py theme={null} client = OpenAI( api_key="API_KEY", #We will use Virtual Key in this base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="azure-openai", obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="AZURE_VIRTUAL_KEY" #Azure Virtual key ) ) ``` If you are using Anthropic with CrewAI, your code would look like this: ```py theme={null} client = OpenAI( api_key="ANTROPIC_API_KEY", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anthropic", obiguard_api_key="sk-obg***", # Your Obiguard API key ) ) ``` To switch to AWS Bedrock as your provider, add your AWS Bedrock details to Obiguard vault ([here's how](/integrations/llms/aws-bedrock)) and use AWS Bedrock using virtual keys, ```py theme={null} client = OpenAI( api_key="api_key", #We will use Virtual Key in this base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="bedrock", obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="AWS_VIRTUAL_KEY" #Bedrock Virtual Key ) ) ``` ### 2. [Metrics](/product/observability) Agent runs can be costly. Tracking agent metrics is crucial for understanding the performance and reliability of your AI agents. Metrics help identify issues, optimize runs, and ensure that your agents meet their intended goals. Obiguard automatically logs comprehensive metrics for your AI agents, including **cost**, **tokens used**, **latency**, etc. Whether you need a broad overview or granular insights into your agent runs, Obiguard's customizable filters provide the metrics you need. For agent-specific observability, add `Trace-id` to the request headers for each agent. ```py theme={null} llm2 = ChatOpenAI( api_key="Anthropic_API_Key", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="anthropic", trace_id="research_agent1" #Add individual trace-id for your agent analytics ) ) ``` ### 3. [Logs](/observability/logs) Agent runs are complex. Logs are essential for diagnosing issues, understanding agent behavior, and improving performance. They provide a detailed record of agent activities and tool use, which is crucial for debugging and optimizing processes. Obiguard offers comprehensive logging features that capture detailed information about every action and decision made by your AI agents. Access a dedicated section to view records of agent executions, including parameters, outcomes, function calls, and errors. Filter logs based on multiple parameters such as trace ID, model, tokens used, and metadata. # CrewAI Source: https://docs.obiguard.ai/integrations/agents/crewai Use Obiguard with CrewAI to take your AI Agents to production ## Introduction CrewAI is a framework for orchestrating role-playing, autonomous AI agents designed to solve complex, open-ended tasks through collaboration. It provides a robust structure for agents to work together, leverage tools, and exchange insights to accomplish sophisticated objectives. Obiguard enhances CrewAI with production-readiness features, turning your experimental agent crews into robust systems by providing: * **Complete observability** of every agent step, tool use, and interaction * **Built-in reliability** with fallbacks, retries, and load balancing * **Cost tracking and optimization** to manage your AI spend * **Access to 200+ LLMs** through a single integration * **Guardrails** to keep agent behavior safe and compliant * **Version-controlled prompts** for consistent agent performance Learn more about CrewAI's core concepts and features ### Installation & Setup ```bash theme={null} pip install -U crewai obiguard ``` Create a Obiguard API key with optional budget/rate limits from the [Obiguard dashboard](https://app.obiguard.ai/). You can also attach configurations for reliability, caching, and more to this key. More on this later. The integration is simple - you just need to update the LLM configuration in your CrewAI setup: ```python theme={null} from crewai import LLM from obiguard import OBIGUARD_GATEWAY_URL, createHeaders # Create an LLM instance with Obiguard integration gpt_llm = LLM( model="gpt-4o", base_url=OBIGUARD_GATEWAY_URL, api_key="dummy", # We are using a Virtual key, so this is a placeholder extra_headers=obiguard.copy_headers() ) #Use them in your Crew Agents like this: @agent def lead_market_analyst(self) -> Agent: return Agent( config=self.agents_config['lead_market_analyst'], verbose=True, memory=False, llm=gpt_llm ) ``` **What are Virtual Keys?** Virtual keys in Obiguard securely store your LLM provider API keys (OpenAI, Anthropic, etc.) in an encrypted vault. They allow for easier key rotation and budget management. [Learn more about virtual keys here](/virtual-keys). ## Production Features ### 1. Enhanced Observability Obiguard provides comprehensive observability for your CrewAI agents, helping you understand exactly what's happening during each execution. Traces provide a hierarchical view of your crew's execution, showing the sequence of LLM calls, tool invocations, and state transitions. Obiguard logs every interaction with LLMs, including: * Complete request and response payloads * Latency and token usage metrics * Cost calculations * Tool calls and function executions All logs can be filtered by metadata, trace IDs, models, and more, making it easy to debug specific crew runs. Obiguard provides built-in dashboards that help you: * Track cost and token usage across all crew runs * Analyze performance metrics like latency and success rates * Identify bottlenecks in your agent workflows * Compare different crew configurations and LLMs ### 2. Model Interoperability CrewAI supports multiple LLM providers, and Obiguard extends this capability by providing access to over 200 LLMs through a unified interface. You can easily switch between different models without changing your core agent logic: ```python theme={null} from crewai import Agent, LLM from obiguard import OBIGUARD_GATEWAY_URL, createHeaders # Set up LLMs with different providers openai_llm = LLM( model="gpt-4o", base_url=OBIGUARD_GATEWAY_URL, api_key="dummy", extra_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) anthropic_llm = LLM( model="claude-3-5-sonnet-latest", max_tokens=1000, base_url=OBIGUARD_GATEWAY_URL, api_key="dummy", extra_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) # Choose which LLM to use for each agent based on your needs researcher = Agent( role="Senior Research Scientist", goal="Discover groundbreaking insights about the assigned topic", backstory="You are an expert researcher with deep domain knowledge.", verbose=True, llm=openai_llm # Use anthropic_llm for Anthropic ) ``` Obiguard provides access to LLMs from providers including: * OpenAI (GPT-4o, GPT-4 Turbo, etc.) * Anthropic (Claude 3.5 Sonnet, Claude 3 Opus, etc.) * Mistral AI (Mistral Large, Mistral Medium, etc.) * Google Vertex AI (Gemini 1.5 Pro, etc.) * Cohere (Command, Command-R, etc.) * AWS Bedrock (Claude, Titan, etc.) * Local/Private Models See the full list of LLM providers supported by Obiguard. ## Set Up Enterprise Governance for CrewAI **Why Enterprise Governance?** If you are using CrewAI inside your organization, you need to consider several governance aspects: * **Cost Management**: Controlling and tracking AI spending across teams * **Access Control**: Managing which teams can use specific models * **Usage Analytics**: Understanding how AI is being used across the organization * **Security & Compliance**: Maintaining enterprise security standards * **Reliability**: Ensuring consistent service across all users Obiguard adds a comprehensive governance layer to address these enterprise needs. Let's implement these controls step by step. You can choose to create a guardrail policy to protect your data and ensure compliance with organizational policies. Add guardrail validators on your LLM inputs and output to govern your LLM usage. Virtual Keys are Obiguard's secure way to manage your LLM provider API keys. Think of them like disposable credit cards for your LLM API keys. To create a virtual key: Go to [Virtual Keys](/virtual-keys) in the Obiguard dashboard. Select the guardrail policy and your LLM provider. Save and copy the virtual key ID Save your virtual key ID - you'll need it for the next step. After setting up your Obiguard API key with the attached config, connect it to your CrewAI agents: ```python theme={null} from crewai import Agent, LLM from obiguard import OBIGUARD_GATEWAY_URL, createHeaders # Configure LLM with your API key obiguard_llm = LLM( model="gpt-4o", base_url=OBIGUARD_GATEWAY_URL, api_key="YOUR_Obiguard_API_KEY" ) # Create agent with Obiguard-enabled LLM researcher = Agent( role="Senior Research Scientist", goal="Discover groundbreaking insights about the assigned topic", backstory="You are an expert researcher with deep domain knowledge.", verbose=True, llm=obiguard_llm ) ``` ### Enterprise Features Now Available **Your CrewAI integration now has:** * Departmental budget controls * Model access governance * Usage tracking & attribution * Security guardrails * Reliability features ## Frequently Asked Questions Obiguard adds production-readiness to CrewAI through comprehensive observability (traces, logs, metrics), reliability features (fallbacks, retries, caching), and access to 200+ LLMs through a unified interface. This makes it easier to debug, optimize, and scale your agent applications. Yes! Obiguard integrates seamlessly with existing CrewAI applications. You just need to update your LLM configuration code with the Obiguard-enabled version. The rest of your agent and crew code remains unchanged. Obiguard supports all CrewAI features, including agents, tools, human-in-the-loop workflows, and all task process types (sequential, hierarchical, etc.). It adds observability and reliability without limiting any of the framework's functionality. Yes! Obiguard uses your own API keys for the various LLM providers. It securely stores them as virtual keys, allowing you to easily manage and rotate keys without changing your code. ## Resources

Official CrewAI documentation

 # OpenAI Agents SDK (Python) Source: https://docs.obiguard.ai/integrations/agents/openai-agents Use Obiguard with OpenAI Agents SDK to take your AI Agents to production ## Introduction OpenAI Agents SDK enables the development of complex AI agents with tools, planning, and memory capabilities. Obiguard enhances OpenAI Agents with observability, reliability, and production-readiness features. Obiguard turns your experimental OpenAI Agents into production-ready systems by providing: * **Complete observability** of every agent step, tool use, and interaction * **Cost tracking and optimization** to manage your AI spend * **Access to 200+ LLMs** through a single integration * **Guardrails** to keep agent behavior safe and compliant Learn more about OpenAI Agents SDK's core concepts ### Installation & Setup ```bash theme={null} pip install -U openai-agents obiguard ``` Create a Obiguard API key There are 3 ways to integrate Obiguard with OpenAI Agents: 1. Set a client that applies to all agents in your application 2. Use a custom provider for selective Obiguard integration 3. Configure each agent individually See the [Integration Approaches](#integration-approaches) section for more details. For a simple setup, we'll use the global client approach: ```python theme={null} from agents import ( set_default_openai_client, set_default_openai_api, Agent, Runner ) from openai import AsyncOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders import os # Set up Obiguard as the global client client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, obiguard_api_key=os.environ["OBIGUARD_API_KEY"], default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) # Register as the SDK-wide default set_default_openai_client(client, use_for_tracing=False) set_default_openai_api("chat_completions") # Responses API → Chat ``` **What are Virtual Keys?** Virtual keys in Obiguard securely store your LLM provider API keys (OpenAI, Anthropic, etc.) in an encrypted vault. They allow for easier key rotation and budget management. [Learn more about virtual keys here](/virtual-keys). ### Getting Started Let's create a simple question-answering agent with OpenAI Agents SDK and Obiguard. This agent will respond directly to user messages using a language model: ```python theme={null} from agents import ( set_default_openai_client, set_default_openai_api, Agent, Runner ) from openai import AsyncOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders import os # Set up Obiguard as the global client client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) # Register as the SDK-wide default set_default_openai_client(client, use_for_tracing=False) set_default_openai_api("chat_completions") # Responses API → Chat # Create agent with any supported model agent = Agent( name="Assistant", instructions="You are a helpful assistant.", model="gpt-4o" # Using Anthropic Claude through Obiguard ) # Run the agent result = Runner.run_sync(agent, "Tell me about quantum computing.") print(result.final_output) ``` In this example: 1. We set up Obiguard as the global client for OpenAI Agents SDK 2. We create a simple agent with instructions and a model 3. We run the agent synchronously with a user query 4. We print the final output Visit your Obiguard dashboard to see detailed logs of this agent's execution! ### Integration Approaches There are three ways to integrate Obiguard with OpenAI Agents SDK, each suited for different scenarios: Set a global client that affects all agents in your application: ```python theme={null} from agents import ( set_default_openai_client, set_default_openai_api, set_tracing_disabled, Agent, Runner ) from openai import AsyncOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders import os # Set up Obiguard as the global client client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, obiguard_api_key=os.environ["OBIGUARD_API_KEY"], default_headers=createHeaders( virtual_key="YOUR_OPENAI_VIRTUAL_KEY" ) ) # Register it as the SDK-wide default set_default_openai_client(client, use_for_tracing=False) # skip OpenAI tracing set_default_openai_api("chat_completions") # Responses API → Chat set_tracing_disabled(True) # optional # Regular agent code—just a model name agent = Agent( name="Haiku Writer", instructions="Respond only in haikus.", model="claude-3-7-sonnet-latest" ) print(Runner.run_sync(agent, "Write a haiku on recursion.").final_output) ``` **Best for**: Whole application migration to Obiguard with minimal code changes Use a custom ModelProvider to control which runs use Obiguard: ```python theme={null} from agents import ( Model, ModelProvider, RunConfig, Runner, Agent ) from agents import OpenAIChatCompletionsModel # concrete Model from openai import AsyncOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders import os, asyncio client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) class ObiguardProvider(ModelProvider): def get_model(self, model_name: str | None) -> Model: return OpenAIChatCompletionsModel( model=model_name or "claude-3-7-sonnet-latest", openai_client=client ) provider = ObiguardProvider() # singleton is fine async def main(): agent = Agent(name="Assistant", instructions="Haikus only.") run_cfg = RunConfig(model_provider=provider) # Only this call uses Obiguard out = await Runner.run(agent, "Weather in Tokyo?", run_config=run_cfg) print(out.final_output) asyncio.run(main()) ``` **Best for**: A/B testing, staged rollouts, or toggling between providers at runtime Attach a specific Model object to each Agent: ```python theme={null} from agents import Agent, Runner, OpenAIChatCompletionsModel from openai import AsyncOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders import os client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) agent = Agent( name="Haiku Writer", instructions="Classic Japanese form.", model=OpenAIChatCompletionsModel( # concrete Model model="claude-3-7-sonnet-latest", openai_client=client ), ) print(Runner.run_sync(agent, "Recursion haiku.").final_output) ``` **Best for**: Mixed agent environments where different agents need different providers or configurations **Comparing the 3 approaches** | Strategy | Code Touchpoints | Best For | | :------------------------------------------------ | :-------------------------------------------- | :------------------------------------------------------- | | **Global Client** via `set_default_openai_client` | One-time setup; agents need only model names | Whole app uses Obiguard; simplest migration | | **ModelProvider in RunConfig** | Add a provider + pass `run_config` | Toggle Obiguard per run; A/B tests, staged rollouts | | **Explicit Model per Agent** | Specify `OpenAIChatCompletionsModel` in agent | Mixed fleet: each agent can talk to a different provider | ## End-to-End Example **Research Agent with Tools**: Here's a more comprehensive agent that can use tools to perform tasks. ```python [expandable] theme={null} from agents import Agent, Runner, Tool, set_default_openai_client from openai import AsyncOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders import os # Configure Obiguard client client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) set_default_openai_client(client) # Define agent tools def get_weather(location: str) -> str: """Get the current weather for a location.""" return f"It's 72°F and sunny in {location}." def search_web(query: str) -> str: """Search the web for information.""" return f"Found information about: {query}" # Create agent with tools agent = Agent( name="Research Assistant", instructions="You are a helpful assistant that can search for information and check the weather.", model="claude-3-opus-20240229", tools=[ Tool( name="get_weather", description="Get current weather for a location", input_schema={ "location": { "type": "string", "description": "City and state, e.g. San Francisco, CA" } }, callback=get_weather ), Tool( name="search_web", description="Search the web for information", input_schema={ "query": { "type": "string", "description": "Search query" } }, callback=search_web ) ] ) # Run the agent result = Runner.run_sync( agent, "What's the weather in San Francisco and find information about Golden Gate Bridge?" ) print(result.final_output) ``` Visit your Obiguard dashboard to see the complete execution flow visualized! *** ## Production Features ### 1. Enhanced Observability Obiguard provides comprehensive observability for your OpenAI Agents, helping you understand exactly what's happening during each execution. Traces provide a hierarchical view of your agent's execution, showing the sequence of LLM calls, tool invocations, and state transitions. ```python theme={null} # Add tracing to your OpenAI Agents client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) set_default_openai_client(client) ``` Obiguard logs every interaction with LLMs, including: * Complete request and response payloads * Latency and token usage metrics * Cost calculations * Tool calls and function executions All logs can be filtered by metadata, trace IDs, models, and more, making it easy to debug specific agent runs. Obiguard provides built-in dashboards that help you: * Track cost and token usage across all agent runs * Analyze performance metrics like latency and success rates * Identify bottlenecks in your agent workflows * Compare different agent configurations and LLMs You can filter and segment all metrics by custom metadata to analyze specific agent types, user groups, or use cases. ### 2. Guardrails for Safe Agents Guardrails ensure your OpenAI Agents operate safely and respond appropriately in all situations. **Why Use Guardrails?** OpenAI Agents can experience various failure modes: * Generating harmful or inappropriate content * Leaking sensitive information like PII * Hallucinating incorrect information * Generating outputs in incorrect formats Obiguard's guardrails protect against these issues by validating both inputs and outputs. Obiguard's guardrails can: * Detect and redact PII in both inputs and outputs * Filter harmful or inappropriate content * Validate response formats against schemas * Check for hallucinations against ground truth * Apply custom business logic and rules Explore Obiguard's guardrail features to enhance agent safety ### 3. Tracing Obiguard provides an opentelemetry compatible backend to store and query your traces. You can trace your OpenAI Agents using any OpenTelemetry compatible tracing library. ## Tool Use in OpenAI Agents OpenAI Agents SDK natively supports tools that enable your agents to interact with external systems and APIs. Obiguard provides full observability for tool usage in your agents: ```python [expandable] theme={null} from agents import Agent, Runner, Tool, set_default_openai_client from openai import AsyncOpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders import os # Configure Obiguard client with tracing client = AsyncOpenAI( base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key trace_id="tools_example", metadata={"agent_type": "research"} ) ) set_default_openai_client(client) # Define tools def get_weather(location: str, unit: str = "fahrenheit") -> str: """Get the current weather in a given location""" return f"The weather in {location} is 72 degrees {unit}" def get_population(city: str, country: str) -> str: """Get the population of a city""" return f"The population of {city}, {country} is 1,000,000" # Create agent with tools agent = Agent( name="Research Assistant", instructions="You are a helpful assistant that can look up weather and population information.", model="gpt-4o-mini", tools=[ Tool( name="get_weather", description="Get the current weather in a given location", input_schema={ "location": { "type": "string", "description": "City and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "description": "Temperature unit (celsius or fahrenheit)", "default": "fahrenheit" } }, callback=get_weather ), Tool( name="get_population", description="Get the population of a city", input_schema={ "city": { "type": "string", "description": "City name" }, "country": { "type": "string", "description": "Country name" } }, callback=get_population ) ] ) # Run the agent result = Runner.run_sync( agent, "What's the weather in San Francisco and what's the population of Tokyo, Japan?" ) print(result.final_output) ``` ## Set Up Enterprise Governance for OpenAI Agents **Why Enterprise Governance?** If you are using OpenAI Agents inside your orgnaization, you need to consider several governance aspects: * **Cost Management**: Controlling and tracking AI spending across teams * **Access Control**: Managing which teams can use specific models * **Usage Analytics**: Understanding how AI is being used across the organization * **Security & Compliance**: Maintaining enterprise security standards * **Reliability**: Ensuring consistent service across all users Obiguard adds a comprehensive governance layer to address these enterprise needs. Let's implement these controls step by step. **Enterprise Implementation Guide** Obiguard allows you to use 1600+ LLMs with your OpenAI Agents setup, with minimal configuration required. Let's set up the core components in Obiguard that you'll need for integration. You can choose to create a guardrail policy to protect your data and ensure compliance with organizational policies. Add guardrail validators on your LLM inputs and output to govern your LLM usage. Virtual Keys are Obiguard's secure way to manage your LLM provider API keys. Think of them like disposable credit cards for your LLM API keys. To create a virtual key: Go to [Virtual Keys](/virtual-keys) in the Obiguard dashboard. Select the guardrail policy and your LLM provider. Save and copy the virtual key ID Save your virtual key ID - you'll need it for the next step. Once you have created your API Key after attaching default config, you can directly pass the API key + base URL in the AsyncOpenAI client. Here's how: ```Python theme={null} from obiguard import createHeaders, OBIGUARD_GATEWAY_URL from openai import AsyncOpenAI client=AsyncOpenAI( obiguard_api_key="vk-obg***", # Your Obiguard virtual key base_url="OBIGUARD_GATEWAY_URL" ) # your rest of the code remains same ``` ### Enterprise Features Now Available **OpenAI Agents now has:** * Departmental budget controls * Model access governance * Usage tracking & attribution * Security guardrails * Reliability features ## Frequently Asked Questions Obiguard adds production-readiness to OpenAI Agents through comprehensive observability (traces, logs, metrics), reliability features (fallbacks, retries, caching), and access to 200+ LLMs through a unified interface. This makes it easier to debug, optimize, and scale your agent applications. Yes! Obiguard integrates seamlessly with existing OpenAI Agents. You only need to replace your client initialization code with the Obiguard-enabled version. The rest of your agent code remains unchanged. Obiguard supports all OpenAI Agents SDK features, including tool use, memory, planning, and more. It adds observability and reliability without limiting any of the SDK's functionality. Obiguard fully supports streaming responses in OpenAI Agents. You can enable streaming by using the appropriate methods in the OpenAI Agents SDK, and Obiguard will properly track and log the streaming interactions. Obiguard allows you to add custom metadata to your agent runs, which you can then use for filtering. Add fields like `agent_name`, `agent_type`, or `session_id` to easily find and analyze specific agent executions. Yes! Obiguard uses your own API keys for the various LLM providers. It securely stores them as virtual keys, allowing you to easily manage and rotate keys without changing your code. ## Resources

Official OpenAI Agents SDK documentation

Example implementations for various use cases

Get personalized guidance on implementing this integration

 # OpenAI Swarm Source: https://docs.obiguard.ai/integrations/agents/openai-swarm The Obiguard x Swarm integration brings advanced AI gateway capabilities, full-stack observability, and reliability features to build production-ready AI agents. Swarm is an experimental framework by OpenAI for building multi-agent systems. It showcases the handoff & routines pattern, making agent coordination and execution lightweight, highly controllable, and easily testable. Obiguard integration extends Swarm's capabilities with production-ready features like observability, reliability, and more. ## Getting Started ```sh theme={null} pip install -U obiguard ``` To build Swarm Agents with Obiguard, you'll need either one of these keys: * **Obiguard API Key**: Sign up on the [Obiguard app](https://app.Obiguard.ai) and copy your API key. * **Virtual Key**: Virtual Keys are a secure way to manage your LLM API KEYS in one place. Instead of handling multiple API keys in your code, you can store your LLM provider API Keys securely in Obiguard's vault Create a Virtual Key in the [Obiguard app](https://app.Obiguard.ai) ```py theme={null} from swarm import Swarm, Agent from obiguard import Obiguard obiguard_client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) client = Swarm(client=obiguard_client) ``` In this example we are building a simple Weather Agent using OpenAI Swarm with Obiguard. ```py theme={null} def get_weather(location) -> str: return "{'temp':67, 'unit':'F'}" agent = Agent( name="Agent", instructions="You are a helpful agent.", functions=[get_weather], ) messages = [{"role": "user", "content": "What's the weather in NYC?"}] response = client.run(agent=agent, messages=messages) print(response.messages[-1]["content"]) ``` ## E2E example with Function Calling in OpenAI Swarm Here's a complete example showing function calling and agent interaction: ```py theme={null} from swarm import Swarm, Agent from obiguard import Obiguard obiguard_client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) client = Swarm(client=obiguard_client) def get_weather(location) -> str: return "{'temp':67, 'unit':'F'}" agent = Agent( name="Agent", instructions="You are a helpful agent.", functions=[get_weather], ) messages = [{"role": "user", "content": "What's the weather in NYC?"}] response = client.run(agent=agent, messages=messages) print(response.messages[-1]["content"]) ``` > The current temperature in New York City is 67°F. ## Enabling Obiguard Features By routing your OpenAI Swarm requests through Obiguard, you get access to the following production-grade features:

Call various LLMs like Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, and AWS Bedrock with minimal code changes.

Get comprehensive logs of agent interactions, including cost, tokens used, response time, and function calls. Send custom metadata for better analytics.

Access detailed logs of agent executions, function calls, and interactions. Debug and optimize your agents effectively.

Implement budget limits, role-based access control, and audit trails for your agent operations.

 ### 1. Interoperability - Calling Different LLMs When building with Swarm, you might want to experiment with different LLMs or use specific providers for different agent tasks. Obiguard makes this seamless - you can switch between OpenAI, Anthropic, Gemini, Mistral, or cloud providers without changing your agent code. Instead of managing multiple API keys and provider-specific configurations, Obiguard's Virtual Keys give you a single point of control. Here's how you can use different LLMs with your Swarm agents: ```python theme={null} obiguard_client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) client = Swarm(client=obiguard_client) ``` ```python theme={null} client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) client = Swarm(client=obiguard_client) ``` ### 2. [Observability - Understand Your Agents](/product/observability) Building agents is the first step - but how do you know they're working effectively? Obiguard provides comprehensive visibility into your agent operations through multiple lenses: **Metrics Dashboard**: Track 40+ key performance indicators like: * Cost per agent interaction * Response times and latency * Token usage and efficiency * Success/failure rates * Cache hit rates ### 3. [Logs and Traces](/product/observability/logs) Logs are essential for understanding agent behavior, diagnosing issues, and improving performance. They provide a detailed record of agent activities and tool use, which is crucial for debugging and optimizing processes. Access a dedicated section to view records of agent executions, including parameters, outcomes, function calls, and errors. Filter logs based on multiple parameters such as trace ID, model, tokens used, and metadata. ### 4. [Security & Compliance - Enterprise-Ready Controls](/product/enterprise-offering/security-Obiguard) When deploying agents in production, security is crucial. Obiguard provides enterprise-grade security features: Set and monitor spending limits per Virtual Key. Get alerts before costs exceed thresholds. Control who can access what. Assign roles and permissions for your team members. Track all changes and access. Know who modified agent settings and when. Configure data retention and processing policies to meet your compliance needs. Configure these settings in the [Obiguard Dashboard](https://app.obiguard.ai). # Pydantic AI Source: https://docs.obiguard.ai/integrations/agents/pydantic-ai Use Obiguard with PydanticAI to take your AI Agents to production ## Introduction PydanticAI is a Python agent framework designed to make it less painful to build production-grade applications with Generative AI. It brings the same ergonomic design and developer experience to GenAI that FastAPI brought to web development. Obiguard enhances PydanticAI with production-readiness features, turning your experimental agents into robust systems by providing: * **Complete observability** of every agent step, tool use, and interaction * **Cost tracking and optimization** to manage your AI spend * **Access to 200+ LLMs** through a single integration * **Guardrails** to keep agent behavior safe and compliant Learn more about PydanticAI's core concepts and features ### Installation & Setup ```bash theme={null} pip install -U pydantic-ai obiguard ``` Create a Obiguard API key with optional budget/rate limits from the [Obiguard dashboard](https://app.obiguard.ai/). You can attach configurations for reliability, caching, and more to this key. For a simple setup, first configure the Obiguard client that will be used with PydanticAI: ```python theme={null} from obiguard import AsyncObiguard # Set up Obiguard client with appropriate metadata for tracking obiguard_client = AsyncObiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` **What are Virtual Keys?** Virtual keys in Obiguard securely store your LLM provider API keys (OpenAI, Anthropic, etc.) in an encrypted vault. They allow for easier key rotation and budget management. [Learn more about virtual keys here](/virtual-keys). After setting up your Obiguard client, you can integrate it with PydanticAI by connecting it to a model provider: ```python theme={null} from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider # Connect Obiguard client to a PydanticAI model via provider agent = Agent( model=OpenAIModel( model_name="gpt-4o", provider=OpenAIProvider(openai_client=obiguard_client), ), system_prompt="You are a helpful assistant." ) ``` ## Basic Agent Implementation Let's create a simple structured output agent with PydanticAI and Obiguard. This agent will respond to a query about Formula 1 and return structured data: ```python theme={null} from obiguard import AsyncObiguard from pydantic import BaseModel, Field from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider # Set up Obiguard client with tracing and metadata obiguard_client = AsyncObiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) # Define structured output using Pydantic class F1GrandPrix(BaseModel): gp_name: str = Field(description="Grand Prix name, e.g. `Emilia Romagna Grand Prix`") year: int = Field(description="The year of the Grand Prix") constructor_winner: str = Field(description="The winning constructor of the Grand Prix") podium: list[str] = Field(description="Names of the podium drivers (1st, 2nd, 3rd)") # Create the agent with structured output type f1_gp_agent = Agent[None, F1GrandPrix]( model=OpenAIModel( model_name="gpt-4o", provider=OpenAIProvider(openai_client=obiguard_client), ), output_type=F1GrandPrix, system_prompt="Assist the user by providing data about the specified Formula 1 Grand Prix" ) # Run the agent async def main(): result = await f1_gp_agent.run("Las Vegas 2023") print(result.output) if __name__ == "__main__": import asyncio asyncio.run(main()) ``` The output will be a structured `F1GrandPrix` object with all fields properly typed and validated: ```json theme={null} gp_name='Las Vegas Grand Prix' year=2023 constructor_winner='Red Bull Racing' podium=['Max Verstappen', 'Charles Leclerc', 'Sergio Pérez'] ``` You can also use the synchronous API if preferred: ```python theme={null} result = f1_gp_agent.run_sync("Las Vegas 2023") print(result.output) ``` ## Advanced Features ### Working with Images PydanticAI supports multimodal inputs including images. Here's how to use Obiguard with a vision model: ```python theme={null} from obiguard import AsyncObiguard from pydantic_ai import Agent, ImageUrl from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider # Set up Obiguard client obiguard_client = AsyncObiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) # Create a vision-capable agent vision_agent = Agent( model=OpenAIModel( model_name="gpt-4o", # Vision-capable model provider=OpenAIProvider(openai_client=obiguard_client), ), system_prompt="Analyze images and provide detailed descriptions." ) # Process an image result = vision_agent.run_sync([ 'What company is this logo from?', ImageUrl(url='https://iili.io/3Hs4FMg.png'), ]) print(result.output) ``` Visit your Obiguard dashboard to see detailed logs of this image analysis request, including token usage and costs. ### Tools and Tool Calls PydanticAI provides a powerful tools system that integrates seamlessly with Obiguard. Here's how to create an agent with tools: ```python theme={null} import random from obiguard import AsyncObiguard from pydantic_ai import Agent, RunContext from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider # Set up Obiguard client obiguard_client = AsyncObiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) # Create an agent with dependency injection (player name) dice_agent = Agent( model=OpenAIModel( model_name="gpt-4o", provider=OpenAIProvider(openai_client=obiguard_client), ), deps_type=str, # Dependency type (player name as string) system_prompt=( "You're a dice game host. Roll the die and see if it matches " "the user's guess. If so, tell them they're a winner. " "Use the player's name in your response." ), ) # Define a plain tool (no context needed) @dice_agent.tool_plain def roll_die() -> str: """Roll a six-sided die and return the result.""" return str(random.randint(1, 6)) # Define a tool that uses the dependency @dice_agent.tool def get_player_name(ctx: RunContext[str]) -> str: """Get the player's name.""" return ctx.deps # Run the agent dice_result = dice_agent.run_sync('My guess is 4', deps='Anne') print(dice_result.output) ``` Obiguard logs each tool call separately, allowing you to analyze the full execution path of your agent, including both LLM calls and tool invocations. ### Multi-agent Applications PydanticAI excels at creating multi-agent systems where agents can call each other. Here's how to integrate Obiguard with a multi-agent setup: This multi-agent system uses three specialized agents: `search_agent` - Orchestrates the flow and validates flight selections `extraction_agent` - Extracts structured flight data from raw text `seat_preference_agent` - Interprets user's seat preferences With Obiguard integration, you get: * Unified tracing across all three agents * Token and cost tracking for the entire workflow * Ability to set usage limits across the entire system * Observability of both AI and human interaction points Here's a diagram of how these agents interact: ```mermaid theme={null} graph TD A[START] --> B[search agent] B --> C[extraction agent] C --> B B --> D[human confirm] D --> E[find seat function] B --> F[FAILED] E --> G[human seat choice] G --> H[find seat agent] H --> E E --> I[buy flights] I --> J[SUCCESS] ``` ```python [expandable] theme={null} import datetime from dataclasses import dataclass from typing import Literal from pydantic import BaseModel, Field from rich.prompt import Prompt from pydantic_ai import Agent, ModelRetry, RunContext from pydantic_ai.messages import ModelMessage from pydantic_ai.usage import Usage, UsageLimits from obiguard import AsyncObiguard # Set up Obiguard clients with shared trace ID for connected tracing obiguard_client = AsyncObiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) # Define structured output types class FlightDetails(BaseModel): """Details of the most suitable flight.""" flight_number: str price: int origin: str = Field(description='Three-letter airport code') destination: str = Field(description='Three-letter airport code') date: datetime.date class NoFlightFound(BaseModel): """When no valid flight is found.""" class SeatPreference(BaseModel): row: int = Field(ge=1, le=30) seat: Literal['A', 'B', 'C', 'D', 'E', 'F'] class Failed(BaseModel): """Unable to extract a seat selection.""" # Dependencies for flight search @dataclass class Deps: web_page_text: str req_origin: str req_destination: str req_date: datetime.date # This agent is responsible for controlling the flow of the conversation from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider search_agent = Agent[Deps, FlightDetails | NoFlightFound]( model=OpenAIModel( model_name="gpt-4o", provider=OpenAIProvider(openai_client=obiguard_client), ), output_type=FlightDetails | NoFlightFound, # type: ignore retries=4, system_prompt=( 'Your job is to find the cheapest flight for the user on the given date. ' ), instrument=True, # Enable instrumentation for better tracing ) # This agent is responsible for extracting flight details from web page text extraction_agent = Agent( model=OpenAIModel( model_name="gpt-4o", provider=OpenAIProvider(openai_client=obiguard_client), ), output_type=list[FlightDetails], system_prompt='Extract all the flight details from the given text.', ) # This agent is responsible for extracting the user's seat selection seat_preference_agent = Agent[None, SeatPreference | Failed]( model=OpenAIModel( model_name="gpt-4o", provider=OpenAIProvider(openai_client=obiguard_client), ), output_type=SeatPreference | Failed, # type: ignore system_prompt=( "Extract the user's seat preference. " 'Seats A and F are window seats. ' 'Row 1 is the front row and has extra leg room. ' 'Rows 14, and 20 also have extra leg room. ' ), ) @search_agent.tool async def extract_flights(ctx: RunContext[Deps]) -> list[FlightDetails]: """Get details of all flights.""" # Pass the usage to track nested agent calls result = await extraction_agent.run(ctx.deps.web_page_text, usage=ctx.usage) return result.output @search_agent.output_validator async def validate_output( ctx: RunContext[Deps], output: FlightDetails | NoFlightFound ) -> FlightDetails | NoFlightFound: """Procedural validation that the flight meets the constraints.""" if isinstance(output, NoFlightFound): return output errors: list[str] = [] if output.origin != ctx.deps.req_origin: errors.append( f'Flight should have origin {ctx.deps.req_origin}, not {output.origin}' ) if output.destination != ctx.deps.req_destination: errors.append( f'Flight should have destination {ctx.deps.req_destination}, not {output.destination}' ) if output.date != ctx.deps.req_date: errors.append(f'Flight should be on {ctx.deps.req_date}, not {output.date}') if errors: raise ModelRetry('\n'.join(errors)) else: return output # Sample flight data (in a real application, this would be from a web scraper) flights_web_page = """ 1. Flight SFO-AK123 - Price: $350 - Origin: San Francisco International Airport (SFO) - Destination: Ted Stevens Anchorage International Airport (ANC) - Date: January 10, 2025 2. Flight SFO-AK456 - Price: $370 - Origin: San Francisco International Airport (SFO) - Destination: Fairbanks International Airport (FAI) - Date: January 10, 2025 ... more flights ... """ # Main application flow async def main(): # Restrict how many requests this app can make to the LLM usage_limits = UsageLimits(request_limit=15) deps = Deps( web_page_text=flights_web_page, req_origin='SFO', req_destination='ANC', req_date=datetime.date(2025, 1, 10), ) message_history: list[ModelMessage] | None = None usage: Usage = Usage() # Run the agent until a satisfactory flight is found while True: result = await search_agent.run( f'Find me a flight from {deps.req_origin} to {deps.req_destination} on {deps.req_date}', deps=deps, usage=usage, message_history=message_history, usage_limits=usage_limits, ) if isinstance(result.output, NoFlightFound): print('No flight found') break else: flight = result.output print(f'Flight found: {flight}') answer = Prompt.ask( 'Do you want to buy this flight, or keep searching? (buy/*search)', choices=['buy', 'search', ''], show_choices=False, ) if answer == 'buy': seat = await find_seat(usage, usage_limits) await buy_tickets(flight, seat) break else: message_history = result.all_messages( output_tool_return_content='Please suggest another flight' ) async def find_seat(usage: Usage, usage_limits: UsageLimits) -> SeatPreference: message_history: list[ModelMessage] | None = None while True: answer = Prompt.ask('What seat would you like?') result = await seat_preference_agent.run( answer, message_history=message_history, usage=usage, usage_limits=usage_limits, ) if isinstance(result.output, SeatPreference): return result.output else: print('Could not understand seat preference. Please try again.') message_history = result.all_messages() async def buy_tickets(flight_details: FlightDetails, seat: SeatPreference): print(f'Purchasing flight {flight_details=!r} {seat=!r}...') ``` Obiguard preserves all the type safety of PydanticAI while adding production monitoring and reliability. ## Production Features ### 1. Enhanced Observability Obiguard provides comprehensive observability for your PydanticAI agents, helping you understand exactly what's happening during each execution. Traces provide a hierarchical view of your agent's execution, showing the sequence of LLM calls, tool invocations, and state transitions. Obiguard logs every interaction with LLMs, including: * Complete request and response payloads * Latency and token usage metrics * Cost calculations * Tool calls and function executions All logs can be filtered by metadata, trace IDs, models, and more, making it easy to debug specific agent runs. Obiguard provides built-in dashboards that help you: * Track cost and token usage across all agent runs * Analyze performance metrics like latency and success rates * Identify bottlenecks in your agent workflows * Compare different agent configurations and LLMs You can filter and segment all metrics by custom metadata to analyze specific agent types, user groups, or use cases. ### 2. Guardrails for Safe Agents Guardrails ensure your PydanticAI agents operate safely and respond appropriately in all situations. **Why Use Guardrails?** PydanticAI agents can experience various failure modes: * Generating harmful or inappropriate content * Leaking sensitive information like PII * Hallucinating incorrect information * Generating outputs in incorrect formats While PydanticAI provides type safety for outputs, Obiguard's guardrails add additional protections for both inputs and outputs. Obiguard's guardrails can: * Detect and redact PII in both inputs and outputs * Filter harmful or inappropriate content * Validate response formats against schemas * Check for hallucinations against ground truth * Apply custom business logic and rules Explore Obiguard's guardrail features to enhance agent safety. ### 3. Model Interoperability PydanticAI supports multiple LLM providers, and Obiguard extends this capability by providing access to over 200 LLMs through a unified interface. You can easily switch between different models without changing your core agent logic: Obiguard provides access to LLMs from providers including: * OpenAI (GPT-4o, GPT-4 Turbo, etc.) * Anthropic (Claude 3.5 Sonnet, Claude 3 Opus, etc.) * Mistral AI (Mistral Large, Mistral Medium, etc.) * Google Vertex AI (Gemini 1.5 Pro, etc.) * Cohere (Command, Command-R, etc.) * AWS Bedrock (Claude, Titan, etc.) * Local/Private Models See the full list of LLM providers supported by Obiguard. ## Set Up Enterprise Governance for PydanticAI **Why Enterprise Governance?** If you are using PydanticAI inside your organization, you need to consider several governance aspects: * **Cost Management**: Controlling and tracking AI spending across teams * **Access Control**: Managing which teams can use specific models * **Usage Analytics**: Understanding how AI is being used across the organization * **Security & Compliance**: Maintaining enterprise security standards * **Reliability**: Ensuring consistent service across all users Obiguard adds a comprehensive governance layer to address these enterprise needs. ### Enterprise Features Now Available **Your PydanticAI integration now has:** * Departmental budget controls * Model access governance * Usage tracking & attribution * Security guardrails * Reliability features ## Frequently Asked Questions Obiguard adds production-readiness to PydanticAI through comprehensive observability (traces, logs, metrics), reliability features (fallbacks, retries, caching), and access to 200+ LLMs through a unified interface. This makes it easier to debug, optimize, and scale your agent applications, all while preserving PydanticAI's strong type safety. Yes! Obiguard integrates seamlessly with existing PydanticAI applications. You just need to replace your client initialization code with the Obiguard-enabled version. The rest of your agent code remains unchanged and continues to benefit from PydanticAI's strong typing. Obiguard supports all PydanticAI features, including structured outputs, tool use, multi-agent systems, and more. It adds observability and reliability without limiting any of the framework's functionality. Yes, Obiguard allows you to use a consistent `trace_id` across multiple agents and requests to track the entire workflow. This is especially useful for multi-agent systems where you want to understand the full execution path. Yes! Obiguard uses your own API keys for the various LLM providers. It securely stores them as virtual keys, allowing you to easily manage and rotate keys without changing your code. ## Resources

Official PydanticAI documentation

Official Obiguard documentation

 # Overview Source: https://docs.obiguard.ai/integrations/ai-apps [//]: # "" [//]: # [//]: # "" [//]: # [//]: # "" [//]: # [//]: # "" [//]: # "" [//]: # [//]: # [//]: # "" [//]: # [//]: # [//]: # "" [//]: # [//]: # [//]: # "" [//]: # [//]: # [//]: # "" [//]: # [//]: # "" [//]: # [//]: # "" # Integrations Source: https://docs.obiguard.ai/integrations/ecosystem # n8n Source: https://docs.obiguard.ai/integrations/libraries/n8n Add usage tracking, cost controls, and security guardrails to your n8n workflows n8n is a workflow automation platform that helps you connect different systems and automate tasks. While n8n delivers robust automation capabilities, Obiguard adds essential enterprise controls for production deployments: * **Unified AI Gateway** - Single interface for 1600+ LLMs with API key management. (not just OpenAI & Anthropic) * **Centralized AI observability**: Real-time usage tracking for 40+ key metrics and logs for every request * **Governance** - Real-time spend tracking in your n8n setup * **Security Guardrails** - PII detection, content filtering, and compliance controls This guide will walk you through integrating Obiguard with n8n and setting up essential enterprise features including usage tracking, access controls, and budget management. # 1. Setting up Obiguard Obiguard allows you to use 1600+ LLMs with your n8n setup, w ith minimal configuration required. Let's set up the core components in Obiguard that you'll need for integration. You can choose to create a guardrail policy to protect your data and ensure compliance with organizational policies. Add guardrail validators on your LLM inputs and output to govern your LLM usage. Virtual Keys are Obiguard's secure way to manage your LLM provider API keys. Think of them like disposable credit cards for your LLM API keys. To create a virtual key: Go to [Virtual Keys](/virtual-keys) in the Obiguard dashboard. Select the guardrail policy and your LLM provider. Save and copy the virtual key ID Save your virtual key ID - you'll need it for the next step. # 2. Integrate Obiguard with n8n Now that you have your Obiguard components set up, let's connect them to n8n. Since Obiguard provides OpenAI API compatibility, integration is straightforward and requires just a few configuration steps in your n8n workflow. You need your Obiguard Virtual Key from [Step 1](#Setting-up-obiguard) before going further. 1. In your n8n workflow, add the OpenAI node where you want to use an LLM 2. Configure the OpenAI credentials with the following settings: * API Key: Your Obiguard virtual key from the setup * Base URL: `https://gateway.obiguard.ai/v1` When saving your Obiguard credentials in n8n, you may encounter an "Internal Server Error" or connection warning. This happens because n8n attempts to fetch available models from the API, but Obiguard doesn't expose a models endpoint in the same way OpenAI does. Despite this warning, your credentials are saved properly and will work in your workflows. 3. In your workflow, configure the OpenAI node to use your preferred model * The model parameter in your config will override the default model in your n8n workflow It is recommended that you define a comprehensive config in Obiguard with your preferred LLM settings. This allows you to maintain all LLM settings in one place. Make sure your virtual key has sufficient budget and rate limits for your expected usage. Also use the complete model name given by the provider. You can monitor your requests and usage in the [Obiguard Logs](https://app.obiguard.ai). ## Obiguard Features ### 1. Comprehensive Metrics Using Obiguard you can track 40+ key metrics including cost, token usage, response time, and performance across all your LLM providers in real time. You can also filter these metrics based on custom metadata that you can set in your configs. Learn more about metadata here. ### 2. Advanced Logs Obiguard's logging dashboard provides detailed logs for every request made to your LLMs. These logs include: * Complete request and response tracking * Metadata tags for filtering * Cost attribution and much more... ### 3. Unified Access to 1600+ LLMs You can easily switch between 1600+ LLMs. Call various LLMs such as Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, AWS Bedrock, and many more by simply passing your Obiguard virtual key in the `obiguard_api_key` field of your requests. ### 4. Advanced Guardrails Protect your Project's data and enhance reliability with real-time checks on LLM inputs and outputs. Leverage guardrails to: * Prevent sensitive data leaks * Enforce compliance with organizational policies * PII detection and masking * Content filtering * Custom security rules * Data compliance checks Implement real-time protection for your LLM interactions with automatic detection and filtering of sensitive content, PII, and custom security rules. Enable comprehensive data protection while maintaining compliance with organizational policies. # FAQs Yes! You can create multiple Virtual Keys (one for each provider) and attach them to a single request. This config can then be connected to your API key, allowing you to use multiple providers through a single API key. You can control model access by: 1. Setting specific models in your Obiguard configs 2. Creating different Virtual Keys with different provider access 3. Using model-specific configs attached to different API keys 4. Implementing conditional routing based on workflow metadata ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Overview Source: https://docs.obiguard.ai/integrations/llms Obiguard integrates seamlessly with leading LLM providers and orchestration frameworks. ## Supported AI Providers OpenAI Anthropic Azure OpenAI AWS Bedrock Cohere Google Gemini Vertex AI Perplexity AI AI21 Anyscale [//]: # "" [//]: # [//]: # "" [//]: # " \"Dashscope\"" [//]: # "" [//]: # "" [//]: # " \"Deepbricks\"" [//]: # "" [//]: # [//]: # "" [//]: # " \"DeepInfra\"" [//]: # "" DeepSeek [//]: # "" [//]: # " \"Fireworks" [//]: # "" Github [//]: # "" [//]: # " \"Google" [//]: # "" Groq [//]: # "" [//]: # " \"Jina" [//]: # "" [//]: # [//]: # "" [//]: # " \"Lambda\"" [//]: # "" [//]: # [//]: # "" [//]: # " \"Lemonfox" [//]: # "" Lepton AI [//]: # "" [//]: # " \"Lingyi" [//]: # "" LocalAI Mistral AI [//]: # [//]: # "" [//]: # " \"MonsterAPI\"" [//]: # "" [//]: # [//]: # "" [//]: # " \"Moonshot\"" [//]: # "" [//]: # [//]: # "" [//]: # " \"nCompass\"" [//]: # "" Nebius Nomic AI [//]: # "" [//]: # " \"Nscale\"" [//]: # "" Ollama Openrouter [//]: # "" [//]: # " \"Predibase\"" [//]: # "" [//]: # [//]: # "" [//]: # " \"Recraft\"" [//]: # "" [//]: # [//]: # "" [//]: # " \"Siliconflow\"" [//]: # "" Snowflake Stability AI Together AI Triton [//]: # "" [//]: # " \"Upstage\"" [//]: # "" [//]: # "" [//]: # " \"Voyage-AI\"" [//]: # "" [//]: # [//]: # "" [//]: # " \"Workers" [//]: # "" [//]: # [//]: # "" [//]: # " \"ZhipuAI\"" [//]: # "" ## Endpoints Supported For a detailed breakdown of supported endpoints and features per provider, refer to the table below: Have a suggestion for an integration with Obiguard? Drop a message on [support@obiguard.com](mailto:support@obiguard.com). # AI21 Source: https://docs.obiguard.ai/integrations/llms/ai21 Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [AI21](https://ai21.com). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. **ai21** ## Obiguard SDK Integration with AI21 Models Obiguard provides a consistent API to interact with models from various providers. To integrate AI21 with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with AI21 AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use AI21 with Obiguard, [get your API key from here](https://studio.ai21.com/account/api-key), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for AI21 ) ``` ### 3. Invoke Chat Completions with AI21 Use the Obiguard instance to send requests to AI21. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'jamba-1-5-large' ) print(completion) ``` The complete list of features supported in the SDK are available on the link below. # Anthropic Source: https://docs.obiguard.ai/integrations/llms/anthropic Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Anthropic's Claude APIs](https://docs.anthropic.com/claude/reference/getting-started-with-the-api). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `anthropic` ## Obiguard SDK Integration with Anthropic Obiguard provides a consistent API to interact with models from various providers. To integrate Anthropic with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Anthropic's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Anthropic with Obiguard, [get your Anthropic API key from here](https://console.anthropic.com/settings/keys), then add it to Obiguard to create your Anthropic virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="ANTHROPIC_API_KEY", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anthropic" obiguard_api_key="OBIGUARD_API_KEY" ) ) ``` ### 3. Invoke Chat Completions with Anthropic Use the Obiguard instance to send requests to Anthropic. You can also override the virtual key directly in the API call if needed. ```python theme={null} chat_completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'claude-3-opus-20240229', max_tokens=250 # Required field for Anthropic ) print(chat_completion.choices[0].message.content) ``` ```python theme={null} chat_completion = client.chat.completions.create( messages = [{ "role": 'user', "content": 'Say this is a test' }], model = 'claude-3-opus-20240229', max_tokens = 250 ) print(chat_completion.choices[0].message.content) ``` ## How to Use Anthropic System Prompt With Obiguard, we make Anthropic models interoperable with the OpenAI schema and SDK methods. So, instead of passing the `system` prompt separately, you can pass it as part of the `messages` body, similar to OpenAI: ```python theme={null} completion = client.chat.completions.create( messages= [ { "role": 'system', "content": 'Your system prompt' }, { "role": 'user', "content": 'Say this is a test' } ], model= 'claude-3-opus-20240229', max_tokens=250 # Required field for Anthropic ) print(completion.choices) ``` ## Vision Chat Completion Usage Obiguard's multimodal Gateway fully supports Anthropic's vision models `claude-3-sonnet`, `claude-3-haiku`, `claude-3-opus`, and the latest `claude-3.5-sonnet`. Obiguard follows the OpenAI schema, which means you can send your image data to Anthropic in the same format as OpenAI. * Anthropic ONLY accepts `base64` -encoded images. Unlike OpenAI, it **does not** support `image URLs`. * With Obiguard, you can use the same format to send base64-encoded images to both Anthropic and OpenAI models. Here's an example using Anthropic `claude-3.5-sonnet` model ```python theme={null} import base64 import httpx from obiguard import Obiguard # Fetch and encode the image image_url = "https://upload.wikimedia.org/wikipedia/commons/a/a7/Camponotus_flavomarginatus_ant.jpg" image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8") client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) # Create the request response = client.chat.completions.create( model="claude-3-5-sonnet-20240620", messages=[ { "role": "system", "content": "You are a helpful assistant, who describes imagse" }, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": f"data:image/jpeg;base64,{image_data}" } } ] } ], max_tokens=1400, ) print(response) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="OBIGUARD_API_KEY" ) ) response = openai.chat.completions.create( model="claude-3-5-sonnet-20240620", messages=[ { "role": "system", "content": "You are a helpful assistant, who describes imagse" }, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": f"data:image/jpeg;base64,{base_64_encoded_image}" } } ] } ], max_tokens=1400, ) print(response) ``` ```sh theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anthropic" \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -d '{ "model": "claude-3-5-sonnet-20240620", "messages": [ { "role": "system", "content": "You are a helpful assistant who describes images." }, { "role": "user", "content": [ { "type": "text", "text": "What's in this image?" }, { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,BASE64_IMAGE_DATA" } } ] } ], "max_tokens": 300 }' ``` To prompt with pdfs, simply update the `url` field inside the `image_url` object to this pattern: `data:application/pdf;base64,BASE64_PDF_DATA` #### [API Reference](#vision-chat-completion-usage) On completion, the request will get logged in Obiguard where any image inputs or outputs can be viewed. Obiguard will automatically render the base64 images to help you debug any issues quickly. ## Calude PDF Support Anthropic Claude can now process PDFs to extract text, analyze charts, and understand visual content from documents. With Obiguard, you can seamlessly integrate this capability into your applications using the familiar OpenAI-compatible API schema. PDF support is available on the following Claude models: * Claude 3.7 Sonnet (`claude-3-7-sonnet-20250219`) * Claude 3.5 Sonnet (`claude-3-5-sonnet-20241022`, `claude-3-5-sonnet-20240620`) * Claude 3.5 Haiku (`claude-3-5-haiku-20241022`) When using PDF support with Obiguard, be aware of these limitations: * Maximum request size: 32MB * Maximum pages per request: 100 * Format: Standard PDF (no passwords/encryption) ### Processing PDFs with Claude Currently, Obiguard supports PDF processing using base64-encoded PDF documents, following the same pattern as image handling in Claude's multimodal capabilities. ```python theme={null} from obiguard import Obiguard import base64 import httpx client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) # Fetch and encode the PDF pdf_url = "https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf" pdf_data = "data:application/pdf;base64," + base64.standard_b64encode(httpx.get(pdf_url).content).decode("utf-8") # Alternative: Load from a local file # with open("document.pdf", "rb") as f: # pdf_data = "data:application/pdf;base64," + base64.standard_b64encode(f.read()).decode("utf-8") # Create the request response = client.chat.completions.create( model="claude-3-5-sonnet-20240620", max_tokens=1024, messages=[ { "role": "system", "content": "You are a helpful document analysis assistant." }, { "role": "user", "content": [ { "type": "text", "text": "What are the key findings in this document?" }, { "type": "file", "file": { # "file_url": "https://pdfobject.com/pdf/sample.pdf", # if you want to pass a pdf file from a url "mime_type": "application/pdf", "file_data": "BASE64_PDF_DATA" } } # { # if you want to pass a plain text file # "type": "file", # "file": { # "mime_type": "text/plain", # "file_data": "This is a plain text file" # } # } ] } ] ) print(response.choices[0].message.content) ``` ```sh theme={null} # First, encode your PDF to base64 (this example uses a command line approach) # For example using curl + base64: PDF_BASE64=$(curl -s "https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf" | base64) # Alternatively, from a local file: # PDF_BASE64=$(base64 -i document.pdf) # Then make the API call with the base64-encoded PDF curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anthropic" \ -H "x-api-key: $ANTHROPIC_API_KEY" -d '{ "model": "claude-3-5-sonnet-20240620", "max_tokens": 1024, "messages": [ { "role": "system", "content": "You are a helpful document analysis assistant." }, { "role": "user", "content": [ { "type": "text", "text": "What are the key findings in this document?" }, { "type": "file", "file": { "mime_type": "application/pdf", "file_data": "BASE64_PDF_DATA" } } // { // if you want to pass a plain text file // "type": "file", // "file": { // "mime_type": "text/plain", // "file_data": "This is a plain text file" // } // } ] } ] }' ``` ### Best Practices for PDF Processing For optimal results when working with PDFs: * Place PDFs before any text content in your requests * Ensure PDFs contain standard fonts and clear, legible text * Verify that pages are properly oriented * Split large PDFs into smaller chunks when they approach size limits * Be specific in your questions to get more targeted analysis ### Calculating Costs When processing PDFs, token usage is calculated based on both text content and the visual representation of pages: * Text tokens: Typically 1,500-3,000 tokens per page, depending on content density * Image tokens: Each page converted to an image adds to the token count similar to image processing **For more info, check out this guide:** ## Prompt Caching Obiguard also works with Anthropic's new prompt caching feature and helps you save time & money for all your Anthropic requests. Refer to this guide to learn how to enable it: ## Extended Thinking (Reasoning Models) (Beta) The assistants thinking response is returned in the `response_chunk.choices[0].delta.content_blocks` array, not the `response.choices[0].message.content` string. Models like `claude-3-7-sonnet-latest` support [extended thinking](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#streaming-extended-thinking). This is similar to openai thinking, but you get the model's reasoning as it processes the request as well. Note that you will have to set [`strict_open_ai_compliance=False`](/product/ai-gateway/strict-open-ai-compliance) in the headers to use this feature. ### Single turn conversation ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) # Create the request response = client.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) # in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array # response = client.chat.completions.create( # ...same config as above but with stream: true # ) # for chunk in response: # if chunk.choices[0].delta: # content_blocks = chunk.choices[0].delta.get("content_blocks") # if content_blocks is not None: # for content_block in content_blocks: # print(content_block) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='Anthropic_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="OBIGUARD_API_KEY" ) ) response = client.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.oobiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anthropic" \ -H "x-obiguard--key: $ANTHROPIC_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "claude-3-7-sonnet-latest", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": false, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] }' ``` ### Multi turn conversation ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key strict_open_ai_compliance=False ) response = client.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='Anthropic_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anthropic", obiguard_api_key="OBIGUARD_API_KEY", strict_open_ai_compliance=False ) ) response = openai.chat.completions.create( model="claude-3-7-sonnet-latest", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anthropic" \ -H "x-obiguard-api-key: $ANTHROPIC_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "claude-3-7-sonnet-latest", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": false, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] }' ``` Extended thinking API through Obiguard is currently in beta. ## Beta features Obiguard enables support for Anthropic's beta features by including the feature name in the header: `"x-obiguard-anthropic-beta": "token-efficient-tools-2025-02-19"` For SDKs, you can specify this during client initialization or use `createHeaders` with the OpenAI SDK: ```py python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key anthropic_beta="token-efficient-tools-2025-02-19", strict_open_ai_compliance=False ) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anthropic", obiguard_api_key="OBIGUARD_API_KEY", strict_open_ai_compliance=False, anthropic_beta="token-efficient-tools-2025-02-19" ) ) ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. # Computer use tool Source: https://docs.obiguard.ai/integrations/llms/anthropic/computer-use Anthropic computer use is fully supported in Obiguard. For more information on the computer use tool, please refer to the [Anthropic documentation](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/computer-use-tool). ### Usage ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) # Create the request response = client.chat.completions.create( model="claude-4-opus-20250514", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, tools=[ { type: "computer", computer: { name: "computer_20250124", # This is the version of the tool display_width_px: 1024, display_height_px: 768, display_number: 1 } }, { type: "text_editor_20250429", name: "str_replace_based_edit_tool" }, { type: "bash_20250124", name: "bash" } ], messages=[ { "role": "user", "content": "Save a picture of a cat to my desktop." } ] ) print(response) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='Anthropic_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anthropic", obiguard_api_key="OBIGUARD_API_KEY", strict_open_ai_compliance=False ) ) response = client.chat.completions.create( model="claude-4-opus-20250514", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=False, tools=[ { type: "computer", computer: { name: "computer_20250124", # This is the version of the tool display_width_px: 1024, display_height_px: 768, display_number: 1 } }, { type: "text_editor_20250429", name: "str_replace_based_edit_tool" }, { type: "bash_20250124", name: "bash" } ], messages=[ { role: "user", content: "Save a picture of a cat to my desktop." } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anthropic" \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "claude-4-opus-20250514", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": false, "tools": [ { "type": "computer", "computer": { "name": "computer_20250124", "display_width_px": 1024, "display_height_px": 768, "display_number": 1 } }, { "type": "text_editor_20250429", "name": "str_replace_based_edit_tool" }, { "type": "bash_20250124", "name": "bash" } ], "messages": [ { "role": "user", "content": "Save a picture of a cat to my desktop." } ] }' ``` # Anyscale Source: https://docs.obiguard.ai/integrations/llms/anyscale-llama2-mistral-zephyr Integrate Anyscale endpoints with Obiguard seamlessly and make your OSS models production-ready Obiguard's suite of features - AI gateway, observability, prompt management, and continuous fine-tuning are all enabled for the OSS models (Llama2, Mistral, Zephyr, and more) available on Anyscale endpoints. Provider Slug. `anyscale` ## Obiguard SDK Integration with Anyscale ### 1. Install the Obiguard SDK ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with Anyscale Virtual Key To use Anyscale with Obiguard, [get your Anyscale API key from here](https://console.anyscale.com/v2/api-keys), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) ``` ### **3. Invoke Chat Completions with Anyscale** ```python theme={null} completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'mistralai/Mistral-7B-Instruct-v0.1' ) ``` ### Directly Using Obiguard's REST API Alternatively, you can also directly call Anyscale models through Obiguard's REST API - it works exactly the same as OpenAI API, with 2 differences: 1. You send your requests to Obiguard's complete Gateway URL `https://gateway.obiguard.ai/v1/chat/completions` 2. You have to add Obiguard specific headers. * `x-obiguard-api-key` for sending your Obiguard API or virtual Key ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $ANYSCALE_API_KEY" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: anyscale" \ -d '{ "model": "mistralai/Mistral-7B-Instruct-v0.1", "messages": [{"role": "user","content": "Hello!"}] }' ``` [List of all possible Obiguard headers](/api-reference/sdk/python#parameters). ## Using the OpenAI Python or Node SDKs for Anyscale You can also use the `baseURL` param in the standard OpenAI SDKs and make calls to Obiguard + Anyscale directly from there. Like the Rest API example, you are only required to change the `baseURL` and add `defaultHeaders` to your instance. You can use the Obiguard SDK to make it simpler: ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders anyscale = OpenAI( api_key="ANYSCALE_API_KEY", # defaults to os.environ.get("OPENAI_API_KEY") base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anyscale", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) chat_complete = anyscale.chat.completions.create( model="mistralai/Mistral-7B-Instruct-v0.1", messages=[{"role": "user", "content": "Say this is a test"}], ) ``` This request will be automatically logged by Obiguard. You can view this in your logs dashboard. Obiguard logs the tokens utilized, execution time, and cost for each request. Additionally, you can delve into the details to review the precise request and response data. *** ## Advanced Use Cases ### Streaming Responses Obiguard supports streaming responses using Server Sent Events (SSE). ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders anyscale = OpenAI( api_key="ANYSCALE-API-KEY", # defaults to os.environ.get("OPENAI_API_KEY") base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="anyscale", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) chat_complete = anyscale.chat.completions.create( model="mistralai/Mistral-7B-Instruct-v0.1", messages=[{"role": "user", "content": "Say this is a test"}], stream=True ) for chunk in chat_complete: print(chunk.choices[0].delta.content, end="", flush=True) ``` ### Obiguard Features Obiguard supports the complete host of it's functionality via the OpenAI SDK so you don't need to migrate away from it. # AWS SageMaker Source: https://docs.obiguard.ai/integrations/llms/aws-sagemaker Route to your AWS Sagemaker models through Obiguard Sagemaker allows users to host any ML model on their own AWS infrastructure. With Obiguard you can manage/restrict access, log requests, and more. Provider Slug. `sagemaker` ## Obiguard SDK Integration with AWS Sagemaker ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Sagemaker's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with a Virtual Key There are multiple ways to integrate Sagemaker with Obiguard. You can use your AWS credentials, or use an assumed role. In this example we will create a virtual key and use it to interact with Sagemaker. This helps you restrict access (specific models, few endpoints, etc). Here's how to find your AWS credentials:
Use your `AWS Secret Access Key`, `AWS Access Key Id`, and `AWS Region` to create your Virtual key.

[**Integration Guide**](/integrations/llms/aws-bedrock#how-to-find-your-aws-credentials)
Take your `AWS Assumed Role ARN` and `AWS Region` to create the virtual key.


 Create a virtual key in the Obiguard dashboard in the virtual keys section. You can select sagemaker as the provider, and fill in deployment details. Initialize the Obiguard SDK with the virtual key. (If you are using the REST API, skip to next step) ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your Sagemaker Virtual Key ) ``` ### 3. Invoke the Sagemaker model ```python theme={null} response = client.post( url="endpoints/{endpoint_name}/invocations", # You can pass any key value pair required by the model, apart from `url`, they are passed as kwargs to the Sagemaker endpoint inputs="my_custom_value", my_custom_key="my_custom_value", ) print(response) ``` ```cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/endpoints/{endpoint_name}/invocations' \ --header 'x-obiguard-virtual-key: {VIRTUAL_KEY}' \ --header 'x-obiguard-api-key: {OBIGUARD_API_KEY}' \ --header 'Content-Type: application/json' \ --data '{ # You can pass any key value pair required by the model, they are passed as kwargs to the Sagemaker endpoint "inputs": "my_custom_value", "my_custom_key": "my_custom_value" }' ``` ## Making Requests without Virtual Keys If you do not want to add your AWS details to Obiguard vault, you can also directly pass them while instantiating the Obiguard client. These are the supported headers/parameters for Sagemaker (Not required if you're using a virtual key): | Node SDK | Python SDK | REST Headers | | -------------------------------- | -------------------------------------- | --------------------------------------------------- | | awsAccessKeyId | aws\_access\_key\_id | x-obiguard-aws-access-key-id | | awsSecretAccessKey | aws\_secret\_access\_key | x-obiguard-aws-secret-access-key | | awsRegion | aws\_region | x-obiguard-aws-region | | awsSessionToken | aws\_session\_token | x-obiguard-aws-session-token | | sagemakerCustomAttributes | sagemaker\_custom\_attributes | x-obiguard-amzn-sagemaker-custom-attributes | | sagemakerTargetModel | sagemaker\_target\_model | x-obiguard-amzn-sagemaker-target-model | | sagemakerTargetVariant | sagemaker\_target\_variant | x-obiguard-amzn-sagemaker-target-variant | | sagemakerTargetContainerHostname | sagemaker\_target\_container\_hostname | x-obiguard-amzn-sagemaker-target-container-hostname | | sagemakerInferenceId | sagemaker\_inference\_id | x-obiguard-amzn-sagemaker-inference-id | | sagemakerEnableExplanations | sagemaker\_enable\_explanations | x-obiguard-amzn-sagemaker-enable-explanations | | sagemakerInferenceComponent | sagemaker\_inference\_component | x-obiguard-amzn-sagemaker-inference-component | | sagemakerSessionId | sagemaker\_session\_id | x-obiguard-amzn-sagemaker-session-id | | sagemakerModelName | sagemaker\_model\_name | x-obiguard-amzn-sagemaker-model-name | ### Example ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="sagemaker", aws_region="us-east-1", # Replace with your AWS region aws_access_key_id="AWS_ACCESS_KEY_ID", # Replace with your AWS access key id aws_secret_access_key="AWS_SECRET_ACCESS_KEY", # Replace with your AWS secret access key amzn_sagemaker_inference_component="SAGEMAKER_INFERENCE_COMPONENT" # Replace with your Sagemaker inference component ) response = client.post( url="endpoints/{endpoint_name}/invocations", # You can pass any key value pair required by the model, apart from `url`, they are passed as kwargs to the Sagemaker endpoint inputs="my_custom_value", my_custom_key="my_custom_value" ) print(response) ``` ```sh theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: sagemaker" \ -H "x-obiguard-aws-access-key-id: $AWS_ACCESS_KEY_ID" \ -H "x-obiguard-aws-secret-access-key: $AWS_SECRET_ACCESS_KEY" \ -H "x-obiguard-aws-region: $AWS_REGION" \ -H "x-obiguard-amzn-sagemaker-inference-component: $SAGEMAKER_INFERENCE_COMPONENT" \ -d '{ # You can pass any key value pair apart from `url` required by the model, they are passed as kwargs to the Sagemaker endpoint "inputs": "my_custom_value", "my_custom_key": "my_custom_value" }' ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. # Azure AI Foundry Source: https://docs.obiguard.ai/integrations/llms/azure-foundry Learn how to integrate Azure AI Foundry with Obiguard to access a wide range of AI models with enhanced observability and reliability features. Azure AI Foundry provides a unified platform for enterprise AI operations, model building, and application development. With Obiguard, you can seamlessly integrate with various models available on Azure AI Foundry and take advantage of features like observability, prompt management, fallbacks, and more. ## Understanding Azure AI Foundry Deployments Azure AI Foundry offers three different ways to deploy models, each with unique endpoints and configurations: 1. **AI Services**: Azure-managed models accessed through Azure AI Services endpoints 2. **Managed**: User-managed deployments running on dedicated Azure compute resources 3. **Serverless**: Seamless, scalable deployment without managing infrastructure You can learn more about the Azure AI Foundry deployment [here](https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/deployments-overview). If you're specifically looking to use OpenAI models on Azure, you might want to use [Azure OpenAI](/integrations/llms/azure-openai) instead, which is optimized for OpenAI models. ## Integrate To integrate Azure AI Foundry with Obiguard, you'll need to create a virtual key. Virtual keys securely store your Azure AI Foundry credentials in Obiguard's vault, allowing you to use a simple identifier in your code instead of handling sensitive authentication details directly. Navigate to the Virtual Keys section in Obiguard and select "Azure AI Foundry" as your provider. ## Creating Your Virtual Key You can create a virtual key for Azure AI Foundry using one of three authentication methods. Each method requires different information from your Azure deployment: The recommended authentication mode using API Keys: Required parameters: * **API Key**: Your Azure AI Foundry API key * **Azure Foundry URL**: The base endpoint URL for your deployment, formatted according to your deployment type: * For AI Services: `https://your-resource-name.services.ai.azure.com/models` * For Managed: `https://your-model-name.region.inference.ml.azure.com/score` * For Serverless: `https://your-model-name.region.models.ai.azure.com` * **Azure API Version**: The API version to use (e.g., "2024-05-01-preview"). This is required if you have api version in your deployment url. For example: * If your URL is `https://mycompany-ai.westus2.services.ai.azure.com/models?api-version=2024-05-01-preview`, the API version is `2024-05-01-preview` * **Azure Deployment Name**: (Optional) Required only when a single resource contains multiple deployments. For managed Azure deployments: Required parameters: * **Azure Managed ClientID**: Your managed client ID * **Azure Foundry URL**: The base endpoint URL for your deployment, formatted according to your deployment type: * For AI Services: `https://your-resource-name.services.ai.azure.com/models` * For Managed: `https://your-model-name.region.inference.ml.azure.com/score` * For Serverless: `https://your-model-name.region.models.ai.azure.com` * **Azure API Version**: The API version to use (e.g., "2024-05-01-preview"). This is required if you have api version in your deployment url. **Examples:** * If your URL is `https://mycompany-ai.westus2.services.ai.azure.com/models?api-version=2024-05-01-preview`, the API version is `2024-05-01-preview` * **Azure Deployment Name**: (Optional) Required only when a single resource contains multiple deployments. To use this authentication your azure application need to have the role of: `conginitive services user`. Enterprise-level authentication with Azure Entra ID: Required parameters: * **Azure Entra ClientID**: Your Azure Entra client ID * **Azure Entra Secret**: Your client secret * **Azure Entra Tenant ID**: Your tenant ID * **Azure Foundry URL**: The base endpoint URL for your deployment, formatted according to your deployment type: * For AI Services: `https://your-resource-name.services.ai.azure.com/models` * For Managed: `https://your-model-name.region.inference.ml.azure.com/score` * For Serverless: `https://your-model-name.region.models.ai.azure.com` * **Azure API Version**: The API version to use (e.g., "2024-05-01-preview"). This is required if you have api version in your deployment url. **Examples:** * If your URL is `https://mycompany-ai.westus2.services.ai.azure.com/models?api-version=2024-05-01-preview`, the API version is `2024-05-01-preview` * **Azure Deployment Name**: (Optional) Required only when a single resource contains multiple deployments. Common in Managed deployments. You can Learn more about these [Azure Entra Resources here](https://learn.microsoft.com/en-us/azure/ai-services/authentication) ## Sample Request Once you've created your virtual key, you can start making requests to Azure AI Foundry models through Obiguard. Install the Obiguard SDK with pip ```sh theme={null} pip install obiguard ``` ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key = "AZURE_FOUNDRY_VIRTUAL_KEY" ) response = client.chat.completions.create( model="DeepSeek-V3-0324", # Replace with your deployed model name messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Tell me about cloud computing"} ] ) print(response.choices[0].message.content) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-virtual-key: $AZURE_FOUNDRY_VIRTUAL_KEY" \ -d '{ "model": "DeepSeek-V3-0324", "messages": [ { "role": "user", "content": "Tell me about cloud computing" } ] }' ``` ## Advanced Features ### Function Calling Azure AI Foundry supports function calling (tool calling) for compatible models. Here's how to implement it with Obiguard: ```python theme={null} tools = [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }] response = client.chat.completions.create( model="DeepSeek-V3-0324", # Use a model that supports function calling messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the weather like in Delhi?"} ], tools=tools, tool_choice="auto" ) print(response.choices[0]) ``` ### Vision Capabilities Process images alongside text using Azure AI Foundry's vision capabilities: ```python theme={null} response = client.chat.completions.create( model="Llama-4-Scout-17B-16E", # Use a model that supports vision messages=[ { "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, { "type": "image_url", "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", }, ], } ], max_tokens=500, ) print(response.choices[0].message.content) ``` ### Structured Outputs Get consistent, parseable responses in specific formats: ```python theme={null} response = client.chat.completions.create( model="cohere-command-a", # Use a model that supports response formats messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "List the top 3 cloud providers with their main services"} ], response_format={"type": "json_object"}, temperature=0 ) import json print(json.loads(response.choices[0].message.content)) ``` ## Relationship with Azure OpenAI For Azure OpenAI specific models and deployments, we recommend using the existing Azure OpenAI provider in Obiguard: Learn how to integrate Azure OpenAI with Obiguard for access to OpenAI models hosted on Azure. # Azure OpenAI Source: https://docs.obiguard.ai/integrations/llms/azure-openai/azure-openai Azure OpenAI is a great alternative to accessing the best models including GPT-4 and more in your private environments. Obiguard provides complete support for Azure OpenAI. With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `azure-openai` ## Obiguard SDK Integration with Azure OpenAI Obiguard provides a consistent API to interact with models from various providers. To integrate Azure OpenAI with Obiguard: ### First, add your Azure details to Obiguard's Virtual Keys **Here's a step-by-step guide:** 1. Request access to Azure OpenAI [here](https://aka.ms/oai/access). 2. Create a resource in the Azure portal [here](https://portal.azure.com/?microsoft%5Fazure%5Fmarketplace%5FItemHideKey=microsoft%5Fopenai%5Ftip#create/Microsoft.CognitiveServicesOpenAI). (This will be your **Resource Name**) 3. Deploy a model in Azure OpenAI Studio [here](https://oai.azure.com/). (This will be your **Deployment Name)** 4. Select your `Foundation Model` from the dropdowon on the modal. 5. Now, on Azure OpenAI studio, go to any playground (chat or completions), click on a UI element called "View code". Note down the API version & API key from here. (This will be your **Azure API Version** & **Azure API Key**) When you input these details, the foundation model will be auto populated. More details in [this guide](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal). If you do not want to add your Azure details to Obiguard, you can also directly pass them while instantiating the Obiguard client. [More on that here.](/integrations/llms/azure-openai/azure-openai#making-requests-without-virtual-keys) **Now, let's make a request using this virtual key!** ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Azure OpenAI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key Set up Obiguard with your virtual key as part of the initialization configuration. You can create a [virtual key](/virtual-keys) for Azure in the Obiguard UI. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="AZURE_VIRTUAL_KEY" # Replace with your virtual key for Azure ) ``` ### **3. Invoke Chat Completions with Azure OpenAI** Use the Obiguard instance to send requests to your Azure deployments. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'custom_model_name' ) print(completion.choices) ``` ## Image Generation Obioguard supports multiple modalities for Azure OpenAI and you can make image generation requests through Obioguard's AI Gateway the same way as making completion calls. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="DALL-E_VIRTUAL_KEY" # Referencing a Dall-E Azure deployment with Virtual Key ) image = client.images.generate( prompt="Lucy in the sky with diamonds", size="1024x1024" ) ``` Obioguard's fast AI gateway captures the information about the request on your Obioguard Dashboard. On your logs screen, you'd be able to see this request with the request and response. api Log view for an image generation request on Azure OpenAI More information on image generation is available in the [API Reference](/api-reference/completions-1#create-image). *** ## Making Requests Without Virtual Keys Here's how you can pass your Azure OpenAI details and secrets directly without using the Virutal Keys feature. ### Key Mapping In a typical Azure OpenAI request, ```sh theme={null} curl https://{YOUR_RESOURCE_NAME}.openai.azure.com/openai/deployments/{YOUR_DEPLOYMENT_NAME}/chat/completions?api-version={API_VERSION} \ -H "Content-Type: application/json" \ -H "api-key: {YOUR_API_KEY}" \ -d '{ "model": "gpt-4o", "messages": [ { "role": "system", "content": "You are a helpful assistant" }, { "role": "user", "content": "what is a deer?" } ] }' ``` | Parameter | Node SDK | Python SDK | REST Headers | | --------------------- | -------------------------- | --------------------------- | ------------------------------ | | AZURE RESOURCE NAME | azureResourceName | azure\_resource\_name | x-obiguard-azure-resource-name | | AZURE DEPLOYMENT NAME | azureDeploymentId | azure\_deployment\_id | x-obiguard-azure-deployment-id | | API VERSION | azureApiVersion | azure\_api\_version | x-obiguard-azure-api-version | | AZURE API KEY | Authorization: "Bearer + " | Authorization = "Bearer + " | Authorization | | AZURE MODEL NAME | azureModelName | azure\_model\_name | x-obiguard-azure-model-name | ### Example ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "azure-openai", azure_resource_name = "AZURE_RESOURCE_NAME", azure_deployment_id = "AZURE_DEPLOYMENT_NAME", azure_api_version = "AZURE_API_VERSION", azure_model_name = "AZURE_MODEL_NAME", Authorization = "Bearer API_KEY" ) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $AZURE_OPENAI_API_KEY" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: azure-openai" \ -H "x-obiguard-azure-resource-name: $AZURE_RESOURCE_NAME" \ -H "x-obiguard-azure-deployment-id: $AZURE_DEPLOYMENY_ID" \ -H "x-obiguard-azure-model-name: $AZURE_MODEL_NAME" \ -H "x-obiguard-azure-api-version: $AZURE_API_VERSION" \ -d '{ "model": "gpt-4o", "messages": [{"role": "user","content": "Hello!"}] }' ``` ### How to Pass JWT (JSON Web Tokens) If you have configured fine-grained access for Azure OpenAI and need to use `JSON web token (JWT)` in the `Authorization` header instead of the regular `API Key`, you can use the `forwardHeaders` parameter to do this. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider = "azure-openai", azure_resource_name = "AZURE_RESOURCE_NAME", azure_deployment_id = "AZURE_DEPLOYMENT_NAME", azure_api_version = "AZURE_API_VERSION", azure_model_name = "AZURE_MODEL_NAME", Authorization = "Bearer API_KEY", # Pass your JWT here forward_headers= [ "Authorization" ] ) ``` For further questions on custom Azure deployments or fine-grained access tokens, reach out to us on [support@obiguard.com](mailto:support@obiguard.com). ## Next Steps The complete list of features supported in the SDK are available on the link below. # AWS Bedrock Source: https://docs.obiguard.ai/integrations/llms/bedrock/aws-bedrock Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including models hosted on AWS Bedrock. With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `bedrock` ## Obiguard SDK Integration with AWS Bedrock Obiguard provides a consistent API to interact with models from various providers. To integrate Bedrock with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Anthropic's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key There are two ways to integrate AWS Bedrock with Obiguard:
Use your `AWS Secret Access Key`, `AWS Access Key Id`, and `AWS Region` to create your Virtual key.

[**Integration Guide**](/integrations/llms/aws-bedrock#how-to-find-your-aws-credentials)
Take your `AWS Assumed Role ARN` and `AWS Region` to create the virtaul key.


[**Integration Guide**](/virtual-keys/bedrock-amazon-assumed-role) ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Bedrock ) ``` #### Using Virtual Key with AWS STS If you're using [AWS Security Token Service](https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html), you can pass your `aws_session_token` along with the Virtual key: ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Bedrock, aws_session_token="" ) ``` #### Not using Virtual Keys? [Check out this example on how you can directly use your AWS details to make a Bedrock request through Obiguard.](/integrations/llms/bedrock/aws-bedrock#making-requests-without-virtual-keys) ### **3. Invoke Chat Completions with AWS bedrock** Use the Obiguard instance to send requests to Bedrock. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'anthropic.claude-v2:1', max_tokens=250 # Required field for Anthropic ) print(completion.choices) ``` ## Using Vision Models Obiguard's multimodal Gateway fully supports Bedrock's vision models `anthropic.claude-3-sonnet`, `anthropic.claude-3-haiku`, and `anthropic.claude-3-opus` For more info, check out this guide: [Vision](/gateway/multimodal-capabilities/vision) ## Extended Thinking (Reasoning Models) (Beta) The assistants thinking response is returned in the `response_chunk.choices[0].delta.content_blocks` array, not the `response.choices[0].message.content` string. Models like `us.anthropic.claude-3-7-sonnet-20250219-v1:0` support [extended thinking](https://aws.amazon.com/blogs/aws/anthropics-claude-3-7-sonnet-the-first-hybrid-reasoning-model-is-now-available-in-amazon-bedrock/). This is similar to openai thinking, but you get the model's reasoning as it processes the request as well. Note that you will have to set [`strict_open_ai_compliance=False`](/gateway/strict-open-ai-compliance) in the headers to use this feature. ### Single turn conversation ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY", # Add your provider's virtual key strict_openai_compliance=False ) response = client.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) # in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array # response = client.chat.completions.create( # ...same config as above but with stream: true # ) # for chunk in response: # if chunk.choices[0].delta: # content_blocks = chunk.choices[0].delta.get("content_blocks") # if content_blocks is not None: # for content_block in content_blocks: # print(content_block) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='BEDROCK_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="bedrock", obiguard_api_key="sk-obg******", # Your Obiguard API key strict_openai_compliance=False ) ) response = openai.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: bedrock" \ -H "x-api-key: $BEDROCK_API_KEY" \ -H "x-obiguard-strict-openai-compliance: false" \ -d '{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": true, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] }' ``` ### Multi turn conversation ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY", # Add your provider's virtual key strict_openai_compliance=False ) # Create the request response = client.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='BEDROCK_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="bedrock", obiguard_api_key="sk-obg******", # Your Obiguard API key strict_openai_compliance=False ) ) response = openai.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: bedrock" \ -H "x-api-key: $BEDROCK_API_KEY" \ -H "x-obiguard-strict-openai-compliance: false" \ -d '{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": true, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] }' ``` ## Inference Profiles [Inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles.html) are a resource in Amazon Bedrock that define a model and one or more Regions to which the inference profile can route model invocation requests. To use inference profiles, your IAM role needs to additionally have the following permissions: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:GetInferenceProfile" ], "Resource": [ "arn:aws:bedrock:*:*:inference-profile/*", "arn:aws:bedrock:*:*:application-inference-profile/*" ] } ] } ``` This is a pre-requisite for using inference profiles, as the gateway needs to fetch the foundation model to process the request. For reference, see the following documentation: [https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html) ## Bedrock Converse API Obiguard uses the [AWS Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) internally for making chat completions requests. If you need to pass [additional input fields or parameters](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html#API_runtime_Converse_RequestSyntax) like `anthropic_beta`, `top_k`, `frequency_penalty` etc. that are specific to a model, you can pass it with this key: ```json theme={null} "additionalModelRequestFields": { "frequency_penalty": 0.4 } ``` If you require the model to [respond with certain fields](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html#API_runtime_Converse_RequestSyntax) that are specific to a model, you need to pass this key: ```json theme={null} "additionalModelResponseFieldPaths": [ "/stop_sequence" ] ``` ## Making Requests without Virtual Keys If you do not want to add your AWS details to Obiguard vault, you can also directly pass them while instantiating the Obiguard client. ### Mapping the Bedrock Details | Node SDK | Python SDK | REST Headers | | ------------------ | ------------------------ | -------------------------------- | | awsAccessKeyId | aws\_access\_key\_id | x-obiguard-aws-access-key-id | | awsSecretAccessKey | aws\_secret\_access\_key | x-obiguard-aws-secret-access-key | | awsRegion | aws\_region | x-obiguard-aws-region | | awsSessionToken | aws\_session\_token | x-obiguard-aws-session-token | ### Example ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="bedrock", aws_access_key_id="", aws_secret_access_key="", aws_region="us-east-1", aws_session_token="" ) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: bedrock" \ -H "x-obiguard-aws-access-key-id: $AWS_ACCESS_KEY_ID" \ -H "x-obiguard-aws-secret-access-key: $AWS_SECRET_ACCESS_KEY" \ -H "x-obiguard-aws-region: $AWS_REGION" \ -H "x-obiguard-aws-session-token: $AWS_TOKEN" \ -d '{ "model": "gpt-4o", "messages": [{"role": "user","content": "Hello!"}] }' ``` *** ## Supported Models *** ## How to Find Your AWS Credentials [Navigate here in the AWS Management Console](https://us-east-1.console.aws.amazon.com/iam/home#/security%5Fcredentials) to obtain your **AWS Access Key ID** and **AWS** **Secret Access Key.** * In the console, you'll find the '**Access keys'** section. Click on '**Create access key**'. * Copy the `Secret Access Key` once it is generated, and you can view the `Access Key ID` along with it. acess * On the same [page](https://us-east-1.console.aws.amazon.com/iam/home#/security%5Fcredentials) under the '**Access keys'** section, where you created your Secret Access key, you will also find your **Access Key ID.** retrieve * And lastly, get Your `AWS Region` from the Home Page of[ AWS Bedrock](https://us-east-1.console.aws.amazon.com/bedrock/home?region=us-east-1#/overview) as shown in the image below. bedrock *** ## Next Steps The complete list of features supported in the SDK are available on the link below. # Embeddings Source: https://docs.obiguard.ai/integrations/llms/bedrock/embeddings Get embeddings from Bedrock Bedrock supports embedding text and images through Amazon Titan and Cohere models. Obiguard provides a standardized interface for embedding multiple modalities. # Bedrock Titan ## Embedding Text ```python Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY", ) embeddings = client.embeddings.create( model="amazon.titan-embed-text-v2:0", input="Hello this is a test", # normalize=False # if you would like to disable normalization # dimensions=1024, # embedding dimensions # encoding_format="float", # embedding format ) ``` ```python OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='NOT_REQUIRED', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) embeddings = client.embeddings.create( model="amazon.titan-embed-text-v2:0", input="Hello this is a test", # normalize=False # if you would like to disable normalization # dimensions=1024, # embedding dimensions # encoding_format="float", # embedding format ) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/embeddings' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --data-raw '{ "model": "amazon.titan-embed-text-v2:0", "input": "Hello this is a test", "normalize": false, // if you would like to disable normalization "dimensions": 1024, // embedding dimensions "encoding_format": "float" // embedding format }' ``` ## Embeddings Images ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY", ) embeddings = client.embeddings.create( model="amazon.titan-embed-image-v1", dimensions=256, input=[ { "text": "this is the caption of the image", "image": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....." } } ] ) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='NOT_REQUIRED', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) embeddings = client.embeddings.create( model="amazon.titan-embed-image-v1", dimensions=256, input=[ { "text": "this is the caption of the image", "image": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....." } } ] ) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/embeddings' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --data-raw '{ "model": "amazon.titan-embed-image-v1", "dimensions": 256, "input": [ { "text": "this is the caption of the image", "image": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....." } } ] }' ``` # Cohere ## Embedding Text ```python Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY", ) embeddings = client.embeddings.create( model="cohere.embed-english-v3", input=["Hello this is a test", "skibidi"], input_type="classification" ) ``` ```python OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='NOT_REQUIRED', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) embeddings = client.embeddings.create( model="cohere.embed-english-v3", input=["Hello this is a test", "skibidi"], input_type="classification" ) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/embeddings' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --data-raw '{ "model": "cohere.embed-english-v3", "input": ["Hello this is a test", "skibidi"], "input_type": "classification" }' ``` ## Embeddings Images ```python Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY", ) embeddings = client.embeddings.create( model="cohere.embed-english-v3", input_type="image", dimensions=256, input=[ { "image": { "base64": "Data:image/webp;base64,UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....." } } ] ) ``` ```python OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='NOT_REQUIRED', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) embeddings = client.embeddings.create( model="cohere.embed-english-v3", input_type="image", dimensions=256, input=[ { image: { base64: "Data:image/webp;base64,UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....." } } ] ) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/embeddings' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --data-raw '{ "model": "cohere.embed-english-v3", "input_type": "image", "dimensions": 256, "input": [ { "image": { "base64": "Data:image/webp;base64,UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....." } } ] }' ``` # Bring Your Own LLM Source: https://docs.obiguard.ai/integrations/llms/byollm Integrate your privately hosted LLMs with Obiguard for unified management, observability, and reliability. Obiguard's Bring Your Own LLM feature allows you to seamlessly integrate privately hosted language models into your AI infrastructure. This powerful capability enables unified management of both private and commercial LLMs through a consistent interface while leveraging Obiguard's comprehensive suite of observability and reliability features. ## Key Benefits * **Unified API Access**: Manage private and commercial LLMs through a single, consistent interface * **Comprehensive Monitoring**: Track performance, usage, and costs alongside your commercial LLM usage * **Simplified Access Control**: Manage team-specific permissions and usage limits * **Secure Credential Management**: Protect sensitive authentication details through Obiguard's secure vault ## Integration Options **Prerequisites** Your private LLM must implement an API specification compatible with one of Obiguard's [supported providers](/integrations/llms) (e.g., OpenAI's `/chat/completions`, Anthropic's `/messages`, etc.). Obiguard offers two primary methods to integrate your private LLMs: 1. **Using Virtual Keys**: Store your deployment details securely in Obiguard's vault 2. **Direct Integration**: Pass deployment details in your requests without storing them ### Option 1: Using Virtual Keys #### Step 1: Add Your Deployment Details Navigate to the [Virtual Keys](/virtual-keys) section in your Obiguard dashboard and create a new Virtual Key. 1. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle 2. Configure your deployment: * Select the matching provider API specification (typically `OpenAI`) * Enter your model's base URL in the `Custom Host` field * Add required authentication headers and their values 3. Click **"Create"** to generate your virtual key #### Step 2: Use Your Virtual Key in Requests After creating your virtual key, you can use it in your applications: ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="YOUR_PRIVATE_LLM_VIRTUAL_KEY" ) response = client.chat.completions.create( model="YOUR_MODEL_NAME", # The model name your private deployment expects messages=[ {"role": "user", "content": "Explain quantum computing in simple terms"} ] ) print(response.choices[0].message.content) ``` ```sh cURL theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-virtual-key: $YOUR_PRIVATE_LLM_VIRTUAL_KEY" \ -d '{ "model": "YOUR_MODEL_NAME", "messages": [ { "role": "user", "content": "Explain quantum computing in simple terms" } ] }' ``` ### Option 2: Direct Integration Without Virtual Keys If you prefer not to store your private LLM details in Obiguard's vault, you can pass them directly in your API requests: ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="openai", # The API spec your LLM implements custom_host="https://your-llm-server.com/v1/", # Include the API version Authorization="Bearer YOUR_AUTH_TOKEN", # Optional: Any auth headers needed forward_headers=["Authorization"] # Headers to forward without processing ) response = client.chat.completions.create( model="YOUR_MODEL_NAME", messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}] ) print(response.choices[0].message.content) ``` ```sh cURL theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -H "x-obiguard-custom-host: https://your-llm-server.com/v1" \ -H "Authorization: Bearer YOUR_AUTH_TOKEN" \ -H "x-obiguard-forward-headers: Authorization" \ -d '{ "model": "YOUR_MODEL_NAME", "messages": [ { "role": "user", "content": "Explain quantum computing in simple terms" } ] }' ``` The `custom_host` must include the API version path (e.g., `/v1/`). Obiguard will automatically append the endpoint path (`/chat/completions`, `/completions`, or `/embeddings`). ## Advanced Features ## Monitoring and Analytics Obiguard provides comprehensive observability for your private LLM deployments, just like it does for commercial providers: * **Log Analysis**: View detailed request and response logs * **Performance Metrics**: Track latency, token usage, and error rates * **User Attribution**: Associate requests with specific users via metadata ## Troubleshooting | Issue | Possible Causes | Solutions | | ----------------------- | --------------------------------------------------- | ------------------------------------------------------------------------------ | | Connection Errors | Incorrect URL, network issues, firewall rules | Verify URL format, check network connectivity, confirm firewall allows traffic | | Authentication Failures | Invalid credentials, incorrect header format | Check credentials, ensure headers are correctly formatted and forwarded | | Timeout Errors | LLM server overloaded, request too complex | Adjust timeout settings, implement load balancing, simplify requests | | Inconsistent Responses | Different model versions, configuration differences | Standardize model versions, document expected behavior differences | ## FAQs Yes, provided it adheres to an API specification supported by Obiguard (e.g., OpenAI, Anthropic, etc.). The model must handle requests and responses in the expected format and be publicly accessible. Create separate virtual keys for each endpoint. Obiguard itself doesn't impose specific request volume limitations for private LLMs. Your throughput will be limited only by your private LLM deployment's capabilities and any rate limits you configure in Obiguard. Yes, you can specify different model names in your requests as long as your private LLM deployment supports them. The model name is passed through to your deployment. Absolutely! One of Obiguard's key benefits is the ability to manage both private and commercial LLMs through a unified interface. ## Next Steps Explore these related resources to get the most out of your private LLM integration: # Cerebras Source: https://docs.obiguard.ai/integrations/llms/cerebras Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including the models hosted on [Cerebras Inference API](https://cerebras.ai/inference). Provider Slug: `cerebras` ## Obiguard SDK Integration with Cerebras Obiguard provides a consistent API to interact with models from various providers. To integrate Cerebras with Obiguard: ### 1. Install the Obiguard SDK ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with Cerebras To use Cerebras with Obiguard, get your API key from [here](https://cerebras.ai/inference), then add it to Obiguard to create the virtual key. ```py theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="CEREBRAS_VIRTUAL_KEY" # Your Cerebras Inference virtual key ) ``` ### 3. Invoke Chat Completions ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'llama3.1-8b' ) ``` *** ## Supported Models Cerebras currently supports `Llama-3.1-8B` and `Llama-3.1-70B`. You can find more info here: [Overview - Starter KitStarter Kit](https://inference-docs.cerebras.ai/introduction) ## Next Steps The complete list of features supported in the SDK are available on the link below. # DeepSeek Source: https://docs.obiguard.ai/integrations/llms/deepseek Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including DeepSeek models. With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug: **deepseek** ## Obiguard SDK Integration with DeepSeek Models Obiguard provides a consistent API to interact with models from various providers. To integrate DeepSeek with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with DeepSeek AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use DeepSeek with Obiguard, [get your API key from here](https://platform.deepseek.com/api_keys), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for DeepSeek ) ``` ### 3. Invoke Chat Completions with DeepSeek Use the Obiguard instance to send requests to DeepSeek. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'deepseek-chat' ) ``` ### 4. Invoke Multi-round Conversation with DeepSeek ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for DeepSeek ) # Round 1 messages = [{"role": "user", "content": "What's the highest mountain in the world?"}] response = client.chat.completions.create( model="deepseek-chat", messages=messages ) messages.append(response.choices[0].message) print(f"Messages Round 1: {messages}") # Round 2 messages.append({"role": "user", "content": "What is the second?"}) response = client.chat.completions.create( model="deepseek-chat", messages=messages ) messages.append(response.choices[0].message) print(f"Messages Round 2: {messages}") ``` ### 5. JSON Output with DeepSeek ```python theme={null} import json from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for DeepSeek ) system_prompt = """ The user will provide some exam text. Please parse the "question" and "answer" and output them in JSON format. EXAMPLE INPUT: Which is the highest mountain in the world? Mount Everest. EXAMPLE JSON OUTPUT: { "question": "Which is the highest mountain in the world?", "answer": "Mount Everest" } """ user_prompt = "Which is the longest river in the world? The Nile River." messages = [{"role": "system", "content": system_prompt}, {"role": "user", "content": user_prompt}] response = client.chat.completions.create( model="deepseek-chat", messages=messages, response_format={ 'type': 'json_object' } ) print(json.loads(response.choices[0].message.content)) ``` ### Supported Endpoints 1. `CHAT_COMPLETIONS` 2. `STREAM_CHAT_COMPLETIONS` The complete list of features supported in the SDK is available on the link below. Learn more about the Obiguard SDK Client # Google Gemini Source: https://docs.obiguard.ai/integrations/llms/gemini Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Google Gemini APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `google` ## Obiguard SDK Integration with Google Gemini Models Obiguard provides a consistent API to interact with models from various providers. To integrate Google Gemini with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Google Gemini's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Gemini with Obiguard, [get your API key from here](https://aistudio.google.com/app/apikey), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ### **3. Invoke Chat Completions with** Google Gemini Use the Obiguard instance to send requests to Google Gemini. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [ { "role": 'system', "content": 'You are not a helpful assistant' }, { "role": 'user', "content": 'Say this is a test' } ], model= 'gemini-1.5-pro' ) print(completion) ``` Obiguard supports the `system_instructions` parameter for Google Gemini 1.5 - allowing you to control the behavior and output of your Gemini-powered applications with ease. Simply include your Gemini system prompt as part of the `{"role":"system"}` message within the `messages` array of your request body. Obiguard Gateway will automatically transform your message to ensure seamless compatibility with the Google Gemini API. ## Function Calling Obiguard supports function calling mode on Google's Gemini Models. ## Document, Video, Audio Processing with Gemini Gemini supports attaching `mp4`, `pdf`, `jpg`, `mp3`, `wav`, etc. file types to your messages. Gemini Docs: * [Document Processing](https://ai.google.dev/gemini-api/docs/document-processing?lang=python) * [Video & Image Processing](https://ai.google.dev/gemini-api/docs/vision?lang=python) * [Audio Processing](https://ai.google.dev/gemini-api/docs/audio?lang=python) Using Obiguard, here's how you can send these media files: ```python Python theme={null} completion = client.chat.completions.create( messages=[ { "role": "system", "content": "You are a helpful assistant" }, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": "gs://cloud-samples-data/generative-ai/image/scones.jpg" } }, { "type": "text", "text": "Describe the image" } ] } ], model='gemini-1.5-pro', max_tokens=200 ) print(completion) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/chat/completions' \ --header 'x-obiguard-provider: vertex-ai' \ --header 'x-obiguard-vertex-region: us-central1' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --header 'Authorization: GEMINI_API_KEY' \ --data '{ "model": "gemini-1.5-pro", "max_tokens": 200, "stream": false, "messages": [ { "role": "system", "content": "You are a helpful assistant" }, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": "gs://cloud-samples-data/generative-ai/image/scones.jpg" } }, { "type": "text", "text": "describe this image" } ] } ] }' ``` This same message format also works for all other media types — just send your media file in the `url` field, like `"url": "gs://cloud-samples-data/video/animals.mp4"`. Your URL should have the file extension, this is used for inferring `MIME_TYPE` which is a required parameter for prompting Gemini models with files. ### Sending base64 Image Here, you can send the `base64` image data along with the `url` field too: ```json theme={null} "url": "data:image/png;base64,UklGRkacAABXRUJQVlA4IDqcAAC....." ``` ## Grounding with Google Search Vertex AI supports grounding with Google Search. This is a feature that allows you to ground your LLM responses with real-time search results. Grounding is invoked by passing the `google_search` tool (for newer models like gemini-2.0-flash-001), and `google_search_retrieval` (for older models like gemini-1.5-flash) in the `tools` array. ```json theme={null} "tools": [ { "type": "function", "function": { "name": "google_search" // or google_search_retrieval for older models } }] ``` If you mix regular tools with grounding tools, vertex might throw an error saying only one tool can be used at a time. ## Extended Thinking (Reasoning Models) (Beta) The assistants thinking response is returned in the `response_chunk.choices[0].delta.content_blocks` array, not the `response.choices[0].message.content` string. Models like `gemini-2.5-flash-preview-04-17` `gemini-2.5-flash-preview-04-17` support [extended thinking](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#claude-3-7-sonnet). This is similar to openai thinking, but you get the model's reasoning as it processes the request as well. Note that you will have to set [`strict_open_ai_compliance=False`](/product/ai-gateway/strict-open-ai-compliance) in the headers to use this feature. ### Single turn conversation ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key strict_open_ai_compliance=False ) # Create the request response = client.chat.completions.create( model="gemini-2.5-flash-preview-04-17", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) # in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array # response = client.chat.completions.create( # ...same config as above but with stream: true # ) # for chunk in response: # if chunk.choices[0].delta: # content_blocks = chunk.choices[0].delta.get("content_blocks") # if content_blocks is not None: # for content_block in content_blocks: # print(content_block) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='VERTEX_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="vertex-ai", obiguard_api_key="OBIGUARD_API_KEY", strict_open_ai_compliance=False ) ) response = openai.chat.completions.create( model="gemini-2.5-flash-preview-04-17", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: vertex-ai" \ -H "x-obiguard-api-key: $VERTEX_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "gemini-2.5-flash-preview-04-17", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": true, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] }' ``` To disable thinking for gemini models like `gemini-2.5-flash-preview-04-17`, you are required to explicitly set `budget_tokens` to `0`. ```json theme={null} "thinking": { "type": "enabled", "budget_tokens": 0 } ``` Gemini grounding mode may not work via Obiguard SDK. Contact [support@obiguard.com](mailto:support@obiguard.com) for assistance. ## Next Steps The complete list of features supported in the SDK are available on the link below. # GitHub Source: https://docs.obiguard.ai/integrations/llms/github Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including the models hosted on [GitHub Models Marketplace](https://github.com/marketplace/models). Provider Slug: `github` ## Obiguard SDK Integration with GitHub Models Obiguard provides a consistent API to interact with models from various providers. To integrate GitHub Models with Obiguard: ### 1. Install the Obiguard SDK ```sh theme={null} pip install -U obiguard ``` ### 2. Initialize Obiguard with GitHub Models To use GitHub with Obiguard, get your API key [from here](https://github.com/settings/tokens), then add it to Obiguard to create the virtual key. ```py theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key virtual_key="GITHUB_VIRTUAL_KEY" # Your GitHub Models virtual key ) ``` ### 3. Invoke Chat Completions ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'Phi-3-small-128k-instruct' ) ``` ## Supported Models Obiguard supports *all* the models (both `Chat/completion` and `Embeddings` capabilities) on the GitHub Models marketplace. ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Groq Source: https://docs.obiguard.ai/integrations/llms/groq Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Groq APIs](https://console.groq.com/docs/quickstart). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `groq` ## Obiguard SDK Integration with Groq Models Obiguard provides a consistent API to interact with models from various providers. To integrate Groq with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Groq AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Groq with Obiguard, [get your API key from here](https://console.groq.com/keys), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Groq ) ``` ### **3. Invoke Chat Completions with** Groq Use the Obiguard instance to send requests to Groq. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'mistral-medium' ) ``` ### Groq Tool Calling Tool calling feature lets models trigger external tools based on conversation context. You define available functions, the model chooses when to use them, and your application executes them and returns results. Obiguard supports Groq Tool Calling and makes it interoperable across multiple providers. ```python Get Weather Tool theme={null} tools = [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }] response = client.chat.completions.create( model="llama-3.3-70b-versatile", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the weather like in Delhi - respond in JSON"} ], tools=tools, tool_choice="auto" ) print(response.choices[0].finish_reason) ``` ```curl Get Weather Tool theme={null} curl -X POST "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: Bearer $OBIGUARD_API_KEY" \ -d '{ "model": "llama-3.3-70b-versatile", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What'\''s the weather like in Delhi - respond in JSON"} ], "tools": [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }], "tool_choice": "auto" }' ``` ### Groq Speech to Text (Whisper) OpenAI's Audio API converts speech to text using the Whisper model. It offers transcription in the original language and translation to English, supporting multiple file formats and languages with high accuracy. ```python Python theme={null} audio_file= open("/path/to/file.mp3", "rb") # Transcription transcription = client.audio.transcriptions.create( model="whisper-large-v3", file=audio_file ) print(transcription.text) # Translation translation = client.audio.translations.create( model="whisper-large-v3", file=audio_file ) print(translation.text) ``` ```curl REST theme={null} # Transcription curl -X POST "https://gateway.obiguard.ai/v1/audio/transcriptions" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "Content-Type: multipart/form-data" \ -F "file=@/path/to/file.mp3" \ -F "model=whisper-large-v3" # Translation curl -X POST "https://gateway.obiguard.ai/v1/audio/translations" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "Content-Type: multipart/form-data" \ -F "file=@/path/to/file.mp3" \ -F "model=whisper-large-v3" ``` ### Groq Text to Speech Groq's Text to Speech (TTS) API converts written text into natural-sounding audio using six distinct voices. It supports multiple languages, streaming capabilities, and various audio formats for different use cases. ```python Python theme={null} from pathlib import Path speech_file_path = Path(__file__).parent / "speech.mp3" response = client.audio.speech.create( model="playai-tts", voice="Fritz-PlayAI", input="Today is a wonderful day to build something people love!" ) with open(speech_file_path, "wb") as f: f.write(response.content) ``` ```curl REST theme={null} curl -X POST "https://gateway.obiguard.ai/v1/audio/speech" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "playai-tts", "voice": "Fritz-PlayAI", "input": "Today is a wonderful day to build something people love!" }' \ --output speech.mp3 ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Hugging Face Source: https://docs.obiguard.ai/integrations/llms/huggingface Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including all the text generation models supported by [Hugging Face's Inference endpoints](https://huggingface.co/docs/api-inference/index). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `Hugging Face` ## Obiguard SDK Integration with Hugging Face Obiguard provides a consistent API to interact with models from various providers. To integrate Hugging Face with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Hugging Face's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Hugging Face with Obiguard, [get your Hugging Face Access token from here](https://huggingface.co/settings/tokens), then add it to Obiguard to create the virtual key. ```py theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key # Replace with your Obiguard API key virtual_key="VIRTUAL_KEY", # Replace with your virtual key for Hugging Face huggingface_base_url="Hugging Face_DEDICATED_URL" # Optional: Use this if you have a dedicated server hosted on Hugging Face ) ``` ```py theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="Hugging Face_ACCESS_TOKEN", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="Hugging Face", huggingface_base_url="Hugging Face_DEDICATED_URL" ) ) ``` ### 3. Invoke Chat Completions with Hugging Face Use the Obiguard instance to send requests to Hugging Face. You can also override the virtual key directly in the API call if needed. ```py theme={null} chat_completion = client.chat.completions.create( messages = [{"role": 'user', "content": 'Say this is a test'}], model = 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot ) print(chat_completion.choices[0].message.content) ``` ```py theme={null} chat_completion = client.chat.completions.create( messages = [{"role": 'user', "content": 'Say this is a test'}], model = 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot ) print(chat_completion.choices[0].message.content) ``` ## [Using Virtual Keys](/virtual-keys) Virtual Keys serve as Obiguard's unified authentication system for all LLM interactions, simplifying the use of multiple providers and Obiguard features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model. 1. Navigate to [Virtual Keys](/virtual-keys) in your Obiguard dashboard 2. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle 3. Configure your deployment: * Select the matching provider API specification (typically `OpenAI`) * Enter your model's base URL in the `Custom Host` field * Add required authentication headers and their values You can now use this virtual key in your requests: ```python theme={null} client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key virtual_key="YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY" ) response = client.chat.completions.create( model="your-self-hosted-model-name", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) ``` For more information about managing self-hosted LLMs with Obiguard, see [Bring Your Own LLM](/integrations/llms/byollm). ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Lepton AI Source: https://docs.obiguard.ai/integrations/llms/lepton Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Lepton AI APIs](https://www.lepton.ai/docs/guides). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `lepton` ## Obiguard SDK Integration with Lepton AI Models Obiguard provides a consistent API to interact with models from various providers. To integrate Lepton AI with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Lepton AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Lepton AI with Obiguard, [get your API key from Lepton AI](https://console.lepton.ai/), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Lepton ) ``` ### 3. Invoke Chat Completions with Lepton AI Use the Obiguard instance to send requests to Lepton AI. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'llama-3-8b-sft-v1' ) print(completion) ``` ```sh theme={null} curl --location 'https://gateway.obiguard.ai/v1/chat/completions' \ -H 'Content-Type: application/json' \ -H 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ -H 'x-obiguard-virtual-key: VIRTUAL_KEY' \ --data '{ "model": "llama-3-8b-sft-v1", "messages": [ { "role": "user", "content": "Say this is a test" } ] }' ``` ## Speech-to-Text (Transcription) Lepton AI provides speech-to-text capabilities through Obiguard's unified API: ```python theme={null} with open("audio.mp3", "rb") as audio_file: transcription = client.audio.transcriptions.create( file=audio_file, model="whisper-large-v3" ) print(transcription.text) ``` ## Advanced Features ### Streaming Responses Lepton AI supports streaming responses to provide real-time generation: ```python theme={null} stream = client.chat.completions.create( messages=[{"role": "user", "content": "Write a story about a robot"}], model="llama-3-8b-sft-v1", stream=True ) for chunk in stream: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="") ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # LocalAI Source: https://docs.obiguard.ai/integrations/llms/local-ai Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including your **locally hosted models through** [**LocalAI**](https://localai.io/). ## Obiguard SDK Integration with LocalAI ### 1. Install the Obiguard SDK ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with LocalAI URL First, ensure that your API is externally accessible. If you're running the API on `http://localhost`, consider using a tool like `ngrok` to create a public URL. Then, instantiate the Obiguard client by adding your LocalAI URL (along with the version identifier) to the `customHost` property, and add the provider name as `openai`. **Note:** Don't forget to include the version identifier (e.g., `/v1`) in the `customHost` URL ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="openai", custom_host="https://7cc4-3-235-157-146.ngrok-free.app/v1" # Your LocalAI ngrok URL ) ``` Obiguard currently supports all endpoints that adhere to the OpenAI specification. This means, you can access and observe any of your LocalAI models that are exposed through OpenAI-compliant routes. [List of supported endpoints here](/integrations/llms/local-ai#localai-endpoints-supported). ### 3. Invoke Chat Completions Use the Obiguard SDK to invoke chat completions from your LocalAI model, just as you would with any other provider. ```python theme={null} completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'ggml-koala-7b-model-q4_0-r2.bin' ) print(completion) ``` ## [Using Virtual Keys](/virtual-keys) Virtual Keys serve as Obiguard's unified authentication system for all LLM interactions, simplifying the use of multiple providers and Obiguard features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model. 1. Navigate to [Virtual Keys](/virtual-keys) in your Obiguard dashboard 2. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle 3. Configure your deployment: * Select the matching provider API specification (typically `OpenAI`) * Enter your model's base URL in the `Custom Host` field * Add required authentication headers and their values 4. Click **"Create"** to generate your virtual key You can now use this virtual key in your requests: ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY" ) response = client.chat.completions.create( model="your-self-hosted-model-name", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) ``` ## LocalAI Endpoints Supported | Endpoint | Resource | | :---------------------------------------------- | :---------------------------------------------------------------- | | /chat/completions (Chat, Vision, Tools support) | [Doc](/provider-endpoints/chat) | | /images/generations | [Doc](/provider-endpoints/images/create-image) | | /embeddings | [Doc](/provider-endpoints/embeddings) | | /audio/transcriptions | [Doc](/product/ai-gateway/multimodal-capabilities/speech-to-text) | ## Next Steps Explore the complete list of features supported in the SDK: # Mistral AI Source: https://docs.obiguard.ai/integrations/llms/mistral-ai Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Mistral AI APIs](https://docs.mistral.ai/api/). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `mistral-ai` ## Obiguard SDK Integration with Mistral AI Models Obiguard provides a consistent API to interact with models from various providers. To integrate Mistral AI with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Mistral AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Mistral AI with Obiguard, [get your API key from here](https://console.mistral.ai/api-keys/), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Mistral AI ) ``` ### 3. Invoke Chat Completions with Mistral AI Use the Obiguard instance to send requests to Mistral AI. You can also override the virtual key directly in the API call if needed. You can also call the new Codestral model here! ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'codestral-latest' ) print(completion) ``` ## Invoke Codestral Endpoint Using Obiguard, you can also call Mistral API's new Codestral endpoint. Just pass the Codestral URL `https://codestral.mistral.ai/v1` with the `customHost` property. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="MISTRAL_VIRTUAL_KEY", custom_host="https://codestral.mistral.ai/v1" ) code_completion = client.chat.completions.create( model="codestral-latest", messages=[{"role": "user", "content": "Write a minimalist Python code to validate the proof for the special number 1729"}] ) print(code_completion.choices[0].message.content) ``` ### Mistral Tool Calling Tool calling feature lets models trigger external tools based on conversation context. You define available functions, the model chooses when to use them, and your application executes them and returns results. Obiguard supports Mistral Tool Calling and makes it interoperable across multiple providers. ```python Get Weather Tool theme={null} tools = [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }] response = client.chat.completions.create( model="your_mistral_model_name", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the weather like in Delhi - respond in JSON"} ], tools=tools, tool_choice="auto" ) print(response.choices[0].finish_reason) ``` ```curl Get Weather Tool theme={null} curl -X POST "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -d '{ "model": "your_mistral_model_name", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What'\''s the weather like in Delhi - respond in JSON"} ], "tools": [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }], "tool_choice": "auto" }' ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Nebius Source: https://docs.obiguard.ai/integrations/llms/nebius Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Nebius AI](https://nebius.ai/). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `nebius` ## Obiguard SDK Integration with Nebius AI Models Obiguard provides a consistent API to interact with models from various providers. To integrate Nebius AI with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Nebius AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Nebius AI with Obiguard, [get your API key from here](https://studio.nebius.com/settings/api-keys), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ### 3. Invoke Chat Completions with Nebius AI Use the Obiguard instance to send requests to Nebius AI. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'deepseek-ai/DeepSeek-V3' ) print(completion) ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Nomic Source: https://docs.obiguard.ai/integrations/llms/nomic Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Nomic](https://docs.nomic.ai/reference/getting-started/). Nomic has especially become popular due to it's superior embeddings and is now available through Obiguard's AI gateway as well. With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `nomic` ## Obiguard SDK Integration with Nomic Obiguard provides a consistent API to interact with embedding models from various providers. To integrate Nomic with Obiguard: ### 1. Create a Virtual Key for Nomic in your Obiguard account You can head over to the virtual keys tab and create one for Nomic. This will be then used to make API requests to Nomic without needing the protected API key. ### 2. Install the Obiguard SDK and Initialize with this Virtual Key Add the Obiguard SDK to your application to interact with Nomic's API through Obiguard's gateway. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Nomic ) ``` ### 3. Invoke the Embeddings API with Nomic Use the Obiguard instance to send requests to your Nomic API. You can also override the virtual key directly in the API call if needed. ```python theme={null} embeddings = client.embeddings.create( input='create vector representation on this sentence', model='nomic-embed-text-v1.5' ) print(embeddings) ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Ollama Source: https://docs.obiguard.ai/integrations/llms/ollama Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including your **locally hosted models through Ollama**. Provider Slug. `ollama` ## Obiguard SDK Integration with Ollama Models Obiguard provides a consistent API to interact with models from various providers. ### 1. Expose your Ollama API Expose your Ollama API by using a tunneling service like [ngrok](https://ngrok.com/) or any other way you prefer. For using Ollama with ngrok, here's a [useful guide](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-use-ollama-with-ngrok) ```sh theme={null} ngrok http 11434 --host-header="localhost:11434" ``` ### 2. Install the Obiguard SDK Install the Obiguard SDK in your application to interact with your Ollama API through Obiguard. ```sh theme={null} pip install obiguard ``` ### 3. Initialize Obiguard with Ollama URL Instantiate the Obiguard client by adding your Ollama publicly-exposed URL to the `customHost` property. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="ollama", custom_host="https://8cc4-2-234-255-255.ngrok-free.app" # Your Ollama ngrok URL ) ``` For the Ollama integration, you only need to pass the base URL to `customHost` without the version identifier (such as `/v1`) - Obiguard takes care of the rest! ### **4. Invoke Chat Completions with** Ollama Use the Obiguard SDK to invoke chat completions from your Ollama model, just as you would with any other provider. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'llama3' ) print(completion) ``` ```sh theme={null} curl --location 'https://gateway.obiguard.ai/v1/chat/completions' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-custom-host: https://1eb6-103-180-45-255.ngrok-free.app' \ --header 'x-obiguard-provider: ollama' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --data '{ "model": "tinyllama", "max_tokens": 200, "stream": false, "messages": [ { "role": "system", "content": [ { "type": "text", "text": "You are Batman" } ] }, { "role": "user", "content": [ { "type": "text", "text": "Who is the greatest detective" } ] }, { "role": "assistant", "content": [ { "type": "text", "text": "is it me?" } ] } ] }' ``` ## [Using Virtual Keys](/virtual-keys) Virtual Keys serve as Obiguard's unified authentication system for all LLM interactions, simplifying the use of multiple providers and Obiguard features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your models. 1. Navigate to [Virtual Keys](/virtual-keys) in your Obiguard dashboard 2. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle 3. Configure your deployment: * Select the matching provider API specification (typically `OpenAI`) * Enter your model's base public URL in the `Custom Host` field * Add required authentication headers and their values 4. Click **"Create"** to generate your virtual key You can now use this virtual key in your requests: ```python theme={null} client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY" ) response = client.chat.completions.create( model="your-self-hosted-model-name", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) print(response) ``` ```py theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="vk-obg******", # Your Obiguard virtual key ) ) completion = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) print(completion.choices[0].message) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -d '{ "model": "gpt-4o", "messages": [ {"role": "user", "content": "Hello!"} ] }' ``` ## Ollama Tool Calling Tool calling feature lets models trigger external tools based on conversation context. You define available functions, the model chooses when to use them, and your application executes them and returns results. Obiguard supports Ollama Tool Calling and makes it interoperable across multiple providers. ```python theme={null} tools = [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }] response = client.chat.completions.create( model="llama-3.3-70b-versatile", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the weather like in Delhi - respond in JSON"} ], tools=tools, tool_choice="auto" ) print(response.choices[0].finish_reason) ``` ```sh theme={null} curl -X POST "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $YOUR_OBIGUARD_VIRTUAL_KEY" \ -d '{ "model": "llama-3.3-70b-versatile", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What'\''s the weather like in Delhi - respond in JSON"} ], "tools": [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }], "tool_choice": "auto" }' ``` ## Next Steps Explore the complete list of features supported in the SDK: # OpenAI Source: https://docs.obiguard.ai/integrations/llms/openai Discover how to integrate OpenAI with Obiguard for seamless completions, prompt management, and advanced features like streaming, function calling, and fine-tuning. Obiguard has native integrations with OpenAI SDKs for Python, and its REST APIs. Provider slug: `openai` ## Using the Obiguard Gateway To integrate the Obiguard gateway with OpenAI, * Set the `baseURL` to the Obiguard Gateway URL * Include Obiguard-specific headers such as `provider`, `obiguardApiKey`, and others. Here's how to apply it to a **chat completion** request: Install the Obiguard SDK with pip ```sh theme={null} pip install obiguard ``` ```py Chat Completions theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key provider='openai', strict_open_ai_compliance=False ) response = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) print(response.choices[0].message) ``` ```sh Chat Completions theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -d '{ "model": "gpt-4o", "messages": [ {"role": "user", "content": "Hello!"} ] }' ``` Install the OpenAI and Obiguard SDKs with `pip` ```sh theme={null} pip install openai obiguard ``` ```py Chat Completions theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, Obiguard obiguard_client = Obiguard( obiguard_api_key='sk-obg***', # Your Obiguard policy group API key provider='openai', ) openai_client = OpenAI( api_key="sk-***", # Your OpenAI API key base_url=OBIGUARD_GATEWAY_URL, default_headers=obiguard_client.copy_headers() ) completion = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) print(completion.choices[0].message) ``` This request will be logged automatically by Obiguard and can be viewed in your logs dashboard. Obiguard tracks the tokens used, execution time, and cost for each request. You can also examine detailed request and response data. Obiguard supports [OpenAI's new "developer" role](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) in chat completions. Starting with o1 models, the `developer` role replaces the previous `system` role. ### Using the Responses API OpenAI has introduced a new Responses API that merges the capabilities of Chat Completions and Assistants APIs. Obiguard provides full support for this API, allowing its use with both the Obiguard SDK and the OpenAI SDK. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) response = client.responses.create( model="gpt-4.1", input="Tell me a three sentence bedtime story about a unicorn." ) print(response) ``` ```python theme={null} import os from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, Obiguard obiguard_client = Obiguard( obiguard_api_key='sk-obg***', # Your Obiguard policy group API key provider='openai', ) openai_client = OpenAI( api_key=os.environ.get("OPENAI_API_KEY"), base_url=OBIGUARD_GATEWAY_URL, default_headers=obiguard_client.copy_headers() ) response = openai_client.responses.create( model="gpt-4o", instructions="You are a coding assistant that talks like a pirate.", input="How do I check if a Python object is an instance of a class?", ) print(response) ``` The Responses API offers a more adaptable framework for creating agentic applications with integrated tools that run automatically. Learn how Obiguard enables Remote MCP support for OpenAI's Responses API. * The same integration approach applies to APIs for [completions](https://platform.openai.com/docs/guides/text-generation/completions-api), [embeddings](https://platform.openai.com/docs/api-reference/embeddings/create), [vision](https://platform.openai.com/docs/guides/vision/quick-start), [moderation](https://platform.openai.com/docs/api-reference/moderations/create), [transcription](https://platform.openai.com/docs/api-reference/audio/createTranscription), [translation](https://platform.openai.com/docs/api-reference/audio/createTranslation), [speech](https://platform.openai.com/docs/api-reference/audio/createSpeech) and [files](https://platform.openai.com/docs/api-reference/files/create). ### Realtime API Obiguard seamlessly integrates with OpenAI's Realtime API, enabling features like logging, cost tracking, and guardrails. ### Streaming Responses Obiguard supports streaming responses through Server-Sent Events (SSE). ```python theme={null} import os from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, Obiguard obiguard_client = Obiguard( obiguard_api_key='sk-obg***', # Your Obiguard policy group API key provider='openai', ) openai_client = OpenAI( api_key=os.environ.get("OPENAI_API_KEY"), base_url=OBIGUARD_GATEWAY_URL, default_headers=obiguard_client.copy_headers() ) stream = openai_client.responses.create( model="gpt-4o", input="Write a one-sentence bedtime story about a unicorn.", stream=True, ) for event in stream: print(event) ``` #### Streaming with the Responses API You can also stream responses from the Responses API: ```python theme={null} response = client.responses.create( model="gpt-4.1", instructions="You are a helpful assistant.", input="Hello!", stream=True ) for event in response: print(event) ``` ```python theme={null} response = client.responses.create( model="gpt-4.1", instructions="You are a helpful assistant.", input="Hello!", stream=True ) for event in response: print(event) ``` ### Vision Models Support Obiguard's multimodal Gateway provides full compatibility with OpenAI vision models. Refer to this guide for additional details: [Vision](/gateway/multimodal-capabilities/vision) #### Using Vision Models with the Responses API The Responses API also enables processing images alongside text: ```python theme={null} from obiguard import Obiguard prompt = "What is in this image?" img_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/d5/2023_06_08_Raccoon1.jpg/1599px-2023_06_08_Raccoon1.jpg" obiguard_client = Obiguard( obiguard_api_key='vk-obg***', # Your Obiguard virtual key proxy to OpenAI provider='openai', ) response = obiguard_client.responses.create( model="gpt-4o-mini", input=[ { "role": "user", "content": [ {"type": "input_text", "text": prompt}, {"type": "input_image", "image_url": f"{img_url}"}, ], } ], ) print(response) ``` ```python theme={null} import os from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, Obiguard prompt = "What is in this image?" img_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/d5/2023_06_08_Raccoon1.jpg/1599px-2023_06_08_Raccoon1.jpg" obiguard_client = Obiguard( obiguard_api_key='sk-obg***', # Your Obiguard policy group API key provider='openai', ) openai_client = OpenAI( api_key=os.environ.get("OPENAI_API_KEY"), base_url=OBIGUARD_GATEWAY_URL, default_headers=obiguard_client.copy_headers() ) response = openai_client.responses.create( model="gpt-4o-mini", input=[ { "role": "user", "content": [ {"type": "input_text", "text": prompt}, {"type": "input_image", "image_url": f"{img_url}"}, ], } ], ) print(response) ``` ### Function Calling Function calls within your OpenAI or Obiguard SDK operations remain standard. These logs will appear in Obiguard, highlighting the utilized functions and their outputs. Additionally, you can define functions within your prompts and invoke the `obiguard.prompts.completions.create` method as above. #### Function Calling with the Responses API The Responses API also supports function calling with the same powerful capabilities: ```python theme={null} tools = [ { "type": "function", "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location", "unit"] } } ] obiguard_client = Obiguard( obiguard_api_key='vk-obg***', # Your Obiguard virtual key proxy to OpenAI provider='openai', ) response = obiguard_client.responses.create( model="gpt-4.1", tools=tools, input="What is the weather like in Boston today?", tool_choice="auto" ) print(response) ``` ```python theme={null} tools = [ { "type": "function", "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location", "unit"] } } ] obiguard_client = Obiguard( obiguard_api_key='sk-obg***', # Your Obiguard policy group API key provider='openai', ) openai_client = OpenAI( api_key=os.environ.get("OPENAI_API_KEY"), base_url=OBIGUARD_GATEWAY_URL, default_headers=obiguard_client.copy_headers() ) response = openai_client.responses.create( model="gpt-4.1", tools=tools, input="What is the weather like in Boston today?", tool_choice="auto" ) print(response) ``` ### Image Generation Obiguard supports multiple modalities for OpenAI and you can make image generation requests through Obiguard's AI Gateway the same way as making completion calls. ```python theme={null} import os from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, Obiguard # Define the OpenAI client as shown above obiguard_client = Obiguard( obiguard_api_key='sk-obg***', # Your Obiguard policy group API key provider='openai', ) openai_client = OpenAI( api_key=os.environ.get("OPENAI_API_KEY"), base_url=OBIGUARD_GATEWAY_URL, default_headers=obiguard_client.copy_headers() ) image = openai_client.images.generate( model="dall-e-3", prompt="Lucy in the sky with diamonds", size="1024x1024" ) ``` ### Audio - Transcription, Translation, and Text-to-Speech Obiguard's multimodal Gateway also supports the `audio` methods on OpenAI API. Check out the below guides for more info: Check out the below guides for more info: [Text-to-Speech](/gateway/multimodal-capabilities/text-to-speech) [Speech-to-Text](/gateway/multimodal-capabilities/speech-to-text) *** ## Integrated Tools with Responses API ### Web Search Tool Web search delivers accurate and clearly-cited answers from the web, using the same tool as search in ChatGPT: ```python theme={null} response = obiguard_client.responses.create( model="gpt-4.1", tools=[{ "type": "web_search_preview", "search_context_size": "medium", # Options: "high", "medium" (default), or "low" "user_location": {# Optional - for localized results "type": "approximate", "country": "US", "city": "San Francisco", "region": "California" } }], input="What was a positive news story from today?" ) print(response) ``` **Options for `search_context_size`:** * `high`: Most comprehensive context, higher cost, slower response * `medium`: Balanced context, cost, and latency (default) * `low`: Minimal context, lowest cost, fastest response Responses include citations for URLs found in search results, with clickable references. ### File Search Tool File search enables quick retrieval from your knowledge base across multiple file types: ```python theme={null} response = obiguard_client.responses.create( model="gpt-4.1", tools=[{ "type": "file_search", "vector_store_ids": ["vs_1234567890"], "max_num_results": 20, "filters": {# Optional - filter by metadata "type": "eq", "key": "document_type", "value": "report" } }], input="What are the attributes of an ancient brown dragon?" ) print(response) ``` This tool requires you to first create a vector store and upload files to it. Supports various file formats including PDFs, DOCXs, TXT, and more. Results include file citations in the response. ### Enhanced Reasoning Control the depth of model reasoning for more comprehensive analysis: ```python theme={null} response = obiguard.responses.create( model="o3-mini", input="How much wood would a woodchuck chuck?", reasoning={ "effort": "high" # Options: "high", "medium", or "low" } ) print(response) ``` ### Computer Use Assistant Obiguard also supports the Computer Use Assistant (CUA) tool, which helps agents control computers or virtual machines through screenshots and actions. This feature is available for select developers as a research preview on premium tiers. Learn More about Computer use tool here ## Managing OpenAI Projects & Organizations in Obiguard When integrating OpenAI with Obiguard, you can specify your OpenAI organization and project IDs along with your API key. This is particularly useful if you belong to multiple organizations or are accessing projects through a legacy user API key. Specifying the organization and project IDs helps you maintain better control over your access rules, usage, and costs. In Obiguard, you can add your Org & Project details by, 2. Defining a guardrail policy 3. Generating your virtual key for the guardrail policy 4. Passing details in a request Let's explore each method in more detail. ### Using Virtual Keys When selecting OpenAI from the dropdown menu while creating a virtual key, Obiguard automatically displays optional fields for the organization ID and project ID alongside the API key field. [Get your OpenAI API key from here](https://platform.openai.com/api-keys), then add it to Obiguard to create the virtual key that can be used throughout Obiguard. ### While Making a Request You can also pass your organization and project details directly when making a request using curl, the OpenAI SDK, or the Obiguard SDK. ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, Obiguard obiguard_client = Obiguard( obiguard_api_key='sk-obg***', # Your Obiguard policy group API key provider='openai', ) openai_client = OpenAI( api_key=os.environ.get("OPENAI_API_KEY"), base_url=OBIGUARD_GATEWAY_URL, default_headers=obiguard_client.copy_headers() ) chat_complete = openai_client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Say this is a test"}], ) ) print(chat_complete.choices[0].message.content) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "x-obiguard-openai-organization: org-xxxxxxx" \ -H "x-obiguard-openai-project: proj_xxxxxxx" \ -H "x-obiguard-api-key: OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -d '{ "model": "gpt-4o", "messages": [{"role": "user","content": "Hello!"}] }' ``` ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key provider='openai', strict_open_ai_compliance=False ) chat_complete = client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Say this is a test"}], ) print(chat_complete.choices[0].message.content) ``` # Files Source: https://docs.obiguard.ai/integrations/llms/openai/files Upload files to OpenAI ## Uploading Files ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) upload_file_response = client.files.create( purpose="batch", file=open("file.pdf", "rb") ) print(upload_file_response) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="OBIGUARD_API_KEY" ) ) upload_file_response = client.files.create( purpose="batch", file=open("file.pdf", "rb") ) print(upload_file_response) ``` ## List Files ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) files = client.files.list() print(files) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="OBIGUARD_API_KEY" ) ) files = openai.files.list() print(files) ``` ## Get File ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) file = client.files.retrieve(file_id="file_id") print(file) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="OBIGUARD_API_KEY" ) ) file = client.files.retrieve(file_id="file_id") print(file) ``` ## Get File Content ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) file_content = client.files.content(file_id="file_id") print(file_content) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="OBIGUARD_API_KEY" ) ) file_content = openai.files.content(file_id="file_id") print(file_content) ``` ## Delete File ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) delete_file_response = client.files.delete(file_id="file_id") print(delete_file_response) ``` ```python theme={null} from openai import OpenAI from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="OBIGUARD_API_KEY" ) ) delete_file_response = client.files.delete(file_id="file_id") print(delete_file_response) ``` # Structured Outputs Source: https://docs.obiguard.ai/integrations/llms/openai/structured-outputs Structured Outputs ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Obiguard supports OpenAI's Structured Outputs feature out of the box with our SDKs & APIs. Structured Outputs is different from OpenAI's `JSON Mode` as well as `Function Calling`. [Check out this table](#difference-between-structured-outputs-json-mode-and-function-calling) for a quick comparison. Obiguard SDKs for [Python](https://github.com/openai/openai-python/blob/main/helpers.md#structured-outputs-parsing-helpers) and [JavaScript](https://github.com/openai/openai-node/blob/master/helpers.md#structured-outputs-parsing-helpers) also make it easy to define object schemas using [Pydantic](https://docs.pydantic.dev/latest/) and [Zod](https://zod.dev/) respectively. Below, you can see how to extract information from unstructured text that conforms to a schema defined in code. ```python theme={null} from obiguard import Obiguard from pydantic import BaseModel class Step(BaseModel): explanation: str output: str class MathReasoning(BaseModel): steps: list[Step] final_answer: str client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", ) completion = client.beta.chat.completions.parse( model="gpt-4o-2024-08-06", messages=[ {"role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step."}, {"role": "user", "content": "how can I solve 8x + 7 = -23"} ], response_format=MathReasoning, ) print(completion.choices[0].message) print(completion.choices[0].message.parsed) ``` The second approach, shown in the subsequent examples, uses a JSON schema directly in the API call. This method is more portable across different languages and doesn't require additional libraries, but lacks the integrated type checking of the Pydantic/Zod approach. Choose the method that best fits your project's needs and language ecosystem. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", ) completion = client.chat.completions.create( model="gpt-4o-2024-08-06", messages=[ {"role": "system", "content": "Extract the event information."}, {"role": "user", "content": "A meteor the size of 1000 football stadiums will hit earth this Sunday"}, ], response_format={ "type": "json_schema", "json_schema": { "name": "math_reasoning", "schema": { "type": "object", "properties": { "steps": { "type": "array", "items": { "type": "object", "properties": { "explanation": { "type": "string" }, "output": { "type": "string" } }, "required": ["explanation", "output"], "additionalProperties": False } }, "final_answer": { "type": "string" } }, "required": ["steps", "final_answer"], "additionalProperties": False }, "strict": True } }, ) print(completion.choices[0].message.content) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o-2024-08-06", "messages": [ { "role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step." }, { "role": "user", "content": "how can I solve 8x + 7 = -23" } ], "response_format": { "type": "json_schema", "json_schema": { "name": "math_reasoning", "schema": { "type": "object", "properties": { "steps": { "type": "array", "items": { "type": "object", "properties": { "explanation": { "type": "string" }, "output": { "type": "string" } }, "required": ["explanation", "output"], "additionalProperties": false } }, "final_answer": { "type": "string" } "final_answer": { "type": "string" } }, "required": ["steps", "final_answer"], "additionalProperties": false }, "strict": true } } }' ``` ## Difference Between Structured Outputs, JSON Mode, and Function Calling * If you are connecting the model to tools, functions, data, etc. in your system, then you should use **function calling.** * And if you want to structure the model's output when it responds to the user, then you should use a structured `response_format`. * In `response_format`, you can set it as `{ "type": "json_object" }` to enable the [JSON Mode](https://platform.openai.com/docs/guides/structured-outputs/json-mode). * And you can set it as `{ "type": "json_schema" }` to use the [Structured Outputs Mode described above](https://platform.openai.com/docs/guides/structured-outputs). For more, refer to OpenAI's [detailed documentation on Structured Outputs here](https://platform.openai.com/docs/guides/structured-outputs/supported-schemas). # OpenRouter Source: https://docs.obiguard.ai/integrations/llms/openrouter Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [OpenRouter](https://openrouter.ai). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `openrouter` ## Obiguard SDK Integration with OpenRouter Models Obiguard provides a consistent API to interact with models from various providers. To integrate OpenRouter with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with OpenRouter AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use OpenRouter with Obiguard, [get your API key from here](https://openrouter.ai/settings/keys), then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ### 3. Invoke Chat Completions with OpenRouter Use the Obiguard instance to send requests to OpenRouter. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'mistral-medium' ) print(completion) ``` ### Open Router Tool Calling Tool calling feature lets models trigger external tools based on conversation context. You define available functions, the model chooses when to use them, and your application executes them and returns results. Obiguard supports Open Router Tool Calling and makes it interoperable across multiple providers. ```python Get Weather Tool theme={null} tools = [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }] response = client.chat.completions.create( model="openai/gpt-4o", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the weather like in Delhi - respond in JSON"} ], tools=tools, tool_choice="auto" ) print(response.choices[0].finish_reason) ``` ```curl Get Weather Tool theme={null} curl -X POST "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -d '{ "model": "openai/gpt-4o", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What'\''s the weather like in Delhi - respond in JSON"} ], "tools": [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }], "tool_choice": "auto" }' ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Perplexity AI Source: https://docs.obiguard.ai/integrations/llms/perplexity-ai Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Perplexity AI APIs](https://docs.perplexity.ai/reference/post%5Fchat%5Fcompletions). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `perplexity-ai` ## Obiguard SDK Integration with Perplexity AI Models Obiguard provides a consistent API to interact with models from various providers. To integrate Perplexity AI with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Perplexity AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Perplexity AI with Obiguard, [get your API key from here,](https://www.perplexity.ai/settings/api) then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ### 3. Invoke Chat Completions with Perplexity AI Use the Obiguard instance to send requests to Perplexity AI. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'pplx-70b-chat' ) print(completion) ``` ## Fetching citations If you need to obtain citations in the response, you can disable [strict open ai compliance](/gateway/strict-open-ai-compliance) ## Perplexity-Specific Features Perplexity AI offers several unique features that can be accessed through additional parameters in your requests: ### Search Domain Filter (Beta) You can limit citations to specific domains using the `search_domain_filter` parameter. This feature is currently in closed beta and limited to 3 domains for whitelisting or blacklisting. ```python theme={null} completion = client.chat.completions.create( messages=[{"role": "user", "content": "Tell me about electric cars"}], model="pplx-70b-chat", search_domain_filter=["tesla.com", "ford.com", "-competitors.com"] # Use '-' prefix for blacklisting ) ``` ### Image Results (Beta) Enable image results in responses from online models using the `return_images` parameter: ```python theme={null} completion = client.chat.completions.create( messages=[{"role": "user", "content": "Show me pictures of electric cars"}], model="pplx-70b-chat", return_images=True # Feature in closed beta ) ``` ### Related Questions (Beta) Get related questions in the response using the `return_related_questions` parameter: ```python theme={null} completion = client.chat.completions.create( messages=[{"role": "user", "content": "Tell me about electric cars"}], model="pplx-70b-chat", return_related_questions=True # Feature in closed beta ) ``` ### Search Recency Filter Filter search results based on time intervals using the `search_recency_filter` parameter: ```python theme={null} completion = client.chat.completions.create( messages=[{"role": "user", "content": "What are the latest developments in electric cars?"}], model="pplx-70b-chat", search_recency_filter="week" # Options: month, week, day, hour ) ``` ### Web Search Options Determines how much search context is retrieved for the model. Options are: * `low`: minimizes context for cost savings but less comprehensive answers. * `medium`: balanced approach suitable for most queries. * `high`: maximizes context for comprehensive answers but at higher cost. ```python theme={null} completion = client.chat.completions.create( messages=[{"role": "user", "content": "What are the latest developments in electric cars?"}], model="sonar", web_search_options={ "search_context_size": "high" } ) ``` ### Search Recency Filter Filters search results based on time (e.g., 'week', 'day'). ```python theme={null} completion = client.chat.completions.create( messages=[{"role": "user", "content": "What are the latest developments in electric cars?"}], model="sonar", search_recency_filter="", ) ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Snowflake Cortex Source: https://docs.obiguard.ai/integrations/llms/snowflake-cortex Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Snowflake Cortex APIs](https://docs.snowflake.com/). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `cortex` ## Obiguard SDK Integration with Snowflake Cortex Models Obiguard provides a consistent API to interact with models from various providers. To integrate Snowflake Cortex with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Snowflake Cortex AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Snowflake Cortex with Obiguard, get your API key/JWT Token from the Snowflake Platform, then add it to Obiguard to create the virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ### 3. Invoke Chat Completions with Snowflake Cortex Use the Obiguard instance to send requests to Snowflake Cortex. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model= 'claude-3-5-sonnet' ) print(completion) ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Stability AI Source: https://docs.obiguard.ai/integrations/llms/stability-ai Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Stability AI APIs](https://platform.stability.ai/docs/api-reference). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `stability-ai` ## Obiguard SDK Integration with Stability AI Obiguard provides a consistent API to interact with image generation models from various providers. To integrate Stability AI with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with the Stability API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Stability AI with Obiguard, [get your API key from here](https://platform.stability.ai/account/keys). Then add it to Obiguard to create the virtual key ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ### 3. Invoke Image Generation with Stability AI Use the Obiguard instance to send requests to Stability AI. You can also override the virtual key directly in the API call if needed. ```py theme={null} image = Obiguard.images.generate( model="stable-diffusion-v1-6", prompt="Lucy in the sky with diamonds", size="1024x1024" ) ``` Notice how we're using the OpenAI's image generation signature to prompt Stability allowing greater flexibility to change models and providers later if necessary. ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Suggest a new integration! Source: https://docs.obiguard.ai/integrations/llms/suggest-a-new-integration Have a suggestion for an integration with Obiguard? Drop a message on [support@obiguard.com](mailto:support@obiguard.com). # Together AI Source: https://docs.obiguard.ai/integrations/llms/together-ai Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Together AI APIs](https://docs.together.ai/reference/inference). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/virtual-keys) system. Provider Slug. `together-ai` ## Obiguard SDK Integration with Together AI Models Obiguard provides a consistent API to interact with models from various providers. To integrate Together AI with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Together AI's API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To use Together AI with Obiguard, [get your API key from here](https://api.together.ai/settings/api-keys). Then add it to Obiguard to create the virtual key ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ### 3. Invoke Chat Completions with Together AI Use the Obiguard instance to send requests to Together AI. You can also override the virtual key directly in the API call if needed. ```python theme={null} completion = client.chat.completions.create( messages= [{"role": 'user', "content": 'Say this is a test'}], model='togethercomputer/llama-2-70b-chat' ) print(completion) ``` ## Next Steps The complete list of features supported in the SDK are available on the link below. Learn more about the Obiguard SDK Client # Triton Source: https://docs.obiguard.ai/integrations/llms/triton Integrate Trtiton-hosted custom models with Obiguard and take them to production Obiguard provides a robust and secure platform to observe, govern, and manage your **locally** or **privately** hosted custom models using Triton. Here's the official [Triton Inference Server documentation](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/getting_started/quickstart.html) for more details. ## Integrating Custom Models with Obiguard SDK Expose your Triton server by using a tunneling service like [ngrok](https://ngrok.com/) or any other way you prefer. You can skip this step if you’re self-hosting the Gateway. ```sh theme={null} ngrok http 11434 --host-header="localhost:8080" ``` ```sh theme={null} pip install obiguard ``` 1. Pass your publicly-exposed Triton server URL to Obiguard with `customHost` 2. Set target `provider` as `triton`. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="triton", custom_host="http://localhost:8000/v2/models/mymodel" # Your Triton Hosted URL Authorization="AUTH_KEY", # If you need to pass auth ) ``` More on `custom_host` [here](/gateway/universal-api#integrating-local-or-private-models). Use the Obiguard SDK to invoke chat completions (generate) from your model, just as you would with any other provider: ```python theme={null} completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }] ) ``` ## Next Steps Explore the complete list of features supported in the SDK: # Google Vertex AI Source: https://docs.obiguard.ai/integrations/llms/vertex-ai Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs), and embedding models into your apps, including [Google Vertex AI](https://cloud.google.com/vertex-ai?hl=en). With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your Vertex auth through a [virtual key](/virtual-keys/) system Provider Slug. `vertex-ai` ## Obiguard SDK Integration with Google Vertex AI Obiguard provides a consistent API to interact with models from various providers. To integrate Google Vertex AI with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Google Vertex AI API through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with the Virtual Key To integrate Vertex AI with Obiguard, you'll need your `Vertex Project Id` Or `Service Account JSON` & `Vertex Region`, with which you can set up the Virtual key. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY" # Replace with your virtual key for Google ) ``` ### **3. Invoke Chat Completions with** Vertex AI and Gemini Use the Obiguard instance to send requests to Gemini models hosted on Vertex AI. You can also override the virtual key directly in the API call if needed. Vertex AI uses OAuth2 to authenticate its requests, so you need to send the **access token** additionally along with the request. ```python theme={null} completion = client.with_options(Authorization="Bearer $YOUR_VERTEX_ACCESS_TOKEN").chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'gemini-1.5-pro-latest' ) print(completion) ``` To use Anthopic models on Vertex AI, prepend `anthropic.` to the model name.
Example: `anthropic.claude-3-5-sonnet@20240620` Similarly, for Meta models, prepend `meta.` to the model name.
Example: `meta.llama-3-8b-8192` ## Document, Video, Audio Processing Vertex AI supports attaching `webm`, `mp4`, `pdf`, `jpg`, `mp3`, `wav`, etc. file types to your Gemini messages. Gemini Docs: * [Document Processing](https://ai.google.dev/gemini-api/docs/document-processing?lang=python) * [Video & Image Processing](https://ai.google.dev/gemini-api/docs/vision?lang=python) * [Audio Processing](https://ai.google.dev/gemini-api/docs/audio?lang=python) Using Obiguard, here's how you can send these media files: ```python Python theme={null} completion = client.chat.completions.create( messages=[ { "role": "system", "content": "You are a helpful assistant" }, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": "gs://cloud-samples-data/generative-ai/image/scones.jpg" } }, { "type": "text", "text": "Describe the image" } ] } ], model='gemini-1.5-pro-001', max_tokens=200 ) print(completion) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/chat/completions' \ --header 'x-obiguard-provider: vertex-ai' \ --header 'x-obiguard-vertex-region: us-central1' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --header 'Authorization: GEMINI_API_KEY' \ --data '{ "model": "gemini-1.5-pro-001", "max_tokens": 200, "stream": false, "messages": [ { "role": "system", "content": "You are a helpful assistant" }, { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": "gs://cloud-samples-data/generative-ai/image/scones.jpg" } }, { "type": "text", "text": "describe this image" } ] } ] }' ``` ## Extended Thinking (Reasoning Models) (Beta) The assistants thinking response is returned in the `response_chunk.choices[0].delta.content_blocks` array, not the `response.choices[0].message.content` string. Gemini models do not support plugging back the reasoning into multi turn conversations, so you don't need to send the thinking message back to the model. Models like `google.gemini-2.5-flash-preview-04-17` `anthropic.claude-3-7-sonnet@20250219` support [extended thinking](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#claude-3-7-sonnet). This is similar to openai thinking, but you get the model's reasoning as it processes the request as well. Note that you will have to set [`strict_open_ai_compliance=False`](/product/ai-gateway/strict-open-ai-compliance) in the headers to use this feature. ### Single turn conversation ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key strict_open_ai_compliance=False ) # Create the request response = client.chat.completions.create( model="anthropic.claude-3-7-sonnet@20250219", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) # in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array # response = client.chat.completions.create( # ...same config as above but with stream: true # ) # for chunk in response: # if chunk.choices[0].delta: # content_blocks = chunk.choices[0].delta.get("content_blocks") # if content_blocks is not None: # for content_block in content_blocks: # print(content_block) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='VERTEX_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="vertex-ai", obioguard_api_key="sk-obg******", # Replace with your Obiguard API key strict_open_ai_compliance=False ) ) response = openai.chat.completions.create( model="anthropic.claude-3-7-sonnet@20250219", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: vertex-ai" \ -H "x-api-key: $VERTEX_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "anthropic.claude-3-7-sonnet@20250219", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": true, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] }' ``` To disable thinking for gemini models like `google.gemini-2.5-flash-preview-04-17`, you are required to explicitly set `budget_tokens` to `0`. ```json theme={null} "thinking": { "type": "enabled", "budget_tokens": 0 } ``` ### Multi turn conversation ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key virtual_key="VIRTUAL_KEY", # Add your provider's virtual key strict_open_ai_compliance=False ) response = client.chat.completions.create( model="anthropic.claude-3-7-sonnet@20250219", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```py OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='Anthropic_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="vertex-ai", obiguard_api_key="OBIGUARD_API_KEY" strict_open_ai_compliance=False ) ) response = client.chat.completions.create( model="anthropic.claude-3-7-sonnet@20250219", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```sh cURL theme={null} curl "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: vertex-ai" \ -H "x-api-key: $VERTEX_API_KEY" \ -H "x-obiguard-strict-open-ai-compliance: false" \ -d '{ "model": "anthropic.claude-3-7-sonnet@20250219", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": true, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] }' ``` ### Sending `base64` Image Here, you can send the `base64` image data along with the `url` field too: ```json theme={null} "url": "data:image/png;base64,UklGRkacAABXRUJQVlA4IDqcAAC....." ``` This same message format also works for all other media types — just send your media file in the `url` field, like `"url": "gs://cloud-samples-data/video/animals.mp4"` for google cloud urls and `"url":"https://download.samplelib.com/mp3/sample-3s.mp3"` for public urls Your URL should have the file extension, this is used for inferring `MIME_TYPE` which is a required parameter for prompting Gemini models with files ## Text Embedding Models You can use any of Vertex AI's `English` and `Multilingual` models through Obiguard, in the familar OpenAI-schema. The Gemini-specific parameter `task_type` is also supported on Obiguard. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY" ) # Generate embeddings def get_embeddings(): embeddings = client.with_options(Authorization="Bearer $YOUR_VERTEX_ACCESS_TOKEN").embeddings.create( input='The vector representation for this text', model='text-embedding-004', task_type="CLASSIFICATION" # Optional ) print(embeddings) get_embeddings() ``` ```sh theme={null} curl 'https://gateway.obiguard.ai/v1/embeddings' \ -H 'Content-Type: application/json' \ -H 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ -H 'x-obiguard-provider: vertex-ai' \ -H 'Authorization: Bearer VERTEX_AI_ACCESS_TOKEN' \ -H 'x-obiguard-virtual-key: $VERTEX_VIRTUAL_KEY' \ --data-raw '{ "model": "textembedding-004", "input": "A HTTP 246 code is used to signify an AI response containing hallucinations or other inaccuracies", "task_type": "CLASSIFICATION" }' ``` ## Image Generation Models Obiguard supports the `Imagen API` on Vertex AI for image generations, letting you easily make requests in the familar OpenAI-compliant schema. ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key ) client.images.generate( prompt = "Cat flying to mars from moon", model = "imagen-3.0-generate-001" ) ``` ```sh cURL theme={null} curl https://gateway.obiguard.ai/v1/images/generations \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_VIRTUAL_KEY" \ -d '{ "prompt": "Cat flying to mars from moon", "model":"imagen-3.0-generate-001" }' ``` [Image Generation API Reference](/api-reference/inference-api/images/create-image) ### List of Supported Imagen Models * `imagen-3.0-generate-001` * `imagen-3.0-fast-generate-001` * `imagegeneration@006` * `imagegeneration@005` * `imagegeneration@002` ## Grounding with Google Search Vertex AI supports grounding with Google Search. This is a feature that allows you to ground your LLM responses with real-time search results. Grounding is invoked by passing the `google_search` tool (for newer models like gemini-2.0-flash-001), and `google_search_retrieval` (for older models like gemini-1.5-flash) in the `tools` array. ```json theme={null} "tools": [ { "type": "function", "function": { "name": "google_search" // or google_search_retrieval for older models } }] ``` If you mix regular tools with grounding tools, vertex might throw an error saying only one tool can be used at a time. ## gemini-2.0-flash-thinking-exp and other thinking/reasoning models `gemini-2.0-flash-thinking-exp` models return a Chain of Thought response along with the actual inference text, this is not openai compatible, however, Obiguard supports this by adding a `\r\n\r\n` and appending the two responses together. You can split the response along this pattern to get the Chain of Thought response and the actual inference text. If you require the Chain of Thought response along with the actual inference text, pass the [strict open ai compliance flag](/product/ai-gateway/strict-open-ai-compliance) as `false` in the request. If you want to get the inference text only, pass the [strict open ai compliance flag](/product/ai-gateway/strict-open-ai-compliance) as `true` in the request. *** ## Making Requests Without Virtual Keys You can also pass your Vertex AI details & secrets directly without using the Virtual Keys in Obiguard. Vertex AI expects a `region`, a `project ID` and the `access token` in the request for a successful completion request. This is how you can specify these fields directly in your requests: ### Example Request ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key vertex_project_id="sample-55646", vertex_region="us-central1", provider="vertex_ai", Authorization="$GCLOUD AUTH PRINT-ACCESS-TOKEN" ) completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'gemini-1.5-pro-latest' ) print(completion) ``` ```sh theme={null} curl 'https://gateway.obiguard.ai/v1/chat/completions' \ -H 'Content-Type: application/json' \ -H 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ -H 'x-obiguard-provider: vertex-ai' \ -H 'Authorization: Bearer VERTEX_AI_ACCESS_TOKEN' \ -H 'x-obiguard-vertex-project-id: sample-94994' \ -H 'x-obiguard-vertex-region: us-central1' \ --data '{ "model": "gemini-1.5-pro", "messages": [ { "role": "system", "content": "You are a helpful assistant" }, { "role": "user", "content": "what is a deer?" } ] }' ``` For further questions on custom Vertex AI deployments or fine-grained access tokens, reach out to us on [support@obiguard.com](mailto:support@obiguard.com). ### How to Find Your Google Vertex Project Details To obtain your **Vertex Project ID and Region,** [navigate to Google Vertex Dashboard](https://console.cloud.google.com/vertex-ai). * You can copy the **Project ID** located at the top left corner of your screen. * Find the **Region dropdown** on the same page to get your Vertex Region. ### Get Your Service Account JSON * [Follow this process](https://cloud.google.com/iam/docs/keys-create-delete) to get your Service Account JSON. When selecting Service Account File as your authentication method, you'll need to: 1. Upload your Google Cloud service account JSON file 2. Specify the Vertex Region This method is particularly important for using self-deployed models, as your service account must have the `aiplatform.endpoints.predict` permission to access custom endpoints. Learn more about permission on your Vertex IAM key [here](https://cloud.google.com/vertex-ai/docs/general/iam-permissions). **For Self-Deployed Models**: Your service account **must** have the `aiplatform.endpoints.predict` permission in Google Cloud IAM. Without this specific permission, requests to custom endpoints will fail. ### Using Project ID and Region Authentication For standard Vertex AI models, you can simply provide: 1. Your Vertex Project ID (found in your Google Cloud console) 2. The Vertex Region where your models are deployed This method is simpler but may not have all the permissions needed for custom endpoints. ## Next Steps The complete list of features supported in the SDK are available on the link below. # Controlled Generations Source: https://docs.obiguard.ai/integrations/llms/vertex-ai/controlled-generations Controlled Generations ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Obiguard supports Vertex AI's Controlled Generations feature out of the box with our SDKs & APIs. Controlled Generations allows you to constrain model responses to predefined sets of values. This is particularly useful for classification tasks, multiple choice responses, and structured data extraction. This feature is available for `Gemini 1.5 Pro` & `Gemini 1.5 Flash` models. ## With Pydantic & Zod Obiguard SDKs for [Python and JavaScript](/api-reference/sdk/python) also make it easy to define object schemas using [Pydantic](https://docs.pydantic.dev/latest/) and [Zod](https://zod.dev/) respectively. Below, you can see how to extract information from unstructured text that conforms to a schema defined in code. ```python theme={null} from obiguard import Obiguard from pydantic import BaseModel class Step(BaseModel): explanation: str output: str class MathReasoning(BaseModel): steps: list[Step] final_answer: str client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY" ) completion = client.beta.chat.completions.parse( model="gemini-1.5-pro-002", messages=[ {"role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step."}, {"role": "user", "content": "how can I solve 8x + 7 = -23"} ], response_format=MathReasoning, ) print(completion.choices[0].message) print(completion.choices[0].message.parsed) ``` ## Using Enums You can also use enums to constrain the model's output to a predefined set of values. This is particularly useful for classification tasks and multiple choice responses. ```python theme={null} from openai import OpenAI from enum import Enum from pydantic import BaseModel class InstrumentClass(Enum): PERCUSSION = "Percussion" STRING = "String" WOODWIND = "Woodwind" BRASS = "Brass" KEYBOARD = "Keyboard" client = Obiguard( obiguard_api_key="sk-obg******", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY" ) # Simple enum classification completion = client.chat.completions.create( model="gemini-1.5-pro-002", messages=[ {"role": "system", "content": "Classify the musical instrument."}, {"role": "user", "content": "What type of instrument is a piano?"} ], response_format={ "type": "json_schema", "json_schema": { "type": "string", "enum": [e.value for e in InstrumentClass], "title": "instrument_classification" } } ) print(completion.choices[0].message.content) ``` ## Using JSON schema Directly This method is more portable across different languages and doesn't require additional libraries, but lacks the integrated type checking of the Pydantic/Zod approach. Choose the method that best fits your project's needs. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY" ) completion = client.chat.completions.create( model="gemini-1.5-pro-002", messages=[ {"role": "system", "content": "Extract the event information."}, {"role": "user", "content": "A meteor the size of 1000 football stadiums will hit earth this Sunday"}, ], response_format={ "type": "json_schema", "json_schema": { "name": "math_reasoning", "schema": { "type": "object", "properties": { "steps": { "type": "array", "items": { "type": "object", "properties": { "explanation": { "type": "string" }, "output": { "type": "string" } }, "required": ["explanation", "output"], "additionalProperties": False } }, "final_answer": { "type": "string" } }, "required": ["steps", "final_answer"], "additionalProperties": False }, "strict": True } }, ) print(completion.choices[0].message.content) ``` ```sh theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-virtual-key: $VERTEX_VIRTUAL_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gemini-1.5-pro-002", "messages": [ { "role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step." }, { "role": "user", "content": "how can I solve 8x + 7 = -23" } ], "response_format": { "type": "json_schema", "json_schema": { "name": "math_reasoning", "schema": { "type": "object", "properties": { "steps": { "type": "array", "items": { "type": "object", "properties": { "explanation": { "type": "string" }, "output": { "type": "string" } }, "required": ["explanation", "output"], "additionalProperties": false } }, "final_answer": { "type": "string" } }, "required": ["steps", "final_answer"], "additionalProperties": false }, "strict": true } } }' ``` For more, refer to Google Vertex AI's [detailed documentation on Controlled Generations here](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-dev). # Embeddings Source: https://docs.obiguard.ai/integrations/llms/vertex-ai/embeddings Get embeddings from Vertex AI Vertex AI offers wide ranging support for embedding text, images and videos. Obiguard provides a standardized interface for embedding multiple modalities. ## Embedding Text ```python Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) embeddings = client.embeddings.create( model="textembedding-gecko@003", input_type="classification", input="The food was delicious and the waiter...", # input=["text to embed", "more text to embed"], # if you would like to embed multiple texts ) ``` ```python OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='NOT_REQUIRED', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) embeddings = client.embeddings.create( model="textembedding-gecko@003", input_type="classification", input="The food was delicious and the waiter...", # input=["text to embed", "more text to embed"], # if you would like to embed multiple texts ) ``` ## Embeddings Images ```python Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) embeddings = client.embeddings.create( model="multimodalembedding@001", input=[ { "text": "this is the caption of the image", "image": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....", # "url": "gcs://..." # if you want to use a url } } ] ) ``` ```python OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='NOT_REQUIRED', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) embeddings = client.embeddings.create( model="multimodalembedding@001", input=[ { "text": "this is the caption of the image", "image": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....", # "url": "gcs://..." # if you want to use a url } } ] ) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/embeddings' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --data-raw '{ "model": "multimodalembedding@001", "input": [ { "text": "this is the caption of the image", "image": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....." # "url": "gcs://..." # if you want to use a url } } ] }' ``` ## Embeddings Videos ```python Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key ) embeddings = client.embeddings.create( model="multimodalembedding@001", input=[ { "text": "this is the caption of the video", "video": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....", "start_offset": 0, "end_offset": 10, "interval": 5, # "url": "gcs://..." # if you want to use a url } } ] ) ``` ```python OpenAI Python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='NOT_REQUIRED', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="openai", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) embeddings = client.embeddings.create( model="multimodalembedding@001", input=[ { "text": "this is the caption of the video", "video": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....", "start_offset": 0, "end_offset": 10, "interval": 5, # "url": "gcs://..." # if you want to use a url } } ] ) ``` ```sh cURL theme={null} curl --location 'https://gateway.obiguard.ai/v1/embeddings' \ --header 'Content-Type: application/json' \ --header 'x-obiguard-api-key: $OBIGUARD_API_KEY' \ --data-raw '{ "model": "multimodalembedding@001", "input": [ { "text": "this is the caption of the video", "video": { "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....", "start_offset": 0, "end_offset": 10, "interval": 5 # "url": "gcs://..." # if you want to use a url } } ] }' ``` # Files Source: https://docs.obiguard.ai/integrations/llms/vertex-ai/files Upload files to Google Cloud Storage for Vertex AI fine-tuning and batch inference To perform fine-tuning or batch inference with Vertex AI, you need to upload files to Google Cloud Storage. With Obiguard, you can easily upload files to GCS and use them for fine-tuning or batch inference with Vertex AI models. ## Uploading Files ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY", # Add your Vertex virtual key vertex_storage_bucket_name="your_bucket_name", # Specify the GCS bucket name provider_file_name="your_file_name.jsonl", # Specify the file name in GCS provider_model="gemini-1.5-flash-001" # Specify the model to use ) upload_file_response = client.files.create( purpose="fine-tune", # Can be "fine-tune" or "batch" file=open("dataset.jsonl", "rb") ) print(upload_file_response) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( virtual_key="VERTEX_VIRTUAL_KEY", obiguard_api_key="sk-obg******", # Your Obiguard API key vertex_storage_bucket_name="your_bucket_name", provider_file_name="your_file_name.jsonl", provider_model="gemini-1.5-flash-001" ) ) upload_file_response = openai.files.create( purpose="fine-tune", # Can be "fine-tune" or "batch" file=open("dataset.jsonl", "rb") ) print(upload_file_response) ``` ## Get File ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY" # Add your Vertex virtual key ) file = client.files.retrieve(file_id="file_id") print(file) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( virtual_key="VERTEX_VIRTUAL_KEY", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) file = openai.files.retrieve(file_id="file_id") print(file) ``` ## Get File Content ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="OBIGUARD_API_KEY", # Your Obiguard API key virtual_key="VERTEX_VIRTUAL_KEY" # Add your Vertex virtual key ) file_content = client.files.content(file_id="file_id") print(file_content) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( virtual_key="VERTEX_VIRTUAL_KEY", obiguard_api_key="sk-obg******", # Your Obiguard API key ) ) file_content = openai.files.content(file_id="file_id") print(file_content) ``` Note: The `ListFiles` endpoint is not supported for Vertex AI. # vLLM Source: https://docs.obiguard.ai/integrations/llms/vllm Integrate vLLM-hosted custom models with Obiguard and take them to production Obiguard provides a robust and secure platform to observe, govern, and manage your **locally** or **privately** hosted custom models using vLLM. Here's a [list](https://docs.vllm.ai/en/latest/models/supported_models.html) of all model architectures supported on vLLM. ## Integrating Custom Models with Obiguard SDK Expose your vLLM server by using a tunneling service like [ngrok](https://ngrok.com/) or any other way you prefer. You can skip this step if you’re self-hosting the Gateway. ```sh theme={null} ngrok http 11434 --host-header="localhost:8080" ``` ```sh theme={null} pip install obiguard ``` 1. Pass your publicly-exposed vLLM server URL to Obiguard with `customHost` (by default, vLLM is on `http://localhost:8000/v1`) 2. Set target `provider` as `openai` since the server follows OpenAI API schema. ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="sk-obg***", # Your Obiguard API key provider="openai", custom_host="https://7cc4-3-235-157-146.ngrok-free.app" # Your vLLM ngrok URL Authorization="AUTH_KEY", # If you need to pass auth ) ``` More on `custom_host` [here](/gateway/universal-api#integrating-local-or-private-models). Use the Obiguard SDK to invoke chat completions from your model, just as you would with any other provider: ```python theme={null} completion = client.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }] ) print(completion) ``` ## [Using Virtual Keys](/virtual-keys) Virtual Keys serve as Obiguard's unified authentication system for all LLM interactions, simplifying the use of multiple providers and Obiguard features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model. 1. Navigate to [Virtual Keys](/virtual-keys) in your Obiguard dashboard 2. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle 3. Configure your deployment: * Select the matching provider API specification (typically `OpenAI`) * Enter your model's base URL in the `Custom Host` field * Add required authentication headers and their values 4. Click **"Create"** to generate your virtual key You can now use this virtual key in your requests: ```python theme={null} client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key virtual_key="YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY" ) response = client.chat.completions.create( model="your-self-hosted-model-name", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) ``` For more information about managing self-hosted LLMs with Obigaurd, see [Bring Your Own LLM](/integrations/llms/byollm). ## Next Steps Explore the complete list of features supported in the SDK: # xAI Source: https://docs.obiguard.ai/integrations/llms/x-ai Obiguard supports xAI's chat completions, completions, and embeddings APIs. Provider Slug. `x-ai` ## Integrate Just paste your xAI API Key to [Obiguard](/virtual-keys) to create your Virtual Key. ## Sample Request Obiguard is a drop-in replacement for xAI. You can make request using the official Obiguard SDK. Popular libraries & agent frameworks like LangChain, CrewAI, AutoGen, etc. are [also supported](#popular-libraries). ```py Python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) response = client.chat.completions.create( model="grok-beta", messages=[{"role": "user", "content": "Hello!"}] ) print(response.choices[0].message) ``` ```py OpenAI Python SDK theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key='OPENAI_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ) completion = client.chat.completions.create( model="grok-beta", messages=[ {"role": "user", "content": "Hello!"} ] ) print(completion.choices[0].message) ``` ```sh cURL theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -d '{ "model": "grok-beta", "messages": [ {"role": "user", "content": "Hello!"} ] }' ``` ## Integration Overview ### xAI Endpoints & Capabilities Obiguard works with *all* of xAI's endpoints and supports all xAI capabilities like function calling and image understanding. Find examples for each below: ```python Python theme={null} tools = [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }] response = client.chat.completions.create( model="grok-beta", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the weather like in Delhi - respond in JSON"} ], tools=tools, tool_choice="auto" ) print(response.choices[0].finish_reason) ``` ```curl REST theme={null} curl -X POST "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -d '{ "model": "grok-beta", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What'\''s the weather like in Delhi - respond in JSON"} ], "tools": [{ "type": "function", "function": { "name": "getWeather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and state"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }], "tool_choice": "auto" }' ``` Process images alongside text using xAI's vision capabilities: ```python Python theme={null} response = client.chat.completions.create( model="grok-beta", messages=[ { "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, { "type": "image_url", "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", }, ], } ], max_tokens=300, ) print(response) ``` ```curl REST theme={null} curl -X POST "https://gateway.obiguard.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_Obiguard_API_KEY" \ -d '{ "model": "grok-beta", "messages": [ { "role": "user", "content": [ {"type": "text", "text": "What'\''s in this image?"}, { "type": "image_url", "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" } ] } ], "max_tokens": 300 }' ``` Generate embeddings for text using xAI's embedding models: `Coming Soon!` ## FAQs You can sign up to xAI [here](https://accounts.x.ai/) and grab your API key. xAI typically gives some amount of free credits without you having to add your credit card. Reach out to their support team if you'd like additional free credits. You can find your current rate limits imposed by xAI on the console. # Milvus Source: https://docs.obiguard.ai/integrations/vector-databases/milvus [Milvus](https://milvus.io/) is an open-source vector database built for GenAI applications. It is built to be performant and scale to tens of billions of vectors with minimal performance loss. Obiguard provides a proxy to Milvus, allowing you to use virtual keys and observability features. ## Obiguard SDK Integration with Milvus Obiguard provides a consistent API to interact with models from various providers. To integrate Milvus with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Milvus through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with a Virtual Key To use Milvus with Obiguard, get your Milvus API key from here, then add it to Obiguard to create your [Milvus virtual key](/virtual-keys#using-virtual-keys). ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key provider="milvus", custom_host="MILVUS_HOST" # Replace with your Milvus host example: https://in03-34d7b37f7ee12c7.serverless.gcp-us-west1.cloud.zilliz.com ) ) ``` ### 3. Use the Obiguard SDK to interact with Milvus ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key custom_host="MILVUS_HOST" # Replace with your Milvus host example: https://in03-34d7b37f7ee12c7.serverless.gcp-us-west1.cloud.zilliz.com ) response = client.post( url="v2/vectordb/collections/list", # Replace with the endpoint you want to call ) print(response) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='MILVUS_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="milvus", obiguard_api_key="vk-obg***", # Your Obiguard virtual key custom_host="MILVUS_HOST" # Replace with your Milvus host example: https://in03-34d7b37f7ee12c7.serverless.gcp-us-west1.cloud.zilliz.com ) ) response = openai.post( url="v2/vectordb/collections/list", # Replace with the endpoint you want to call ) print(response) ``` ```sh theme={null} curl --location --request POST 'https://gateway.obiguard.ai/v1/v2/vectordb/collections/list' \ -H 'x-obiguard-custom-host: https://in03-34d7b37f7de12c7.serverless.gcp-us-west1.cloud.zilliz.com' \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ ``` # Qdrant Source: https://docs.obiguard.ai/integrations/vector-databases/qdrant [Qdrant](https://qdrant.tech/) is an open-source vector similarity search engine built for production-ready vector search applications. It provides a convenient API to store, search, and manage vectors with additional payload data. Obiguard provides a proxy to Qdrant, allowing you to use virtual keys and observability features. ## Obiguard SDK Integration with Qdrant Obiguard provides a consistent API to interact with models from various providers. To integrate Qdrant with Obiguard: ### 1. Install the Obiguard SDK Add the Obiguard SDK to your application to interact with Qdrant through Obiguard's gateway. ```sh theme={null} pip install obiguard ``` ### 2. Initialize Obiguard with a Virtual Key To use Qdrant with Obiguard, get your API key from [Qdrant App](https://cloud.qdrant.io/), then add it to Obiguard to create your [Qdrant virtual key](/virtual-keys#using-virtual-keys). ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key ) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders client = OpenAI( api_key="QDRANT_API_KEY", base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( obiguard_api_key="vk-obg***", # Your Obiguard virtual key provider="qdrant", custom_host="QDRANT_HOST" # Replace with your Qdrant host ) ) ``` ### 3. Use the Obiguard SDK to interact with Qdrant ```python theme={null} from obiguard import Obiguard client = Obiguard( obiguard_api_key="vk-obg***", # Your Obiguard virtual key custom_host="QDRANT_HOST" # Replace with your Qdrant host ) response = client.post( url="https://xxxx-xxx-xxx-xx-xxxxxx.us-west-2-0.aws.cloud.qdrant.io", # Qdrant search endpoint, you can use any Qdrant endpoint ) print(response) ``` ```python theme={null} from openai import OpenAI from obiguard import OBIGUARD_GATEWAY_URL, createHeaders openai = OpenAI( api_key='QDRANT_API_KEY', base_url=OBIGUARD_GATEWAY_URL, default_headers=createHeaders( provider="qdrant", obiguard_api_key="vk-obg***", # Your Obiguard virtual key custom_host="QDRANT_HOST" # Replace with your Qdrant host ) ) response = openai.post( url="https://xxxx-xxx-xxx-xx-xxxxxx.us-west-2-0.aws.cloud.qdrant.io", # Qdrant search endpoint, You can use any Qdrant endpoint ) print(response) ``` ```sh theme={null} curl --location --request POST 'https://xxxx-xxx-xxx-xx-xxxxxx.us-west-2-0.aws.cloud.qdrant.io' \ -H 'x-obiguard-custom-host: $QDRANT_HOST' \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ ``` # Making Your First Request Source: https://docs.obiguard.ai/making-your-first-request Integrate Obiguard and secure your first LLM call in 2 minutes! # 1. Create your guardrail policy [Create](https://app.obiguard.ai/auth) or [log in](https://app.obiguard.ai/sign-in) to your Obiguard account. Identify your orgnization and project. Go to Guardrail Policies and choose the default Guardrail Policy or create a new one. # 2. Get your Obiguard API Key Grab your guardrail's API key from the "API Keys" page. Copy your Obiguard API key Based on your access level, you might see the relevant permissions on the API key modal - tick the ones you'd like, name your API key, and save it. ## 3. Integrate Obiguard Obiguard offers a variety of integration options, including SDKs, REST APIs, and native connections with platforms like OpenAI, Langchain, and LlamaIndex, among others. ### Through the OpenAI SDK If you're using the **OpenAI SDK**, import the Obiguard SDK and configure it within your OpenAI client object: ### Obiguard SDK You can also use the **Obiguard SDK / REST APIs** directly to make the chat completion calls. This is a more versatile way to make LLM calls across any provider: Once, the integration is ready, you can view the requests reflect on your Obiguard dashboard. ### Other Integration Guides ## 3. Next Steps Now that you're up and running with Obiguard, you can dive into the various Obiguard features to learn about all of the supported functionalities: # Analytics Source: https://docs.obiguard.ai/observability/analytics Obiguard Analytics provides a powerful dashboard for visualizing and analyzing your LLM request data. Once Obiguard is integrated, you gain immediate access to comprehensive, real-time analytics on cost, latency, and accuracy for all your LLM requests. The analytics dashboard offers an interactive experience to help you understand your LLM application. It displays a variety of graphs and metrics, including request distribution across LLMs, costs, latencies, token usage, user activity, feedback, cache hits, errors, and more. These metrics in the Analytics section enable you to assess your application's efficiency, uncover trends, identify optimization opportunities, and more. ## Charts The dashboard delivers detailed insights into users, errors, cache activity, feedback, and aggregates data by metadata. ### Overview The overview tab provides a high-level summary of your application’s performance, highlighting total cost, tokens consumed, average latency, request volume, and key information about users and top models. Start here to get a broad understanding before exploring specific metrics. ### Errors Obiguard automatically tracks both API and accuracy errors. The error charts provide a quick overview of error rates, helping you identify and troubleshoot issues efficiently. You can also see how many requests were recovered by Obiguard using various AI gateway strategies. # Logs Source: https://docs.obiguard.ai/observability/logs The Logs section displays a time-ordered record of all requests handled by Obiguard. Each log entry provides useful data such as the timestamp, request type, LLM used, tokens generated, thinking tokens and cost. For multimodal models, Logs will also show the image sent with vision/image models, as well as the image generated. By clicking on an entry, a side panel opens up, revealing the entire raw data with the request and response objects. This detailed log can be invaluable when troubleshooting issues or understanding specific interactions. It provides full transparency into each request and response, enabling you to see exactly what data was sent and received. ## Share Logs with Teammates Every log entry in Obiguard has its own unique URL. Simply copy the link from your browser’s address bar to share it directly with anyone in your organization. ## Request Status Guide The `Status` column on the Logs page provides an overview of gateway activity for each request. You can easily monitor and optimize your usage, as Obiguard tracks all gateway features here along with their current states (such as disabled, triggered, etc.). ## Debug Requests with Log Replay Quickly troubleshoot issues by replaying any request directly from the log details page. Click the Replay button to open the request in a new prompt playground, where you can modify and rerun it until you achieve the desired result. The Replay button will be disabled in the following scenarios: * The request was sent to an endpoint other than `/chat/completions`, `/completions`, or `/embeddings` * The virtual key used in the log has been deleted on Obiguard ## DO NOT TRACK The `DO NOT TRACK` option lets you process requests without storing the request and response data in the logs. When this is enabled, only summary metrics such as tokens used, cost, and latency are recorded; the actual request and response content is excluded. This is useful for handling sensitive information or meeting data privacy requirements, as it allows you to monitor key operational metrics without logging confidential data. To activate DO NOT TRACK for a request, set the debug flag to false when creating your Obiguard or OpenAI client, or add the `x-obiguard-debug:false` header to your request. ```python theme={null} from obiguard import Obiguard client = Obiguard( provider='openai', base_url='https://gateway.obiguard.ai/v1', obiguard_api_key='vk-obg***', # Your Obiguard virtual key here debug=False, # Disable logging for this request ) ``` ``` from openai import OpenAI from obiguard import createHeaders client = OpenAI( base_url='https://gateway.obiguard.ai/v1', api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="sk-obg***", # Your Obiguard API key here debug=False, ) ) ``` ``` curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "x-obiguard-api-key: $OBIGUARD_API_KEY" \ -H "x-obiguard-provider: openai" \ -H "x-obiguard-debug: false" \ -d '{ "model": "gpt-4o-mini", "messages": [{ "role": "user", "content": "Hello!" }] }' ``` # Observability Source: https://docs.obiguard.ai/observability/overview Monitor performance, analyze metrics, and accelerate troubleshooting with our all-in-one observability platform. Gaining visibility into all your LLM requests can be challenging. How do you effectively track and measure cost, latency, and accuracy? Obiguard’s observability suite offers comprehensive monitoring for every request, while its analytics dashboards deliver the actionable insights you need—quickly. ## Features Obiguard records all your multimodal requests and responses, making it easy to view, monitor, and debug interactions. Obiguard supports request tracing to help you monitor your applications throughout the lifecycle of a request. Obiguard supports request tracing to help you monitor your applications throughout the lifecycle of a request. Analytics # Tracing Source: https://docs.obiguard.ai/observability/tracing Obiguard Tracing lets you track the full lifecycle of your LLM requests in a single, chronological view. This is perfect for agentic workflows, chatbots, or multi-step LLM calls, by helping you understand and optimize your AI application’s performance. ## How Tracing Works Obiguard uses OpenTelemetry-compliant tracing to monitor your LLM requests. By attaching a trace ID to your requests, all related LLM calls are automatically grouped in the Traces View and displayed as “spans” within a trace. A “span” represents a subgroup of LLM calls. Depending on your instrumentation, a span can refer to a group within a trace or a single LLM call. ## Trace Tree Structure Obiguard implements tracing using a tree-like structure, similar to OpenTelemetry. Each node in this tree is a span, identified by a unique `spanId` and an optional `spanName`. Child spans reference their parent using `parentSpanId`. Spans without a parent become root nodes. ``` traceId ├─ parentSpanId │ ├─ spanId │ ├─ spanName ``` | Key | Node | Key - Python | Expected Value | Required? | | ------------ | ---------------- | ------------- | -------------- | --------- | | traceId | trace\_id | Unique string | YES | | | spanId | span\_id | Unique string | NO | | | spanName | span\_name | string | NO | | | parentSpanId | parent\_span\_id | Unique string | NO | | ## Benefits of Tracing * **Cost Analysis**: Monitor total LLM costs for each trace. * **Debugging**: Quickly review all related requests in a trace and spot issues. * **Performance Monitoring**: Gain visibility into the full request lifecycle and overall trace timing. # Projects Source: https://docs.obiguard.ai/projects Developers frequently rely on multiple GenAI-powered applications to handle various use cases, with each application comprising several components that utilize LLMs for different tasks. Obiguard is built to seamlessly protect and operate across every part of your application that leverages GenAI for your agentic systems. We use the concept of a project to define, configure, monitor, and analyze the security and threat profiles of each AI-driven application and its components. A project represents a collection of Obiguard guardrail policies that you can oversee for performance and manage through a unified defense policy configuration. Your projects can represent various applications, environments, individual components of an app, or any combination of these. Projects enable you to: * Customize and fine-tune defenses for each use case by linking them to specific policies. * Monitor external, customer-facing GenAI applications separately from internal GenAI tools * Compare performance across development and production environments * Pinpoint exactly which part of your application is being targeted or exploited For instance, you could create a project for your production customer support chatbot and another for your internal employee knowledge base Q\&A tool. Alternatively, you might set up separate projects for the test and production environments of each application to monitor and compare them independently. ## Managing Projects in Obiguard Customers can manage their projects within the Obiguard app platform user interface. The Projects page allows you to create new projects and lists each of your existing projects along with its details. Each project is given a unique ID and contains a set of resources, including API and virtual keys, guardrail policies, and trace data. If your application needs to apply different policies based on specific scenarios, you’ll need to create multiple projects to support that setup. Each project may contain multiple policy groups, which defines the defense configuration that Obiguard will use when running screening requests for that project. If you wish for your application to use different policies depending on the situation then you will need to create multiple policy groups, one for each situattion, with the relevant policies for grouping. Then generate an API key for each policy group to use in your application. ## Creating a Project Only admins create projects. When you first sign up for Obiguard, you will be prompted to create an organization and a project. This is a one-time setup process that allows you to start using Obiguard immediately. You can create additional projects at any time through the Projects page in the Obiguard app platform user interface. You can create additional organizations, and during this time you will be prompted to create a project for that organization as well. Create a project The only required field is the project name. The project name must be unique within your organization. You can optionally add a project description to the project, which can be helpful for other users to understand the purpose of the project. Next, create a guardrail policy for the project from the sidebar. A guardrail policy defines the rules and configurations that Obiguard will use to protect your application. ## Editing projects Projects can only be edited by admins. Click on the three dots next to the project's name to access options for viewing details, editing, or deleting it. When you select Edit, a popup will appear where you can modify the project's name and description, then save your changes. ## Deleting projects Projects can only be deleted by admins. Admins can delete a project by clicking on the three dots next to the project's name and selecting the delete option. Confirm the deletion in the popup that appears. Deleting projects is permanent and cannot be undone. Once a project is deleted, its historic requests will no longer be linked to any project, meaning they will lose access to the deleted project's logs and guardrail policies. Additionally, all policies and API keys associated with the project will be removed. Requests using these API keys will no longer be protected by the project's guardrail policies, and will experience errors. # Common Errors and Resolutions Source: https://docs.obiguard.ai/support/common-errors-and-resolutions Common errors encountered when using the Obiguard Gateway and their resolutions. ## Identifying the Source of Errors 1. Errors that originate from Obiguard are prefixed with `Obiguard Error`. 2. Errors from your LLM providers (or frameworks) are returned without any modifications. ## Verifying if the Error is from Obiguard To determine if the issue is caused by Obiguard, try executing the same request without using Obiguard. If the request succeeds, the problem likely lies with Obiguard or its integration. ## Common Obiguard Errors 1. **Missing Mandatory Headers**: This error occurs when required headers are not included in the request. Ensure all necessary headers specified in the feature documentation are present. 2. **Invalid Header Values**: This error happens when a header contains an incorrect or unsupported value. Verify that the provided values match the allowed ones in the documentation. # Contact Us Source: https://docs.obiguard.ai/support/contact-us Even with thorough troubleshooting and testing, you might encounter unresolved issues. Contact the Obiguard team below, and we will respond as soon as possible. [support@obiguard.com](mailto:support@obiguard.com) To help us resolve your issue quickly and accurately, please provide as much of the following information as possible: * **Issue Description**: Include any error messages and describe the behavior you’re experiencing, highlighting how it differs from your expectations. * **Steps to Reproduce**: Provide clear, step-by-step instructions to help us replicate the issue. * **Code Samples**: Share relevant code snippets causing the issue, ensuring sensitive data is removed. * **Request and Response Data**: If applicable, include the request details and the response received, with sensitive information redacted. * **Screenshots or Recordings**: Visual aids like screenshots or screen recordings can help us diagnose the issue faster. * **Environment Details**: Specify your environment, such as programming language, library versions, and whether the issue occurs in all environments (e.g., development, production). # Virtual Keys Source: https://docs.obiguard.ai/virtual-keys Obiguard’s virtual key system lets you securely store your LLM API keys in our vault and manage them easily using a unique virtual identifier. This feature offers several advantages: * Simplified key rotation * Ability to create multiple virtual keys linked to a single API key * Flexible restrictions based on cost, request volume, or user access You can manage these options in your account under the **Virtual Keys** tab. ## Creațing a Virtual Key To set up a virtual key, follow these instructions: 1. Go to the **Virtual Keys** tab in your Obiguard sidebar. 2. Select **Create**. 3. Enter a unique name to your key, select the guardrail policy to apply on this key, choose your AI provider, and specify any usage details if necessary. If your AI provider is not listed, you can select **Local/Privately hosted provider** and enter the API endpoint URL. Tip: You can add multiple keys for a single provider or use distinct names for the same key to simplify management. ## How are the provider API keys stored? Your API keys are encrypted and stored in secure vaults, accessible only during a request. Decryption happens solely within isolated workers and only when required, providing maximum data security. ## How are provider keys associated with virtual keys? Virtual keys are generated randomly and mapped to the securely stored provider keys. This ensures that your original API keys cannot be derived or reconstructed from the virtual keys. ## Using Virtual Keys ### Using the Obiguard SDK Add the virtual key to the Obiguard Python SDK initialization configuration. ```python theme={null} from obiguard import Obiguard client = Obiguard( provider='openai', base_url='https://gateway.obiguard.ai/v1', obiguard_api_key='vk-obg***', # Your Obiguard virtual key here ) ``` ### Using the OpenAI SDK Add the virtual key directly to the initialization configuration for the OpenAI client. ```python theme={null} from openai import OpenAI from obiguard import createHeaders client = OpenAI( base_url='https://gateway.obiguard.ai/v1', api_key='sk-***', # Your OpenAI API key here default_headers=createHeaders( provider="openai", api_key="vk-obg***", # Your Obiguard virtual key here ) ) ``` ### Using cURL Add the virtual key directly to the request headers of your API call. ```curl theme={null} curl https://gateway.obiguard.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-obiguard-api-key: vk-obg***" \ -d '{ "model": "Qwen/Qwen2.5-32B-Instruct", "messages": [{ "role": "user", "content": "Hello!" }] }' ``` # What is Obiguard? Source: https://docs.obiguard.ai/what-is-obiguard Obiguard is a comprehensive platform designed to streamline the security of agentic AI systems for developers and organizations. It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in your agentic AI apps. It takes 2 minutes to integrate and with that, it starts monitoring all of your LLM requests and makes your app resilient, secure, performant, and more accurate at the same time. ## Features A collection of policies that you can oversee and manage. Enables the screening of LLM interactions to address various threats. Protect your requests and responses. Securely store your LLM API keys in our vault. Monitor performance, analyze metrics, and accelerate troubleshooting A unified interface to interact with 250+ AI models. ## FAQs Obiguard is hosted on computing machines throughout the world, ensuring minimal latency. Our benchmarks estimate a total latency addition between 20-40ms compared to direct API calls. This slight increase is often offset by the benefits of the security of your LLM agents. We maintain best practices for service security, data storage, and retrieval. All data is encrypted in transit and at rest using industry-standard AES-256 encryption. Obiguard is built on scalable infrastructure and can handle millions of requests per minute with very high concurrency. We currently serve over 25M requests daily with a 99.99% uptime. Our edge architecture & scaling capabilities ensure we can accommodate sudden spikes in traffic without performance degradation. We *DO NOT* impose any explicit timeout for our free OR paid plans currently. While we don't time out requests on our end, we recommend implementing client-side timeouts appropriate for your use case to handle potential network issues or upstream API delays. Yes! We support SSO with any custom OIDC provider. We're available on our support email - [support@obiguard.com](mailto:support@obiguard.com) # AI Agent vs Agentic AI Source: https://docs.obiguard.ai/white-papers/ai-agent-vs-agentic-ai Understanding AI Agents and Agentic AI, and the Role of Obiguard ## Executive Summary Artificial intelligence is evolving rapidly, and with it, the terminology and concepts around autonomous systems are becoming more nuanced. Two key terms often discussed are AI agents and agentic AI. While related, they represent different levels of autonomy and complexity in AI systems. Understanding this distinction is crucial for effectively governing and securing these technologies, a role that solutions like Obiguard are uniquely positioned to fulfill. # Banking Source: https://docs.obiguard.ai/white-papers/banking Working Paper: AI-Driven Transaction Governance Using Obiguard in Financial Institutions ## Executive Summary As financial institutions increasingly deploy autonomous AI agents to initiate and manage transactions, robust governance frameworks are essential to ensure compliance, security, and operational integrity. Obiguard, a Malaysian and Bumiputra-owned AI governance Platform-as-a-Service (PaaS), provides real-time policy validation, continuous monitoring, and comprehensive audit trails to govern AI-driven transaction requests. This working paper outlines the transaction workflow using Obiguard, emphasizing alignment with Bank Negara Malaysia (BNM) regulatory standards to ensure responsible AI adoption in banking and credit institutions. # Cloud Source: https://docs.obiguard.ai/white-papers/cloud Working Paper: Obiguard AI Governance Platform in Cloud Data Center Infrastructure ## Executive Summary As financial institutions and enterprises increasingly deploy autonomous AI agents to manage critical transactions and operations, ensuring secure, compliant, and transparent AI governance becomes paramount. Obiguard, a Malaysian and Bumiputra-owned AI governance Platform-as-a-Service (PaaS), provides a comprehensive solution that integrates seamlessly within cloud data center infrastructures. This working paper outlines the workflow of Obiguard’s AI governance platform, highlighting its role in policy validation, real-time monitoring, and audit trail maintenance to ensure responsible AI-driven transaction management aligned with Bank Negara Malaysia (BNM) regulatory standards # Cryptocurrencies Source: https://docs.obiguard.ai/white-papers/cryptocurrencies Working Paper: Leveraging Obiguard for AI Governance in Cryptocurrency Platforms ## Executive Summary The cryptocurrency industry is rapidly evolving, driven by innovations in blockchain technology and autonomous AI agents that optimize trading, asset management, and fraud detection. However, the decentralized and volatile nature of crypto markets introduces significant governance, security, and compliance challenges. This working paper explores how Obiguard, a Malaysian and Bumiputra-owned AI governance Platform-as-a-Service (PaaS), addresses these challenges by providing real-time validation, policy enforcement, risk management, and regulatory compliance for AI-driven cryptocurrency operations. # Obiguard for CEOs Source: https://docs.obiguard.ai/white-papers/for-ceos Building Trusted and Safe Agents: An AI White Paper for CEOs ## Executive Summary Autonomous AI agents are transforming enterprise operations by automating complex tasks at unprecedented speed and scale. However, this autonomy introduces new risks that can lead to security breaches, regulatory violations, operational disruptions, and reputational damage. This white paper outlines the critical importance of embedding safety and governance directly inside AI agents. It presents real-world risk examples across financial services, insurance, cryptocurrency, and government sectors, and introduces Obiguard as a guardian, policy safeguard, and governance caretaker. Obiguard acts as a specialized firewall between AI agents and AI models, enforcing real-time controls and ensuring compliance. CEOs and enterprise leaders will find a practical blueprint for deploying trusted and safe AI agents that balance innovation with risk management. # Obiguard White Paper Source: https://docs.obiguard.ai/white-papers/our-white-paper Obiguard: A Secure, Scalable, and High-Performance Platform for AI Agent Governance ## Abstract The rapid adoption of autonomous artificial intelligence (AI) agents across industries introduces complex governance challenges, including security vulnerabilities, regulatory compliance, and ethical considerations. This white paper presents Obiguard, an embedded AI governance platform that delivers continuous, real-time oversight and control of autonomous AI agents. By integrating governance mechanisms directly within AI agents, Obiguard enables scalable, high-performance management that adapts dynamically to evolving regulatory and organizational demands. This paper details Obiguard’s architecture, core functionalities, and practical applications, demonstrating its effectiveness in managing AI governance complexities while maintaining operational efficiency. The findings suggest that embedding governance at the agent level is a critical strategy for secure, compliant, and transparent AI deployment at scale.