LocalAI

Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including your locally hosted models through LocalAI.

Obiguard SDK Integration with LocalAI

1. Install the Obiguard SDK

Python SDK

pip install obiguard

2. Initialize Obiguard with LocalAI URL

First, ensure that your API is externally accessible. If you’re running the API on http://localhost, consider using a tool like ngrok to create a public URL. Then, instantiate the Obiguard client by adding your LocalAI URL (along with the version identifier) to the customHost property, and add the provider name as openai.

Note: Don’t forget to include the version identifier (e.g., /v1) in the customHost URL

Python SDK

from obiguard import Obiguard

client = Obiguard(
  obiguard_api_key="ak***",  # Your Obiguard access key
  provider="openai",
  custom_host="https://7cc4-3-235-157-146.ngrok-free.app/v1" # Your LocalAI ngrok URL
)

Obiguard currently supports all endpoints that adhere to the OpenAI specification. This means, you can access and observe any of your LocalAI models that are exposed through OpenAI-compliant routes.

List of supported endpoints here.

3. Invoke Chat Completions

Use the Obiguard SDK to invoke chat completions from your LocalAI model, just as you would with any other provider.

Python SDK

completion = client.chat.completions.create(
    messages= [{ "role": 'user', "content": 'Say this is a test' }],
    model= 'ggml-koala-7b-model-q4_0-r2.bin'
)

print(completion)

Using Provider Credentials and Access Keys

Obiguard stores the credentials for your provider in its vault as a provider credential, and your application authenticates to the gateway with an access key. This simplifies the use of multiple providers and Obiguard features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model.

Add your self-hosted model to Obiguard as a provider credential
Configure your deployment:
- Select the matching provider API specification (typically OpenAI)
- Enter your model’s base URL as the custom host
- Add required authentication headers and their values
Create an access key for the agent or project that will call it

You can now use the access key in your requests:

Python SDK

from obiguard import Obiguard

client = Obiguard(
  obiguard_api_key="ak***",  # Your Obiguard access key
)

response = client.chat.completions.create(
  model="your-self-hosted-model-name",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

LocalAI Endpoints Supported

Endpoint	Resource
/chat/completions (Chat, Vision, Tools support)	Doc
/images/generations	Doc
/embeddings	Doc
/audio/transcriptions	Doc

Next Steps

Explore the complete list of features supported in the SDK:

Ecosystem

AI Apps

Agents

LLMs

Vector Databases

Obiguard SDK Integration with LocalAI

1. Install the Obiguard SDK

2. Initialize Obiguard with LocalAI URL

3. Invoke Chat Completions

Using Provider Credentials and Access Keys

LocalAI Endpoints Supported

Next Steps

SDK

​Obiguard SDK Integration with LocalAI

​1. Install the Obiguard SDK

​2. Initialize Obiguard with LocalAI URL

​3. Invoke Chat Completions

​Using Provider Credentials and Access Keys

​LocalAI Endpoints Supported

​Next Steps

SDK

Obiguard SDK Integration with LocalAI

1. Install the Obiguard SDK

2. Initialize Obiguard with LocalAI URL

3. Invoke Chat Completions

Using Provider Credentials and Access Keys

LocalAI Endpoints Supported

Next Steps