Lepton AI

Obiguard provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including Lepton AI APIs.

With Obiguard, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a virtual key system.

​

Obiguard SDK Integration with Lepton AI Models

Obiguard provides a consistent API to interact with models from various providers. To integrate Lepton AI with Obiguard:

​

1. Install the Obiguard SDK

Add the Obiguard SDK to your application to interact with Lepton AI’s API through Obiguard’s gateway.

pip install obiguard

pip install obiguard

​

2. Initialize Obiguard with the Virtual Key

To use Lepton AI with Obiguard, get your API key from Lepton AI, then add it to Obiguard to create the virtual key.

from obiguard import Obiguard

client = Obiguard(
  obiguard_api_key="sk-obg***", # Your Obiguard API key
  virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Lepton
)

from obiguard import Obiguard

client = Obiguard(
  obiguard_api_key="sk-obg***", # Your Obiguard API key
  virtual_key="VIRTUAL_KEY" # Replace with your virtual key for Lepton
)

​

3. Invoke Chat Completions with Lepton AI

Use the Obiguard instance to send requests to Lepton AI. You can also override the virtual key directly in the API call if needed.

completion = client.chat.completions.create(
  messages= [{"role": 'user', "content": 'Say this is a test'}],
  model= 'llama-3-8b-sft-v1'
)

print(completion)

completion = client.chat.completions.create(
  messages= [{"role": 'user', "content": 'Say this is a test'}],
  model= 'llama-3-8b-sft-v1'
)

print(completion)

curl --location 'https://gateway.obiguard.ai/v1/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'x-obiguard-api-key: $OBIGUARD_API_KEY' \
  -H 'x-obiguard-virtual-key: VIRTUAL_KEY' \
  --data '{
    "model": "llama-3-8b-sft-v1",
    "messages": [
      {
        "role": "user",
        "content": "Say this is a test"
      }
    ]
  }'

​

Speech-to-Text (Transcription)

Lepton AI provides speech-to-text capabilities through Obiguard’s unified API:

with open("audio.mp3", "rb") as audio_file:
  transcription = client.audio.transcriptions.create(
    file=audio_file,
    model="whisper-large-v3"
  )

print(transcription.text)

with open("audio.mp3", "rb") as audio_file:
  transcription = client.audio.transcriptions.create(
    file=audio_file,
    model="whisper-large-v3"
  )

print(transcription.text)

​

Advanced Features

​

Streaming Responses

Lepton AI supports streaming responses to provide real-time generation:

stream = client.chat.completions.create(
  messages=[{"role": "user", "content": "Write a story about a robot"}],
  model="llama-3-8b-sft-v1",
  stream=True
)

for chunk in stream:
  if chunk.choices[0].delta.content:
    print(chunk.choices[0].delta.content, end="")

stream = client.chat.completions.create(
  messages=[{"role": "user", "content": "Write a story about a robot"}],
  model="llama-3-8b-sft-v1",
  stream=True
)

for chunk in stream:
  if chunk.choices[0].delta.content:
    print(chunk.choices[0].delta.content, end="")

​

Next Steps

The complete list of features supported in the SDK are available on the link below.