elsai Model

Package: elsai-model v2.0.0

Unified LLM providers with a consistent .invoke() and .stream() / .stream_text() API across backends.

Legacy connector API

The v1.4.x *Connector and GeminiService API is archived at v2.0 LLM Models.

For agent usage (Agent(model=…)), see Model Providers in the elsai Agents docs.

Installation

bash

pip install --extra-index-url https://core-packages.elsai.ai/root/elsai-model/ elsai-model==2.0.0

Requirements: Python >= 3.10, provider credentials in environment or client_args

Optional package extras

Most providers work with the base install above. The following need optional extras — extra pip dependency groups on elsai-model:

bash

# LangChain backend for OpenAI / Azure OpenAI (implementation="langchain")
pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[langchain]==2.0.0"

# OpenAI Responses API (OpenAIResponsesModel)
pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[openai-responses]==2.0.0"

# Meta Llama API (LlamaAPIModel)
pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[llamaapi]==2.0.0"

# Amazon SageMaker endpoints (SageMakerAIModel)
pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[sagemaker]==2.0.0"

Extra	Install	Used for
`langchain`	`elsai-model[langchain]==2.0.0`	`implementation="langchain"` on `OpenAIModel` / `AzureOpenAIModel`
`openai-responses`	`elsai-model[openai-responses]==2.0.0`	`OpenAIResponsesModel`
`llamaapi`	`elsai-model[llamaapi]==2.0.0`	`LlamaAPIModel`
`sagemaker`	`elsai-model[sagemaker]==2.0.0`	`SageMakerAIModel`

Supported providers

Provider	Model class	Import
OpenAI	`OpenAIModel`	`elsai_model.openai`
Azure OpenAI	`AzureOpenAIModel`	`elsai_model.azure_openai`
Amazon Bedrock	`BedrockModel`	`elsai_model.bedrock`
Anthropic (direct)	`AnthropicModel`	`elsai_model.anthropic`
Anthropic via Bedrock SDK	`AnthropicBedrockModel`	`elsai_model.anthropic_bedrock`
Google Gemini	`GeminiModel`	`elsai_model.gemini`
LiteLLM	`LiteLLMModel`	`elsai_model.litellm`
Ollama	`OllamaModel`	`elsai_model.ollama`
Mistral	`MistralModel`	`elsai_model.mistral`
Writer	`WriterModel`	`elsai_model.writer`
Meta Llama API	`LlamaAPIModel`	`elsai_model.llamaapi`
llama.cpp server	`LlamaCppModel`	`elsai_model.llama_cpp`
OpenAI Responses	`OpenAIResponsesModel`	`elsai_model.openai_responses`
SageMaker	`SageMakerAIModel`	`elsai_model.sagemaker`

OpenAI

Environment variables: OPENAI_API_KEY, OPENAI_MODEL_NAME

python

import os
from elsai_model.openai import OpenAIModel

model = OpenAIModel(
    model_id=os.getenv("OPENAI_MODEL_NAME", "gpt-4o-mini"),
    client_args={"api_key": os.environ["OPENAI_API_KEY"]},
    params={"temperature": 0.2},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.choices[0].message.content)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

LangChain backend

Use the LangChain-backed client instead of the native OpenAI SDK. Requires the langchain extra:

bash

pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[langchain]==2.0.0"

python

model = OpenAIModel(
    model_id="gpt-4o-mini",
    client_args={"api_key": os.environ["OPENAI_API_KEY"]},
    params={"temperature": 0.2},
    implementation="langchain",  # default is "native"
)

Bedrock Mantle routing

Route OpenAIModel through Amazon Bedrock's OpenAI-compatible Mantle endpoint (uses AWS credentials, not OPENAI_API_KEY):

python

model = OpenAIModel(
    model_id=os.getenv("BEDROCK_MANTLE_MODEL_ID", "openai.gpt-oss-120b"),
    bedrock_mantle_config={"region": os.getenv("AWS_REGION", "us-east-1")},
    params={"temperature": 0.2, "max_tokens": 256},
)

Azure OpenAI

Environment variables: AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, OPENAI_API_VERSION, AZURE_OPENAI_DEPLOYMENT_NAME

python

import os
from elsai_model.azure_openai import AzureOpenAIModel

model = AzureOpenAIModel(
    model_id=os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"],
    client_args={
        "azure_endpoint": os.environ["AZURE_OPENAI_ENDPOINT"],
        "api_key": os.environ["AZURE_OPENAI_API_KEY"],
        "api_version": os.environ["OPENAI_API_VERSION"],
    },
    params={"temperature": 0.2},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.choices[0].message.content)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

LangChain backend

bash

pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[langchain]==2.0.0"

python

model = AzureOpenAIModel(
    model_id=os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"],
    client_args={
        "azure_endpoint": os.environ["AZURE_OPENAI_ENDPOINT"],
        "api_key": os.environ["AZURE_OPENAI_API_KEY"],
        "api_version": os.environ["OPENAI_API_VERSION"],
    },
    params={"temperature": 0.2},
    implementation="langchain",
)

Amazon Bedrock

Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, BEDROCK_MODEL_ID

python

import os
from elsai_model.bedrock import BedrockModel

model = BedrockModel(
    model_id=os.getenv("BEDROCK_MODEL_ID", "us.anthropic.claude-3-5-sonnet-20241022-v2:0"),
    region_name=os.getenv("AWS_REGION", "us-east-1"),
    max_tokens=256,
    temperature=0.2,
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Configuration

Parameter	Description
`model_id`	Bedrock foundation model ID (e.g. `us.anthropic.claude-3-5-sonnet-20241022-v2:0`)
`region_name`	AWS region where the model is enabled
`max_tokens`	Maximum tokens to generate
`temperature`	Sampling temperature

Credentials are read from the environment (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) or the default AWS credential chain (IAM role, ~/.aws/credentials).

Anthropic (direct API)

Environment variables: ANTHROPIC_API_KEY, ANTHROPIC_MODEL_NAME

python

import os
from elsai_model.anthropic import AnthropicModel

model = AnthropicModel(
    model_id=os.getenv("ANTHROPIC_MODEL_NAME", "claude-3-5-sonnet-latest"),
    max_tokens=256,
    client_args={"api_key": os.environ["ANTHROPIC_API_KEY"]},
    params={"temperature": 0.2},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response["content"][0]["text"])

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Anthropic via Bedrock SDK

Environment variables: AWS credentials + ANTHROPIC_BEDROCK_MODEL_ID

python

import os
from elsai_model.anthropic_bedrock import AnthropicBedrockModel

model = AnthropicBedrockModel(
    model_id=os.getenv("ANTHROPIC_BEDROCK_MODEL_ID", "anthropic.claude-3-5-sonnet-20241022-v2:0"),
    region_name=os.getenv("AWS_REGION", "us-east-1"),
    max_tokens=256,
    temperature=0.2,
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Google Gemini

Environment variables: GEMINI_API_KEY, GEMINI_MODEL_NAME

python

import os
from elsai_model.gemini import GeminiModel

model = GeminiModel(
    model_id=os.getenv("GEMINI_MODEL_NAME", "gemini-2.5-flash"),
    client_args={"api_key": os.environ["GEMINI_API_KEY"]},
    params={"temperature": 0.2},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.text)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

LiteLLM

Route to 100+ providers through a single interface. Browse supported model names at the LiteLLM model hub.

Environment variables: provider keys + LITELLM_MODEL

python

import os
from elsai_model.litellm import LiteLLMModel

model = LiteLLMModel(
    model_id=os.getenv("LITELLM_MODEL", "gpt-4o-mini"),
    client_args={"api_key": os.environ["OPENAI_API_KEY"]},
    params={"temperature": 0.2},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.choices[0].message.content)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Ollama

Ollama runs open-source models locally on your machine. No cloud API key is required — install Ollama, pull a model, then point OllamaModel at your local server.

Local setup

bash

# Install Ollama (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model (downloads weights to ~/.ollama/models)
ollama pull llama3.2

# Verify the server is running (default http://localhost:11434)
ollama list

Environment variables: OLLAMA_HOST (optional, default http://localhost:11434), OLLAMA_MODEL_NAME

python

import os
from elsai_model.ollama import OllamaModel

model = OllamaModel(
    host=os.getenv("OLLAMA_HOST"),  # local server; omit for localhost default
    model_id=os.getenv("OLLAMA_MODEL_NAME", "llama3.2"),
    temperature=0.2,
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.message.content)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Popular local models: llama3.2, mistral, codellama, phi3, gemma2.

Mistral

Environment variables: MISTRAL_API_KEY, MISTRAL_MODEL_NAME

python

import os
from elsai_model.mistral import MistralModel

model = MistralModel(
    api_key=os.environ["MISTRAL_API_KEY"],
    model_id=os.getenv("MISTRAL_MODEL_NAME", "mistral-small-latest"),
    temperature=0.2,
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.choices[0].message.content)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Writer

Environment variables: WRITER_API_KEY, WRITER_MODEL_NAME

python

import os
from elsai_model.writer import WriterModel

model = WriterModel(
    client_args={"api_key": os.environ["WRITER_API_KEY"]},
    model_id=os.getenv("WRITER_MODEL_NAME", "palmyra-x4"),
    temperature=0.2,
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.choices[0].message.content)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Meta Llama API

Hosted Meta Llama models. Requires the llamaapi extra:

bash

pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[llamaapi]==2.0.0"

Environment variables: LLAMA_API_KEY, LLAMA_API_MODEL_ID

python

import os
from elsai_model.llamaapi import LlamaAPIModel

model = LlamaAPIModel(
    model_id=os.environ["LLAMA_API_MODEL_ID"],
    client_args={"api_key": os.environ["LLAMA_API_KEY"]},
    temperature=0.2,
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.choices[0].message.content)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

llama.cpp (local server)

Run a local GGUF model with llama.cpp's HTTP server. Download a .gguf weights file, start the server, then connect with LlamaCppModel.

Local setup

bash

# Download a GGUF model (example — pick a model suited to your hardware)
# https://huggingface.co/models?library=gguf

# Start the llama.cpp server (model must be loaded locally)
llama-server -m /path/to/model.gguf --host 0.0.0.0 --port 8080

Environment variables: LLAMACPP_BASE_URL (default http://localhost:8080), LLAMACPP_MODEL_ID

python

import os
from elsai_model.llama_cpp import LlamaCppModel

model = LlamaCppModel(
    base_url=os.getenv("LLAMACPP_BASE_URL", "http://localhost:8080"),
    model_id=os.getenv("LLAMACPP_MODEL_ID", "default"),
    params={"temperature": 0.2},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response["choices"][0]["message"]["content"])

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

OpenAI Responses

OpenAI's Responses API (distinct from Chat Completions). Requires the openai-responses extra:

bash

pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[openai-responses]==2.0.0"

Environment variables: OPENAI_API_KEY, OPENAI_RESPONSES_MODEL

python

import os
from elsai_model.openai_responses import OpenAIResponsesModel

model = OpenAIResponsesModel(
    model_id=os.getenv("OPENAI_RESPONSES_MODEL", "gpt-4o-mini"),
    client_args={"api_key": os.environ["OPENAI_API_KEY"]},
    params={"temperature": 0.2},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response.output_text)

for chunk in model.stream_text(messages):
    print(chunk, end="", flush=True)

Amazon SageMaker

Invoke a deployed SageMaker endpoint. Requires the sagemaker extra and AWS credentials:

bash

pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[sagemaker]==2.0.0"

Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, SAGEMAKER_ENDPOINT_NAME

python

import os
from elsai_model.sagemaker import SageMakerAIModel

model = SageMakerAIModel(
    endpoint_config={
        "endpoint_name": os.environ["SAGEMAKER_ENDPOINT_NAME"],
        "region_name": os.getenv("AWS_REGION", "us-west-2"),
    },
    payload_config={"max_tokens": 256, "temperature": 0.2, "stream": False},
)
messages = [{"role": "user", "content": "Say hello in one short sentence."}]

response = model.invoke(messages)
print(response["choices"][0]["message"]["content"])

Streaming uses the async stream() API (stream_text() is not available for SageMaker):

python

import asyncio
import os
from elsai_model.sagemaker import SageMakerAIModel


async def stream_response() -> None:
    model = SageMakerAIModel(
        endpoint_config={
            "endpoint_name": os.environ["SAGEMAKER_ENDPOINT_NAME"],
            "region_name": os.getenv("AWS_REGION", "us-west-2"),
        },
        payload_config={"max_tokens": 256, "temperature": 0.2, "stream": True},
    )
    messages = [{"role": "user", "content": [{"text": "Count from 1 to 5, one number per line."}]}]

    async for event in model.stream(messages):
        if "contentBlockDelta" in event:
            delta = event["contentBlockDelta"].get("delta", {})
            if "text" in delta:
                print(delta["text"], end="", flush=True)
    print()


asyncio.run(stream_response())

Migration from v1.4.x connectors

Legacy (v2 docs)	v3 `*Model`
`OpenAIConnector(model_name=…)`	`OpenAIModel(model_id=…, client_args=…)`
`BedrockConnector(aws_access_key=…)`	`BedrockModel(model_id=…, region_name=…)`
`GeminiService.generate_text(…)`	`GeminiModel.invoke(messages)`
`LiteLLMConnector(model_name=…)`	`LiteLLMModel(model_id=…)`
`AnthropicBedrockConnector(…)`	`AnthropicBedrockModel(…)`

Full connector reference: v2.0 LLM Models.

Version history

Version	Changes
2.0.0	Unified `*Model` API; native agent integration; new providers (Mistral, Writer, Llama API, llama.cpp, OpenAI Responses, SageMaker)
1.4.1	Last `*Connector` release — see v2 legacy docs
1.0.0	Initial connector API — see v1 legacy docs

elsai Model ​

Installation ​

Optional package extras ​

Supported providers ​

OpenAI ​

LangChain backend ​

Bedrock Mantle routing ​

Azure OpenAI ​

LangChain backend ​

Amazon Bedrock ​

Configuration ​

Anthropic (direct API) ​

Anthropic via Bedrock SDK ​

Google Gemini ​

LiteLLM ​

Ollama ​

Local setup ​

Mistral ​

Writer ​

Meta Llama API ​

llama.cpp (local server) ​

Local setup ​

OpenAI Responses ​

Amazon SageMaker ​

Migration from v1.4.x connectors ​

Version history ​

elsai Model

Installation

Optional package extras

Supported providers

OpenAI

LangChain backend

Bedrock Mantle routing

Azure OpenAI

LangChain backend

Amazon Bedrock

Configuration

Anthropic (direct API)

Anthropic via Bedrock SDK

Google Gemini

LiteLLM

Ollama

Local setup

Mistral

Writer

Meta Llama API

llama.cpp (local server)

Local setup

OpenAI Responses

Amazon SageMaker

Migration from v1.4.x connectors

Version history