Ollama

Run open-source models locally with Ollama — no API key required.

Install

pip install --extra-index-url https://core-packages.elsai.ai/root/elsai-model/ elsai-model

# Install Ollama itself (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull llama3

Basic usage

Set the Ollama API base URL in your environment, and use LiteLLMConnector from elsai_model:

export OLLAMA_API_BASE="http://localhost:11434"

from elsai import Agent
from elsai_model.litellm import LiteLLMConnector

model = LiteLLMConnector(
    model_name="ollama/llama3",
)

agent = Agent(model=model)
result = agent("Explain how neural networks work")

Configuration

import os
from elsai_model.litellm import LiteLLMConnector

# Set custom host/port if running Ollama remotely
os.environ["OLLAMA_API_BASE"] = "http://localhost:11434"

model = LiteLLMConnector(
    model_name="ollama/mistral:7b",
    temperature=0.3,
)

Popular models

ollama pull llama3         # Meta Llama 3 8B
ollama pull llama3:70b     # Meta Llama 3 70B
ollama pull mistral        # Mistral 7B
ollama pull codellama      # Code Llama
ollama pull phi3           # Microsoft Phi-3
ollama pull gemma2         # Google Gemma 2

Remote Ollama server

Point to any Ollama instance:

import os
from elsai_model.litellm import LiteLLMConnector

os.environ["OLLAMA_API_BASE"] = "http://your-server:11434"
model = LiteLLMConnector(
    model_name="ollama/llama3",
)