Amazon SageMaker

Invoke your own deployed SageMaker endpoint with SageMakerAIModel. Use this when you host a custom or fine-tuned model on AWS SageMaker.

For standalone invoke / stream usage, see LLM Models — Amazon SageMaker.

Install

pip install --extra-index-url https://core-packages.elsai.ai/root/ "elsai-model[sagemaker]==2.0.0"
pip install --extra-index-url https://elsai-agents.elsai.ai/root/ elsai-agents==0.2.0

Setup

1. Deploy a SageMaker endpoint in your AWS account.
2. Configure AWS credentials and export the endpoint name:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_REGION=us-west-2
export SAGEMAKER_ENDPOINT_NAME=your-endpoint-name

Agent — basic

import os
from elsai import Agent
from elsai_model.sagemaker import SageMakerAIModel

model = SageMakerAIModel(
    endpoint_config={
        "endpoint_name": os.environ["SAGEMAKER_ENDPOINT_NAME"],
        "region_name": os.getenv("AWS_REGION", "us-west-2"),
    },
    payload_config={"max_tokens": 256, "temperature": 0.2},
)
agent = Agent(
    model=model,
    system_prompt="You are a concise assistant.",
)
result = agent("What is 17 + 28? Reply with just the number.")
print(result.message["content"][0]["text"])