Appearance
User Guide
Complete guide to implementing and using elsai ARMS in your AI projects
Core Functionalities
Import and Initialization
Import the Library
python
from elsai_arms.elsai_arms import ElsaiARMSInitialize ARMS
python
arms = ElsaiARMS('project_name')This sets up the monitoring environment for a new or existing project. It will:
- Check if the project exists
- Create a new project if needed
- Load the latest project session data
LLM Monitoring
LLM Interaction Monitoring
Capture comprehensive metrics for each LLM call including latency, token usage, cost, and governance metrics.
Basic Metrics
Performance Metrics
Content and Cost
Governance and Safety Metrics
Content Safety
- Hate Speech: Risk assessment (Low/Medium/High)
- Violence: Risk assessment (Low/Medium/High)
- Overall Safety: Comprehensive rating
Prompt Security
- Injection Detection: Prompt manipulation risk
- Integrity Check: Prompt validation status
Quality Assessment
- Response Quality: Overall assessment score
- Relevance: Query-response alignment
Implementation Example
python
@arms.monitor_llm_call
def get_response(prompt: str):
return llm.invoke(prompt)Streaming LLM Monitoring
For async streaming LLM calls, use the monitor_llm_astream decorator:
python
from langchain_core.messages import HumanMessage
from langchain_openai import ChatOpenAI
async def main():
@arms.monitor_llm_astream
async def run_astream(prompt, llm):
return llm.astream_events([HumanMessage(content=prompt)])
llm = ChatOpenAI(model="gpt-4o-mini", streaming=True)
async for event in run_astream("Explain quantum computing in simple terms.", llm=llm):
if event["event"] == "on_chat_model_stream":
print(event["data"]["chunk"].content, end="", flush=True)OCR Monitoring
OCR Operation Monitoring
Track text extraction performance, confidence scores, and processing efficiency across multiple OCR models.
Operation Metrics
Performance Metrics
Supported OCR Models
EasyOCROpen Source
AWS TextractCloud Service
Azure Document IntelligenceCloud Service
Azure Computer VisionCloud Service
Google Vision AICloud Service
PaddleOCROpen Source
TesseractOpen Source
Vision LLMAI-Powered
Implementation Example
python
@arms.monitor_ocr_call("OCR_name")
def extract_text(image_path: str):
return ocr_model.extract(image_path)RAG Monitoring
RAG Operation Monitoring
Track document retrieval, query processing, and relevance scoring for your RAG systems.
Query Metrics
Retrieval Metrics
Performance Metrics
Error Handling
Implementation Example
python
@arms.monitor_rag_call
def retrieve_documents(query: str):
return rag_system.search(query)Embedding Monitoring
Embedding Operation Monitoring
Monitor vector generation, processing performance, and cost efficiency.
Input Metrics
Performance Metrics
Error Handling
Implementation Example
python
@arms.monitor_embedding_call
def get_embedding(text: str):
return embedding_model.encode(text)Agent Monitoring
On-prem ARMS supports two agent integration paths. See the Agent Monitoring overview for a full comparison.
LangChain and LangGraph
LangChain Agent Monitoring
Monitor LangChain agents and graphs by adding arms.langchain_callback to track agent execution, tool usage, and overall performance.
Agent Information
Performance Metrics
LLM Interactions
python
from langchain_core.messages import HumanMessage
from langgraph.graph import StateGraph
graph = StateGraph(...)
messages = [HumanMessage(content="Your query here")]
result = graph.invoke(
{"messages": messages},
config={"callbacks": [arms.langchain_callback]}
)elsai Agents (on-prem)
elsai Agents Hook Monitoring
Monitor elsai Agents SDK applications by registering arms.elsai_agents_hook on the manager or orchestrator. Child agents, Graph nodes, Swarm handoffs, and as_tool calls are traced automatically.
Span types
Run metrics
python
from elsai import Agent
from elsai.agent import AgentConfig
agent = Agent(
model=model,
tools=[...],
config=AgentConfig(
name="manager",
hooks=[arms.elsai_agents_hook],
),
)
result = await agent.invoke_async("Your query here")
arms.end_run()Not the same as monitor_llm_call
@arms.monitor_llm_call records individual LLM calls in llm_calls. The elsai-agents hook builds a full distributed trace tree. Use the hook for agent workflows.
Full guide: elsai Agents monitoring
Additional Features
Model Resolution
elsai ARMS implements a dual-stage model resolution pipeline. If an explicit model name is not provided, the system automatically switches to auto-extraction to ensure accurate telemetry and cost tracking.
Step 1: Explicit Override
Manually specify the model name to ensure consistent naming. This takes the highest priority and bypasses automatic extraction.
For Direct LLM Calls
python
@arms.monitor_llm_call(model="gpt-4o")
def get_response(prompt: str):
return llm.invoke(prompt)For Agents and LangChain
python
# Pass override in configuration metadata
config = {
"callbacks": [arms.langchain_callback],
"metadata": {"arms_override_model": "gpt-4o"}
}
result = agent.invoke({"input": query}, config=config)Step 2: Auto-Extraction
If no override is provided in Step 1, ARMS automatically switches to extracting model details from raw API responses or execution contexts. This ensures monitoring remains active even without manual configuration.
Custom Metrics
Track custom business metrics and internal KPIs as key-value pairs.
python
arms.log_custom_metric('Metric Name', metric_value)Built-in Logging
elsai ARMS provides built-in logging for tracking important events and errors during project execution.
Info Logs
python
arms.info('Log Operation')Warning Logs
python
arms.warning('Log Warning')Error Logs
python
arms.error('Log Error')Data Export
Export comprehensive project data for analysis, reporting, or integration with external systems.
python
arms.export()Session Management
Finalize and complete your monitoring session for successful project runs.
python
arms.end_run()