elsai Agents Monitoring

Monitor applications built with the elsai Agents SDK using arms.elsai_agents_hook. ARMS listens to elsai-agents hook events and builds a distributed span tree — agent invocations, LLM generations, tool calls, Graph nodes, Swarm handoffs, and nested agent.as_tool() calls.

On-prem only

This integration is available in on-prem ARMS deployments. Install the optional elsai-agents extra as described below.

Prerequisites

elsai ARMS on-prem installed (Installation)
elsai Agents SDK >=0.2.1
Database connection configured (MongoDB, DynamoDB, or ClickHouse)
ELSAI_ARMS_API_KEY and API_BASE_URL configured

Install with elsai-agents support

bash

pip install --extra-index-url https://arms-packages.elsaifoundry.ai/root/elsai-arms/ 'elsai-arms[elsai-agents]'

This installs ARMS plus elsai-agents>=0.2.1. Spawn mode and parallel as_tool tracing require SDK 0.2.1 or later.

Quick start

Register arms.elsai_agents_hook on your manager or orchestrator agent only. Child agents inside Graph nodes or as_tool workers receive the hook automatically — you do not need to pass hooks=[...] on every agent.

python

import asyncio
from elsai import Agent, tool
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai_arms.elsai_arms import ElsaiARMS

@tool(name="echo", description="Echoes the input once.")
def echo(message: str) -> str:
    return f"Echo: {message}"

async def main():
    arms = ElsaiARMS(project_name="my-project")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 400},
    )

    agent = Agent(
        model=model,
        tools=[echo],
        config=AgentConfig(
            name="demo-agent",
            hooks=[arms.elsai_agents_hook],
        ),
    )

    result = await agent.invoke_async("Hello from ARMS")
    print(result)

    arms.end_run()
    arms.flush()

asyncio.run(main())

The hook owns one trace context per ARMS run. Register it on the top-level manager or orchestrator. Nested agents are instrumented via hook injection and SDK propagation.

Integration paths compared

Approach	API	Produces trace tree	Use when
elsai Agents hook	`arms.elsai_agents_hook`	Yes	elsai `Agent`, Graph, Swarm
LangChain callback	`arms.langchain_callback`	Yes	LangChain / LangGraph apps
LLM decorator	`@arms.monitor_llm_call`	No	Single LLM calls only

Do not mix the hook and LangChain callback for the same run. Pick one integration path per application.

Framework tags

ARMS writes metadata.framework on span observations. For elsai Agents hook runs, the value depends on which span produced it — not every span in a Graph or Swarm run shares the same tag.

`framework` value	Where it appears
`elsai_agents`	Agent, generation, and tool spans; node and handoff spans; trace document for standalone agent runs
`elsai_graph`	Orchestrator root chain span only (elsai `Graph` runs)
`elsai_swarm`	Orchestrator root chain span only (elsai `Swarm` runs)

Node and handoff spans keep framework: "elsai_agents" and carry orchestrator context in other metadata — elsai_orchestrator ("graph" or "swarm"), elsai_node_id, handoff_from, handoff_to, and so on.

Trace document vs span metadata

The trace row framework field is set from the first span that opens the trace. For Graph and Swarm runs, agent and generation spans are typically written before the orchestrator span closes at the end of the run. As a result, the trace document often shows framework: "elsai_agents" even when the application used Graph or Swarm.

To identify the orchestration type in the dashboard or exported data:

Find the orchestrator root chain span (parent_span_id is null) and check metadata.framework for elsai_graph or elsai_swarm.
Or read metadata.elsai_orchestrator on any node or handoff span ("graph" or "swarm").

Span hierarchies

Standalone agent

Standalone runs tag all spans and the trace document with framework: "elsai_agents".

Graph orchestrator

The orchestrator root chain span uses framework: "elsai_graph". Agent, generation, tool, and node spans use elsai_agents.

Swarm orchestrator

Swarm adds handoff instant spans when control transfers between agents:

The orchestrator root chain span uses framework: "elsai_swarm". Handoff spans use elsai_agents with handoff_from, handoff_to, and handoff_message metadata.

Nested `agent.as_tool()`

When a manager calls another agent as a tool:

The tool span and child agent may share the same name. Use observation_type and metadata.elsai_as_tool to distinguish them in the dashboard.

Patterns

Standalone agent

Use for a single Agent with tools. The hook creates one trace per invocation with agent, generation, and tool spans underneath.

python

import asyncio
from elsai import Agent, tool
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai_arms.elsai_arms import ElsaiARMS

@tool(name="echo", description="Echoes the input once.")
def echo(message: str) -> str:
    return f"Echo: {message}"

async def main():
    arms = ElsaiARMS(project_name="standalone-agent")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 400},
    )

    agent = Agent(
        model=model,
        tools=[echo],
        system_prompt="Call echo once with the user message, then reply in one sentence.",
        config=AgentConfig(
            name="standalone-agent",
            hooks=[arms.elsai_agents_hook],
        ),
    )

    result = await agent.invoke_async("Hello from ARMS")
    print(result)

    arms.end_run()
    arms.flush()

asyncio.run(main())

All spans use framework: "elsai_agents".

Graph orchestrator

Register the hook on the Graph via GraphBuilder.set_hook_providers. ARMS records orchestrator and node chain spans, then nests each node's agent spans underneath. Do not register the hook on individual leaf agents.

python

import asyncio
from elsai import Agent
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai.multiagent import GraphBuilder
from elsai_arms.elsai_arms import ElsaiARMS

async def main():
    arms = ElsaiARMS(project_name="graph-orchestrator")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 500},
    )

    extractor = Agent(
        model=model,
        system_prompt="Extract key facts from the given text.",
        config=AgentConfig(name="extractor"),
    )
    analyst = Agent(
        model=model,
        system_prompt="Analyse the extracted facts and identify trends.",
        config=AgentConfig(name="analyst"),
    )
    reporter = Agent(
        model=model,
        system_prompt="Write a one-sentence executive summary.",
        config=AgentConfig(name="reporter"),
    )

    builder = GraphBuilder()
    builder.add_node(extractor, "extractor")
    builder.add_node(analyst, "analyst")
    builder.add_node(reporter, "reporter")
    builder.add_edge("extractor", "analyst")
    builder.add_edge("analyst", "reporter")
    builder.set_hook_providers([arms.elsai_agents_hook])

    graph = builder.build()
    result = await graph.invoke_async("Analyse quarterly sales: revenue up 12%, costs flat.")
    print(result.status)

    arms.end_run()
    arms.flush()

asyncio.run(main())

The orchestrator root span is tagged elsai_graph. Node IDs appear in nodes_executed run metrics. See Framework tags for how this differs from the trace document framework field.

Swarm orchestrator

Register the hook on the Swarm instance — not on leaf agents. ARMS records orchestrator and node chain spans plus handoff instant spans when agents transfer control.

python

import asyncio
from elsai import Agent, tool
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai.multiagent import Swarm
from elsai_arms.elsai_arms import ElsaiARMS

@tool(name="calculate", description="Evaluate a simple arithmetic expression.")
def calculate(expression: str) -> str:
    return f"Result: {eval(expression, {'__builtins__': {}}, {})}"

async def main():
    arms = ElsaiARMS(project_name="swarm-orchestrator")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 500},
    )

    researcher = Agent(
        model=model,
        system_prompt=(
            "You are the researcher. Summarise the topic briefly, then hand off to analyst "
            "using handoff_to_agent."
        ),
        config=AgentConfig(name="researcher"),
    )
    analyst = Agent(
        model=model,
        tools=[calculate],
        system_prompt=(
            "You are the analyst. Call calculate once, then hand off to writer "
            "using handoff_to_agent."
        ),
        config=AgentConfig(name="analyst"),
    )
    writer = Agent(
        model=model,
        system_prompt="You are the writer. Reply in one concise sentence.",
        config=AgentConfig(name="writer"),
    )

    swarm = Swarm(
        nodes=[researcher, analyst, writer],
        hooks=[arms.elsai_agents_hook],
        id="swarm-demo",
    )

    result = await swarm.invoke_async(
        "Research AI agent trends, estimate 100 * 1.25, and summarize."
    )
    print(result.status)
    print(result.node_history)

    arms.end_run()
    arms.flush()

asyncio.run(main())

The orchestrator root span is tagged elsai_swarm. Handoff spans record handoff_from, handoff_to, and handoff_message in metadata.

`agent.as_tool()` modes

Register the hook on the manager only. The inner researcher agent does not need hooks=[...] — ARMS injects tracing at tool-call time.

Mode	Behavior	ARMS tracing
`reject`	Same inner agent; concurrent calls rejected	Hook injected on inner agent at tool call
`queue`	Same inner agent; calls serialized	Same as reject
`spawn`	New worker agent per call; parallel OK	Hook propagated to spawn workers via SDK

Reject mode

One tool call per run. A second concurrent call to the same inner agent is rejected.

python

import asyncio
from elsai import Agent
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai_arms.elsai_arms import ElsaiARMS

async def main():
    arms = ElsaiARMS(project_name="as-tool-reject")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 500},
    )

    researcher = Agent(
        model=model,
        system_prompt="Answer in 2-3 short bullet points.",
        config=AgentConfig(name="researcher"),
    )

    manager = Agent(
        model=model,
        tools=[researcher.as_tool()],  # default mode is reject
        system_prompt=(
            "Delegate research to the researcher tool once, then summarize in one sentence."
        ),
        config=AgentConfig(
            name="manager",
            hooks=[arms.elsai_agents_hook],
        ),
    )

    result = await manager.invoke_async("Research current trends in AI agents.")
    print(result)

    arms.end_run()
    arms.flush()

asyncio.run(main())

Queue mode

Same inner agent instance; multiple tool calls are serialized. Use when the manager calls the researcher twice in sequence.

python

import asyncio
from elsai import Agent
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai_arms.elsai_arms import ElsaiARMS

async def main():
    arms = ElsaiARMS(project_name="as-tool-queue")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 500},
    )

    researcher = Agent(
        model=model,
        system_prompt="Answer in 2-3 short bullet points.",
        config=AgentConfig(name="researcher"),
    )

    manager = Agent(
        model=model,
        tools=[researcher.as_tool(mode="queue")],
        system_prompt=(
            "Call the researcher tool twice on different topics, "
            "then summarize both in one sentence."
        ),
        config=AgentConfig(
            name="manager",
            hooks=[arms.elsai_agents_hook],
        ),
    )

    result = await manager.invoke_async(
        "Ask about solar trends, then wind energy outlook, then summarize."
    )
    print(result)

    arms.end_run()
    arms.flush()

asyncio.run(main())

Spawn mode

A new worker agent is created per tool call. Supports parallel calls. Requires elsai-agents>=0.2.1.

python

import asyncio
from elsai import Agent
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai_arms.elsai_arms import ElsaiARMS

async def main():
    arms = ElsaiARMS(project_name="as-tool-spawn")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 500},
    )

    researcher = Agent(
        model=model,
        system_prompt="Answer in 2-3 short bullet points.",
        config=AgentConfig(name="researcher"),
    )

    manager = Agent(
        model=model,
        tools=[researcher.as_tool(mode="spawn")],
        system_prompt=(
            "Call the researcher tool twice in parallel on different topics, "
            "then summarize both in one sentence."
        ),
        config=AgentConfig(
            name="manager",
            hooks=[arms.elsai_agents_hook],
        ),
    )

    result = await manager.invoke_async(
        "Research solar trends and wind energy outlook in parallel, then summarize."
    )
    print(result)

    arms.end_run()
    arms.flush()

asyncio.run(main())

Tool spans include metadata.elsai_as_tool and metadata.elsai_as_tool_mode. Each tool call produces a nested tool → child agent → generation span chain under the manager.

What gets recorded

Span types

agentgenerationtoolorchestrator (chain)node (chain)handoff

Per span

parent_span_idusage_detailscoststatusframework metadata

Run rollup

agent_metrics (trace IDs)trace_completeness_scoreoverall_tool_success_ratenodes_executed

Framework values (span metadata)

`framework` value	Span type	Notes
`elsai_agents`	Agent, generation, tool, node, handoff	Default for all hook-path spans; trace document value for standalone runs
`elsai_graph`	Orchestrator root chain span	Set when `orchestrator_type` is `"graph"`
`elsai_swarm`	Orchestrator root chain span	Set when `orchestrator_type` is `"swarm"`

For Graph and Swarm runs, the trace document framework field may still read elsai_agents because it is derived from the first span written. Use the orchestrator root span or metadata.elsai_orchestrator to distinguish orchestration type.

End of run

Always call arms.end_run() after your agent completes. Token usage and cost are aggregated from generation spans, not from the legacy llm_calls array.

On the hook path:

llm_calls on the run record may be empty — this is expected
Tokens and cost roll up from generation span usage_details
agent_metrics contains trace IDs for dashboard drill-down

python

result = await agent.invoke_async(user_message)
arms.end_run()
arms.flush()  # when using buffered async mode

Complete example

python

import asyncio
from elsai import Agent, tool
from elsai.agent import AgentConfig
from elsai_model.openai import OpenAIModel
from elsai_arms.elsai_arms import ElsaiARMS

@tool(name="echo", description="Echoes the input once.")
def echo(message: str) -> str:
    return f"Echo: {message}"

async def main():
    arms = ElsaiARMS(project_name="elsai-hook-demo")

    model = OpenAIModel(
        model_id="gpt-4o-mini",
        client_args={"api_key": "..."},
        params={"temperature": 0, "max_tokens": 400},
    )

    agent = Agent(
        model=model,
        tools=[echo],
        system_prompt="Call echo once, then reply in one sentence.",
        config=AgentConfig(
            name="clickhouse-demo",
            hooks=[arms.elsai_agents_hook],
        ),
    )

    result = await agent.invoke_async("Hello from ARMS + elsai-agents")
    print("Agent result:", result)

    arms.end_run()
    arms.flush()

    print(arms.export())

asyncio.run(main())

Troubleshooting

Empty LLM calls panel in dashboard

Expected on the hook path. Model usage lives in generation spans and agent_metrics trace IDs, not in llm_calls.

Wrong span tree in UI

Confirm the trace ID matches the latest run
Verify the hook is registered on the manager only, not duplicated on every child agent
Ensure elsai-agents>=0.2.1 for spawn and parallel as_tool modes

`tool_success: 0` with tools visible

Usually means tool spans were not closed. Check that AfterToolCall events fire and that you are on a current ARMS version with as_tool bridge fixes.

Hook not firing

Confirm you installed elsai-arms[elsai-agents]
Confirm hooks=[arms.elsai_agents_hook] is on the orchestrator AgentConfig
Call arms.end_run() after invocation

Mixing integrations

Do not use arms.langchain_callback and arms.elsai_agents_hook in the same run. Use the hook for elsai Agents apps and the callback for LangChain apps.

Next steps

LangChain and LangGraph monitoring — alternative integration path
Agent Monitoring overview — compare frameworks
elsai Agents documentation — SDK reference
User Guide — full monitoring features

elsai Agents Monitoring ​

Prerequisites ​

Install with elsai-agents support ​

Quick start ​

Integration paths compared ​

Framework tags ​

Trace document vs span metadata ​

Span hierarchies ​

Standalone agent ​

Graph orchestrator ​

Swarm orchestrator ​

Nested agent.as_tool() ​

Patterns ​

Standalone agent ​

Graph orchestrator ​

Swarm orchestrator ​

agent.as_tool() modes ​

Reject mode ​

Queue mode ​

Spawn mode ​

What gets recorded ​

Span types

Per span

Run rollup

Framework values (span metadata) ​

End of run ​

Complete example ​

Troubleshooting ​

Empty LLM calls panel in dashboard ​

Wrong span tree in UI ​

tool_success: 0 with tools visible ​

Hook not firing ​

Mixing integrations ​

Next steps ​

elsai Agents Monitoring

Prerequisites

Install with elsai-agents support

Quick start

Integration paths compared

Framework tags

Trace document vs span metadata

Span hierarchies

Standalone agent

Graph orchestrator

Swarm orchestrator

Nested `agent.as_tool()`

Patterns

Standalone agent

Graph orchestrator

Swarm orchestrator

`agent.as_tool()` modes

Reject mode

Queue mode

Spawn mode

What gets recorded

Framework values (span metadata)

End of run

Complete example

Troubleshooting

Empty LLM calls panel in dashboard

Wrong span tree in UI

`tool_success: 0` with tools visible

Hook not firing

Mixing integrations

Next steps