Tutorial

Usage Example

Install the SaaS package:

pip install --extra-index-url https://arms-packages.elsaifoundry.ai/root/elsai-arms-sass/ elsai-arms==1.3.4

Set your API key:

export ELSAI_ARMS_API_KEY=your-api-key-here

Sample Code

from elsai_arms.elsai_arms import ElsaiARMS
from elsai_model.azure_openai import AzureOpenAIConnector
import easyocr

arms = ElsaiARMS("Project_Name")

try:
    connector = AzureOpenAIConnector()
    llm = connector.connect_azure_open_ai("gpt-4o-mini")

    arms.info("Script started")

    @arms.monitor_llm_call
    def get_response(prompt: str):
        return llm.invoke(prompt)

    @arms.monitor_ocr_call(model_name="EasyOCR")
    def run_easyocr(image_path):
        reader = easyocr.Reader(['en'])
        return reader.readtext(image_path, detail=1)

    @arms.monitor_embedding_call
    def get_embedding(text):
        embedding = ["Generated Embeddings"]
        return embedding

    @arms.monitor_rag_call
    def retrieve_documents(query, docs, top_k=2):
        results = ["RAG Results"]
        return results[:top_k]

    llm_result = get_response("Tell me a joke")
    ocr_result = run_easyocr("path_to_image.png")

    documents = ["Load Documents"]

    query = "Enter query"
    embedding = get_embedding(query)
    retrieved_docs = retrieve_documents(query, documents)

    arms.log_custom_metric("Number of response", 1)
    arms.info("Run completed")
    arms.warning("This is a sample code")

    arms.end_run()

    print(arms.export())

except Exception as e:
    arms.error(f"An error occurred: {e}")

Steps

Initialize elsai ARMS in the application

arms = ElsaiARMS('Project_Name')

The Project Manager checks if it's a new project or an existing project

Project Manager creates a new project, if new, otherwise loads existing project

A new run is initiated under the project

To automatically capture LLM-specific metrics

@arms.monitor_llm_call

To monitor async streaming LLM calls

@arms.monitor_llm_astream
async def run_astream(prompt, llm):
    return llm.astream_events([HumanMessage(content=prompt)])

To capture OCR metrics

@arms.monitor_ocr_call

To monitor RAG performance

@arms.monitor_rag_call

To monitor embedding metrics

@arms.monitor_embedding_call

To monitor LangChain agents and graphs

graph.invoke(
    {"messages": messages},
    config={"callbacks": [arms.langchain_callback]}
)

To log custom metrics

arms.log_custom_metric('Metric Name', metric_value)

To log operation

arms.info('Log Operation')

To log warning

arms.warning('Log Warning')

To log errors

arms.error('Log Error')

To export data as JSON

arms.export()

To end the run

arms.end_run()

Run data is sent to the elsai ARMS cloud backend. View results in the dashboard.