GPU Performance Monitoring

ARMS uses OpenTelemetry to help you monitor NVIDIA and AMD GPUs for AI applications. Track GPU metrics like utilization, temperature, memory usage, and power consumption during AI training and inference workloads.

Choose your method

GPU monitoring can be implemented in two ways depending on your setup and requirements:

• ARMS SDK — It is useful if you already have an AI application running on GPU that's instrumented with ARMS. It extends your existing observability to include GPU metrics alongside your LLM traces.
• OpenTelemetry GPU Collector — It is useful for remote GPUs with only LLM models hosted, containerized deployments. This approach allows you to get GPU metrics without modifying application code.

Supported Parameters

SDK Configuration Options

Parameter	Type	Default	Description
collect_system_metrics	boolean	False	Enable GPU and system metrics collection
otlp_endpoint	string	None	OpenTelemetry OTLP endpoint URL
otlp_headers	string	None	Authentication headers for OTLP endpoint
service_name	string	"unknown_service"	Name of your AI application
environment	string	None	Deployment environment (dev, staging, prod)

Environment Variables

Variable	Description	Example
ELSAI_ARMS_COLLECT_SYSTEM_METRICS	Enable GPU monitoring	true
OTEL_EXPORTER_OTLP_ENDPOINT	OTLP endpoint URL	http://127.0.0.1:4318
OTEL_SERVICE_NAME	Service name for telemetry	my-gpu-app
OTEL_DEPLOYMENT_ENVIRONMENT	Deployment environment	production