Appearance
Environment Variables
Core
| Variable | Required | Purpose |
|---|---|---|
LITELLM_MASTER_KEY | Yes | Master key of the LiteLLM proxy. |
GATEWAY_ADMIN_KEY | Yes | Protects /admin/v1. Without it the admin API answers 503. |
DATABASE_URL | For virtual keys | PostgreSQL connection string. |
GATEWAY_DATABASE_URL | No | Database for the gateway's own tables, when different from DATABASE_URL. |
REDIS_PASSWORD | No | Password for the shared Redis cache (may be empty). |
GATEWAY_POLICY_FILE | No | Path to policy.yaml (set for you by LLMGateway). |
GATEWAY_DEFAULT_MODEL | No | Model used by /v1/invoke when none is given. |
GATEWAY_AUTO_ALIAS | No | Name of the virtual routing alias (default auto). |
GATEWAY_TEMPLATES_DIR | No | Extra directory of guardrail templates; wins over built-ins with the same id. |
Cache
| Variable | Purpose |
|---|---|
GATEWAY_CACHE_SCOPE | Who may share cache entries: team (default), global, org, domain or key. |
GATEWAY_CACHE_DEFAULT_TTL | TTL used when no policy sets one. |
Alerts
| Variable | Purpose |
|---|---|
GATEWAY_ALERT_WEBHOOK_URL | Budget alerts are always logged; when set, they are also posted here as JSON with a Slack-compatible text field. |
Edge lockdown
| Variable | Purpose |
|---|---|
GATEWAY_LOCKDOWN | off disables the deny-by-default edge (development only). |
GATEWAY_ENABLE_DOCS | true exposes /docs, /redoc, /openapi.json. |
GATEWAY_ENABLE_METRICS | true exposes /metrics. |
GATEWAY_EXTRA_ALLOWED_PREFIXES | Comma-separated extra route prefixes to allow. |
AWS Secrets Manager
Secrets are loaded once at start-up, before LiteLLM or the gateway read the environment, so provider keys, the master and admin keys, DATABASE_URL and the alert webhook can all come from Secrets Manager with no other change. With neither of the first two variables set, AWS is never contacted.
| Variable | Purpose |
|---|---|
GATEWAY_AWS_SECRETS | Comma-separated secret ids or ARNs, each a JSON object {"VAR": "value"}. Every key becomes an environment variable. |
GATEWAY_AWS_SECRET_VARS | Comma-separated NAME=secret-id (whole secret is the value) or NAME=secret-id#json-key (one field). Applied after GATEWAY_AWS_SECRETS, so it wins. |
GATEWAY_AWS_SECRETS_REGION | Region of the secrets. Default: AWS_REGION, AWS_DEFAULT_REGION or AWS_REGION_NAME. |
GATEWAY_AWS_SECRETS_ENDPOINT | Custom endpoint (LocalStack or moto for testing). |
GATEWAY_AWS_SECRETS_OVERRIDE | true: a secret replaces a variable that is already set. Default false: the environment wins. |
GATEWAY_AWS_SECRETS_REQUIRED | Default true: refuse to start if a secret cannot be read. false: log it and carry on. |
AWS credentials come from the standard chain (IAM role, or AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY). Values are never logged; only variable names and where they came from.
bash
export GATEWAY_AWS_SECRETS="prod/llm-gateway/core"
export GATEWAY_AWS_SECRET_VARS="AZURE_OPENAI_API_KEY=prod/llm-gateway/azure#api_key"Container start-up
The elsai-llm-gateway command and python -m elsai_llm_gateway.bootstrap load secrets, check the essentials, then hand over to LiteLLM through the entrypoint in GATEWAY_ENTRYPOINT (default /app/docker/prod_entrypoint.sh). If a required value is missing it stops and names it.