Response Caching

LiteLLM's exact-match cache key is built from the model, messages and parameters only, so two teams sending the same prompt would share an entry. The gateway therefore sets the cache namespace itself (callers cannot choose it) and decides the TTL.

Who shares entries

GATEWAY_CACHE_SCOPE chooses the tenant boundary: team (default), global, org, domain or key.

TTL

TTL cap       = min(policy cache_ttl over org/domain/team/key, model cache_ttl)   or the gateway default
effective TTL = min(cap, TTL asked for by the caller)

0 means caching is off for the request.

keys:
  demo-key:
    cache_ttl: 0          # never cache for this key

Caller controls

Callers may always bypass the cache with Cache-Control: no-cache or skip storing with no-store. They may only shorten the TTL, never extend it.