API, routing, model access, and dashboard updates for LLM Gateway.
Routing learns cache-hit rates and output-to-input proportions from recent project/model usage to compare providers for large prompts and sessions. Available automatically across plans, with workload defaults before enough history exists and explicit overrides on Enterprise.
Read more about Adaptive Cache-Aware Provider Selection →
@llmgateway/ai-sdk-provider 4.0 targets Vercel AI SDK 7 and adds llmgateway.video() for experimental_generateVideo: the provider submits a gateway video job, the SDK polls it, and you get the MP4 bytes back, with image-to-video, first and last frames, reference inputs, and audio control. Stay on 3.x for AI SDK 6.
Read more about AI SDK 7 Provider with Video Generation →
Enterprise organizations can now enforce zero data retention across provider routing, LLM Gateway storage, response caches, and the Responses API. Conflicting retention and caching settings are blocked before they can weaken the policy.
Read more about Zero Data Retention Controls →
Anthropic's Claude Fable 5.1 is live on the gateway with a 1M context window, always-on adaptive thinking, stronger long-horizon agentic work, and cache reads at a quarter of Fable 5's price.
Read more about Claude Fable 5.1 →
Group developers under one shared policy — a project ceiling, per-developer budgets, and IAM rules — instead of configuring each person by hand. Microsoft Entra groups map onto teams over SCIM, and a default team catches everyone who joins without one. Available on the Enterprise plan.
Read more about Organization Teams and Directory Sync →
Claude now routes through Microsoft Foundry under a new azure-anthropic provider. The models directory gained lifecycle status filters, model pages sort providers by price, speed, or context, and API key lists flag the keys that are near or at their limit.
Read more about Claude on Foundry, Model Status & More →
Gateway API keys are now stored only as keyed HMAC-SHA-256 fingerprints and provider credentials only as AES-256-GCM ciphertext. Every plaintext read path is gone, so a key's secret is visible exactly once per issuance — at creation or roll.
Read more about Hash-Only API Key Storage →
Provider keys now take a description that follows them through key lists and request drilldowns, the master API returns each key's creator and project by name, and the catalogue picks up Grok 4.6 on AWS Bedrock and Vertex AI plus GLM-5.2 Turbo and an Australian region on SCX.ai.
Read more about Provider Key Descriptions, Grok 4.6 & More →
Every organization now has a fleet-wide budget of concurrent in-flight requests that scales with your trust tier — bounding the long-running streams a per-minute limit can't see. Over-budget requests get a retryable 429, and a momentarily saturated gateway sheds load with a 529 instead of queueing until it degrades.
Read more about Concurrent Request Limits and Overload Protection →
Provider cache writes is now a three-way setting. The new Client-managed mode forwards the cache markers your client sends and never adds any of its own, so one API key can serve a coding agent that manages its own caching alongside traffic that should not pay the cache-write premium.
Read more about Client-Managed Prompt Caching →
Showing 10 of 104 updates
Load more