Multi-tenant cloud
The default. Shared gateway and router, dedicated project isolation, nine regions. Nothing to operate.
Product
Built to stay up while the models underneath it change.
Five components sit between your application and the models. Each one is independently scaled, independently deployable, and degrades to something useful rather than to an error.
Client
SDKspython · node · go · ruby · java · rust
RESTapi.vantafold.ai/v1
Streamingserver-sent events
Gateway
Authkeys · SAML · mTLS
Policyrate limits · spend caps
Cacheprompt prefix cache
Router
Scorercost · latency · quality
Healthlive provider probes
Failoverordered fallback chain
Model pool
Frontierhosted providers
Open weightsvantafold-operated
Privateyour VPC or single-tenant
Observability spans the whole path — every stage emits spans, token counts, and
cost to your traces and to your OTLP collector.
What happens when something breaks.
A provider outage is a routine event, not an incident. The router keeps live health probes against every model in your pool and removes a degraded endpoint from rotation before your error rate moves.
- Active-active across regions. No region is a single point of failure; traffic shifts on health, not on a manual runbook.
- Ordered fallback chains. Up to four attempts per request across distinct providers, with the full attempt list returned to you.
- Warm capacity pools. Autoscaling runs ahead of traffic, so the first request of a spike costs the same as the millionth.
- Graceful degradation. If observability writes fail, requests still serve; telemetry buffers and replays.
- 99.99%Uptime, measured on the public status page
- < 4Fallback attempts before an error is returned
- 38 msMedian routing, auth, and logging overhead
- 9Regions with active-active failover
Request lifecycle on failure
A single request that hits a degraded provider, told in the order it happens.
- Gateway validates the key and applies the project's spend cap.
- Router selects
frontier-lg— cheapest model clearing the quality floor. - Provider returns 529. The attempt is recorded and the model is marked degraded.
- Router retries against
reason-pro, the next entry in the chain. - Response returns with
route.attempts: 2. Your code sees one successful call.
Where requests are served and stored.
Pin inference and storage to a region to satisfy data-residency requirements. Private deployments add single-tenant regions on request.
| Region | Location | Inference | Trace storage | Status |
|---|---|---|---|---|
| us-east | N. Virginia | Active | Active | Operational |
| us-west | Oregon | Active | Active | Operational |
| eu-central | Frankfurt | Active | Active | Operational |
| eu-west | Dublin | Active | Active | Operational |
| uk-south | London | Active | Active | Operational |
| ap-northeast | Tokyo | Active | Active | Operational |
| ap-southeast | Singapore | Active | Active | Operational |
| ap-south | Mumbai | Active | Regional | Operational |
| sa-east | São Paulo | Active | Regional | Operational |
Three ways to run it.
Single-tenant
Your own gateway, router, and trace store in a region we operate. Same API, isolated infrastructure and upgrade schedule.
In your VPC
Run the gateway and router inside your own network. Model traffic egresses only to the providers you allow.
Read the architecture in more depth.
The engineering team publishes design notes as systems change.