Fireworks AI

Last updated: Sep 22, 2026

Rationale

Fireworks AI is our inference provider for open source (open-weight) models, enabling experimentation with and production use of models from multiple labs.

The main reasons why we chose it over other alternatives are the following:

It adds the most recent models from relevant labs far faster than Amazon Bedrock, which can take months to make a newly released open source model available, even though Bedrock offers open source models in theory.
It proved stable and consistent under sustained load in our own testing, unlike the alternatives evaluated (see Alternatives below).
It supports the request concurrency (requests per minute) we need for both experimentation and production workloads.
It offers Zero Data Retention (ZDR) by default.
It holds the following compliance certifications: SOC 2 Type II, HIPAA, GDPR, ISO 27001:2022, ISO 27701, and ISO/IEC 42001:2023.
Its infrastructure is hosted in the US.

Alternatives

The following alternatives were considered but not chosen for the stated reasons:

DeepInfra

It was inconsistent under load. At a low target of 10 requests per minute, a pilot run only completed 70% of requests successfully, with a mean latency of over 30 seconds and frequent timeouts, so testing was aborted.
At a higher target of 1,000 requests per minute, a completed load run performed better, and even faster than Fireworks AI, but the inconsistency observed at lower concurrency raised doubts about its overall reliability.

Baseten

Its basic account tier only supports a small number of requests per minute, well below the concurrency we require, so it could not even be load-tested at our target scale.

Amazon Bedrock

Open source models are available in theory, but it takes Bedrock months to add the most recent models from relevant labs, which does not meet the pace we need to adopt new open source releases.

We use Fireworks AI as an inference provider for open source models. It runs alongside Amazon Bedrock, Anthropic, and OpenAI, which remain our providers for proprietary models. Any of our AI-powered features may use Fireworks AI for open source model inference, depending on internal evaluation of that feature's needs, for both experimentation and production loads.