Bedrock

Rationale

Amazon Bedrock  is our main point of contact while interacting with LLMs.

The main reasons why we chose it over other alternatives are the following:

• It offers a broad spectrum of foundational models.
• It offers open-source and proprietary foundational models.
• It does not forward information to third parties.
• It is GDPR and SOC 2 compliant.
• Its infrastructure is hosted in the US.
• It is easily configurable through Terraform.
• It offers guardrails and other protection mechanisms.

Alternatives

The following alternatives were considered but not chosen for the stated reasons:

Anthropic

• Its models are accessible through Amazon Bedrock.
• It has less variety of foundational models.

Self-hosted open-source models

• This implies high maintenance costs.
• This option lacks access to some proprietary models.

Usage

We use Amazon Bedrock  to interact with foundational models.