Sops

Rationale

Sops  is the tool we use for managing most of our organizational secrets, like passwords, access keys, and PII , among others. It allows us to version encrypted files within our Git  repositories in a stateless approach.

The main reasons why we chose it over other alternatives are:

• It is open-source .
• It is Serverless , meaning that it does not require maintaining servers, firewalls, load balancers, or any other typical infrastructure required for common Secrets Engines .
• It supports AWS KMS , which allows us to encrypt files using symmetric AES256  keys that only exist within the KMS boundaries, granting almost-impossible key leakage. Access to such keys can be easily managed with a user-level granularity by using AWS IAM .
• It is free. Only costs for decrypting secret files using AWS KMS are incurred.
• As secrets are written as code , it allows software versioning , as encrypted secret files can be securely pushed to git repositories .
• It allows reproducibility and auditability as secrets are versioned.
• It is DevOps-friendly , as secret management is now done through Merge Requests , allowing us to run CI/CD  integrations on the secrets.
• Although secret KMS keys are very easy to rotate , Sops does not support secret-specific automatic rotations.
• It integrates with other services like PGP , Age , GCP KMS , Azure Key Vault , and Hashicorp Vault .
• It supports YAML, JSON, Env, INI and Binary  formats.

Alternatives

The following alternatives were considered but not chosen:

AWS Secrets Manager

• AWS Secrets Manager  is not open-source.
• It is SaaS, meaning that infrastructure complexity is handled by them.
• It supports its own key management system, AWS KMS.
• It charges based on secrets .
• It is a common Secrets Engine , meaning that secrets are not stored as code, losing git versioning. It supports its own versioning .
• It should support auditability and reproducibility as it has its own versioning.
• It is a service that must be managed separately, making integration with DevOps flows harder.
• It supports automatic rotation for some services  that are not relevant to us.
• It does not integrate with any other services.
• It does not support any text format, but instead is configurable via the GUI.

HashiCorp Vault

• Vault  is open-source.
• It is SaaS, meaning that infrastructure complexity is handled by them.
• It supports external key management systems  like AWS KMS and GCP KMS.
• It charges based on secrets .
• It is a common Secrets Engine , meaning that secrets are not stored as code, losing git versioning. It supports its own versioning .
• It should support auditability and reproducibility as it has its own versioning.
• It is a service that must be managed separately, making integration with DevOps flows harder.
• It supports automatic rotation for some services  that are not relevant to us.
• It integrates with Datadog .
• It does not support any text format, but instead is configurable via the GUI.

Infiscal

• Infiscal  is open-source.
• It is SaaS, meaning that infrastructure complexity is handled by them.
• It supports external key management systems  like AWS KMS and GCP KMS.
• It is pretty expensive as it charges based on identities , which are machines or humans that talk to it. It does not scale well with horizontal systems like ours.
• It is a common Secrets Engine , meaning that secrets are not stored as code, losing git versioning. It supports its own versioning .
• It should support auditability and reproducibility as it has its own versioning.
• It is a service that must be managed separately, making integration with DevOps flows harder.
• It supports automatic rotation for some services  that are not relevant to us.
• It integrates with Slack .
• It does not support any text format, but instead is configurable via the GUI.

Torus

• Torus  was used a few years ago, but it got discontinued. One year later, they relaunched their service. It is not open-source.
• It is SaaS, meaning that infrastructure complexity is handled by them.
• It does not support any external key management service.
• It does not publish its prices.
• It is a common Secrets Engine , meaning that secrets are not stored as code, losing versioning.
• It does not support auditability and reproducibility, as there is no versioning.
• It is a service that must be managed separately, making integration with DevOps flows harder.
• It does not seem to support automatic rotations.
• It does not integrate with any other services.
• It does not support any text format, but instead is configurable via the GUI.

CyberArk Secretless Broker

• CyberArk Secretless Broker  is yet another solution that involves secure brokers. It is open-source.
• It forces us to maintain the entire service on our Kubernetes  cluster and deploy sidecar agents to generate trust relationships.
• It does not support any external key management service.
• It is free. No costs appear to be incurred.
• It is a common Secrets Engine , meaning that secrets are not stored as code, losing versioning.
• It does not support auditability and reproducibility as there is no versioning.
• It is a service that must be managed separately, making integration with DevOps flows harder.
• It only supports automatic rotations for MySQL and PostgreSQL, meaning that manual rotation is still needed.
• It only integrates with other CyberArk services like CyberArk Conjur .
• It does not support any text format, but instead is configurable via CLI.

Usage

Used for managing most of our organizational secrets within the Universe repository .

We use GitLab CI/CD Variables  over Sops  for:

• Exporting Cachix  authorization tokens as Nix requires them to be available prior to Sops initialization.
• Exporting environment variables required by containers that do not support Sops.