EBS

Rationale

AWS EBS  (Elastic Block Store) is the service we use for block-level storage . It allows us to have hard drives  in the cloud . The main reasons why we chose it over other alternatives are the following:

• It seamlessly integrates with AWS EC2 , allowing us to connect external hard drives to instances.
• It complies with several  certifications from ISO  and CSA . Many of these certifications are focused on ensuring that the entity follows best practices regarding secure cloud-based  environments and information security.
• It provides a wide range of disk types  that go from SSDs  with a size of 64 TiB  and a throughput of 4,000 MiB/s  to HHDs  with a size of 16 TiB and a throughput of 500 MiB/s.
• It provides disks with different specializations . There are General Purpose and Provisioned IOPS SSDs and Throughput Optimized and Cold HHDs. By having all these different types of disks, we can easily select which one to work with, depending on the nature of the problem we are trying to solve.
• It supports point-in-time snapshots  designed to back up all data that exists within a disk.
• Disks can be easily attached  and detached  from AWS EC2 machines, allowing us to easily change general machine configurations without losing any data.
• Disks can be encrypted  using AWS KMS  keys, which allows encryption of data moving between the disk and the instance using it, data at rest inside the volume, disk snapshots, and all volumes created from these snapshots.
• It supports data lifecycle policies , allowing us to create, retain and delete disks based on created policies.
• It supports monitoring and metrics  using AWS CloudWatch .

Alternatives

Google Compute Engine (GCE)  is one alternative that did not exist at the time we migrated to the cloud. GCP  does not offer an equivalent to EBS. Instead, their entire disk service  exists within GCE. It does not support disk encryption.

Usage

We use AWS EBS  for

• GitLab CI  bastion (we use a 16 GiB GP2  disk, as it only needs to have basic software installed, such as GitLab Runner  and Docker Machine ; high disk throughput is not required);
• GitLab CI workers  (we use 10 GiB GP3  disks just for hosting our workers’ operating system . Additionally, workers come with high throughput 50 GiB internal NVMe disks , which are very useful for achieving as-fast-as-possible job performance within our CI);
• Batch  processing workers  (we use 8 GiB GP2 disks just for hosting the operating system; these workers also come with 50 GiB internal NVMe disks);
• Kubernetes  cluster workers  (we use 50 GiB GP2 disks for hosting the base operating system and stored containers for applications like our platform ; high disk throughput is not required as our ARM does not store any data within local disks);
• Okta RADIUS Agent  (we use a 50 GiB GP2 disk; it is probably oversized since only the base operating system and RADIUS agent  are required; high disk throughput is not required), and
• ERP  (we use two disks: a 50 GiB GP2 disk for hosting the base operating system and a 200 GiB GP2 disk for hosting the ERP data).