Simple AWS #5: EBS (feat. DLM for snapshots)

What is EBS, volume types, tips to save money on EBS and automated backups with DLM

Guille Ojeda
December 19, 2022

Welcome to Simple AWS! A free newsletter that helps you build on AWS without being an expert. This is issue #5. Shall we?

AWS Service: EBS (feat. DLM for snapshots)

tl;dr: Elastic Block Store is a block-level storage service for EC2 instances. It's a virtual SSD or HDD that you attach to EC2 instances for persistent storage. Honestly, EBS is pretty boring to talk about, but if you're storing a ton of data, knowing the fine details can save you a lot of money. Let's start with the basics:

An EC2 instance has a root volume where it boots from (the OS is in the volume) and can have additional volumes.
EBS volumes can be resized.
EBS volumes are zonal resources. They exist in one AZ, so no high availability.
They're redundant within that AZ, so data loss is less likely than with a single disk (99.8%-99.9% durability in a year).
Their lifecycle is separate from that of the EC2 instance. You can create them, attach them, detach them and delete them on their own. You can also set up the EC2 instance to delete them when it's terminated (which is the default for the root volume, not for the others).
An EBS volume can be attached to a maximum of one instance at a time (except for io2). That means, they're not a shared file system (you can use EFS (Linux) or FSx (Windows) for that).
EBS volumes can be encrypted with KMS. It's transparent to you.

Here comes the boring part, the volume types. If EBS is less than 10% of your AWS bill, feel free to skip ahead to the tips.

General purpose: gp3. SSD that you use for everything. You can configure size and IOPS separately (unlike the previous gen, gp2).
- Limits: Size 1 GB to 16 TB, 16,000 IOPS, 1000 MB/s throughput.
- Price: $0.08/GB-month, $0.005/provisioned IOPS-month (first 3,000 are free).
For More performance: io2. SSD for things that require more performance (e.g. databases). You can configure size and IOPS separately. It can be attached to multiple instances at the same time.
- Limits: Size 4 GB to 16 TB, 64,000 IOPS, 1,000 MB/s throughput (256,000 IOPS and 4,000 MB/s for io2 Block Express).
- Price: $0.125/GB-month, $0.065/provisioned IOPS-month (no free IOPS)
Throughput-intensive: st1. HDD (yeah, spinning disks) that performs well for use cases that read contiguous data (e.g. logs), for half the price.
- Limits: Size 125 GB to 16 TB, 500 IOPS, 500 MB/s throughput.
- Price: $0.045/GB-month
Infrequent access: sc1. Slow but really cheap HDD, ideal for infrequently accessed data. An alternative is S3 Infrequent Access, which has more durability and is cheaper for storage (though you're charged for reads), but is slower to access.
- Limits: Size 125 GB – 16 TB, 250 IOPS, 250 MB/s throughput.
- Price: $0.015/GB-month

Actionable tips

If EBS is less than 10% of your AWS bill, you probably have bigger fish to fry.
Use gp3 volumes unless you know you need more performance or have a specific use case. Not sure? Here's how to benchmark. Also, some performance tips. And if you need extreme performance, use instance store.
EC2 instances have a cap on EBS performance. If you're near it, use EBS-optimized instances.
Migrate gp2 volumes to gp3, it's easy and you save 20%.
Encrypt your EBS volumes.
If you have different data with different requirements, set up multiple EBS volumes.
If you're storing important data, remember to back it up with snapshots. They're basically an incremental backup stored in S3, and can be encrypted.
- After restoring from a snapshot, the first time you read a block is slow because the data is lazy loaded from S3. You can eager load it by initializing the EBS volume, or you can enable fast snapshot restore (it's pricey).
- Snapshots are regional. If you want to use them for Disaster Recovery, you need to copy them to your DR region, either manually or automatically with DLM. If you're encrypting snapshots, use a multi-region KMS key.

Recommended tool

Today's tool is actually another AWS service: Data Lifecycle Manager. It can be used to automate snapshot creation and copying to another region by creating a snapshot policy.

Here's a CloudFormation template to set it up:

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
KmsKeyArn:
Type: String
Description: The ARN of the KMS key to use for encrypting cross-Region snapshot copies
DestinationRegion:
Type: String
Description: The destination region to copy the snapshots to
Resources:
SnapshotPolicy:
Type: AWS::DLM::LifecyclePolicy
Properties:
Description: EBS snapshot policy with cross-Region copy
PolicyDetails:
ResourceTypes:
- VOLUME
TargetTags:
-
Key: Snapshot
Value: true
Schedules:
- Name: DailySnapshot
CopyTags: true
CreateRule:
Interval: 1
IntervalUnit: DAYS
RetainRule:
Count: 7
Parameters:
ExcludeBootVolume: true
RestorablePeriod: 0
CrossRegionCopy:
DestinationRegion: !Ref DestinationRegion
Encrypted: true
KmsKeyArn: !Ref KmsKeyArn

Haiku(s)

EBS can store
Data on block devices
For your instances.

Encrypt your volumes.
And if you want to share them,
Also share the key.

Misc.

I know, EBS is boring. But knowing this stuff helped me reduce a client's AWS bill by $700/month, and another one's by $500/month. Still, I promise next issue is going to be more exciting.

On the previous issue I talked about ECS as an alternative to Kubernetes, wrote a haiku against multi-cloud and promised a blog post in favor of vendor lock-in. Here's the post. What are your thoughts on vendor lock-in?

Thank you for reading! See ya on the next issue.