Faster first boot on large instances

This applies to AMIs that carry a large model on disk, such as the Qwen3.8-27B server. The first start of a new instance is slow because of how AWS restores a disk from a snapshot, not because of the model server. Later restarts are much faster.

What we measured

Qwen3.8-27B server AMI, cold launch of a new g6e.xlarge in us-east-1, from the launch request until the server answered its health check:

• 30 min 29 s with a default launch (about 12 to 22 MB/s from the disk).
• 8 min 11 s with a volume initialization rate of 300 MiB/s set at launch.

The first authenticated answer followed within a few seconds in both runs. These are single measurements, not averages. Once the data was on the disk, the model itself loaded and compiled in a few minutes.

Why

An EBS volume created from a snapshot loads each block from S3 the first time it is read. A model server that reads tens of gigabytes at start-up therefore waits on that first read. Reading the same blocks again, including after a stop and start of the same instance, is fast. Adding gp3 throughput or IOPS to the volume does not remove this wait.

What you can do

1. Plan for it. Treat the first start of an instance as taking about half an hour for the 27B AMI unless you set an initialization rate (below). Do not put a new instance behind a load balancer or an auto-scaling policy that expects it ready in a few minutes.
2. Stop and start instead of terminate and relaunch. A stopped instance keeps its volume, so the blocks already read stay local. Restarting the model server on a running instance took a few minutes in our tests. You pay for the volume while it is stopped.
3. Start earlier. Launch the instance before you need it, and wait for the health check at /health to return before sending traffic.
4. Set a volume initialization rate when you launch. This is the change that made the difference in our test: 8 min 11 s instead of 30 min 29 s. AWS loads the volume from its snapshot at the rate you choose, from 100 to 300 MiB/s, for a one-time charge per volume of $0.0024 per GB of snapshot data (100 to 200 MiB/s) or $0.0036 per GB (201 to 300 MiB/s) in us-east-1. For this AMI's 120 GB snapshot that is at most about $0.29 or $0.43 per launch, using AWS's published prices at the time of writing; check the EBS pricing page. We have only measured 300 MiB/s so far.

   aws ec2 run-instances --image-id <ami-id> --instance-type g6e.xlarge   --key-name <key> --subnet-id <subnet> --security-group-ids <sg>   --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":120,"VolumeType":"gp3","VolumeInitializationRate":300}}]'

   The setting cannot be stored in the AMI, so a plain launch from the Marketplace console does not use it. It works through the EC2 API, a launch template, or the CloudFormation AWS::EC2::LaunchTemplate and AWS::EC2::Volume resources. It is not available on AWS::EC2::Instance. Use the device name shown for the AMI in the EC2 console if it differs from /dev/xvda.

What does not work

Fast snapshot restore is enabled on the snapshot owner's account and does not carry over to the copy of the snapshot that your account receives from the Marketplace. Reading the model files in parallel did not make the disk faster in our tests.

Questions or a measurement that differs from ours: contact us. See also launching an instance.