Faster first boot on large instances

This applies to AMIs that carry a large model on disk, such as the Qwen3.8-27B server. The first start of a new instance is slow because of how AWS restores a disk from a snapshot, not because of the model server. Later restarts are much faster.

What we measured

Qwen3.8-27B server AMI, cold launch of a new g6e.xlarge in us-east-1, from the launch request until the server answered its health check:

• 10 min 4 s with a default launch.
• 6 min 34 s with a volume initialization rate of 300 MiB/s set at launch.

The first authenticated answer followed within a few seconds in both runs. These are single measurements, not averages. Once the data is on the disk, the model starts in a few minutes.

Why

An EBS volume created from a snapshot loads each block from S3 the first time it is read. A model server that reads tens of gigabytes at start-up therefore waits on that first read. Reading the same blocks again, including after a stop and start of the same instance, is fast. Adding gp3 throughput or IOPS to the volume does not remove this wait.

What you can do

1. Plan for it. Treat the first start of an instance as taking about ten minutes for the 27B AMI, or about seven with an initialization rate (below). Do not put a new instance behind a load balancer or an auto-scaling policy that expects it ready in a minute or two.
2. Stop and start instead of terminate and relaunch. A stopped instance keeps its volume, so the blocks already read stay local. Restarting the model server on a running instance took a few minutes in our tests. You pay for the volume while it is stopped.
3. Start earlier. Launch the instance before you need it, and wait for the health check at /health to return before sending traffic.
4. Set a volume initialization rate when you launch (optional). AWS loads the volume from its snapshot at the rate you choose, from 100 to 300 MiB/s, for a one-time charge per volume of $0.0024 per GB of snapshot data (100 to 200 MiB/s) or $0.0036 per GB (201 to 300 MiB/s) in us-east-1. For this AMI's 48 GB snapshot that is at most about $0.12 or $0.17 per launch, using AWS's published prices at the time of writing; check the EBS pricing page. It cut our cold start from 10 min 4 s to 6 min 34 s at 300 MiB/s; we have not measured other rates.

   aws ec2 run-instances --image-id <ami-id> --instance-type g6e.xlarge   --key-name <key> --subnet-id <subnet> --security-group-ids <sg>   --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":48,"VolumeType":"gp3","VolumeInitializationRate":300}}]'

   The setting cannot be stored in the AMI, so a plain launch from the Marketplace console does not use it. It works through the EC2 API, a launch template, or the CloudFormation AWS::EC2::LaunchTemplate and AWS::EC2::Volume resources. It is not available on AWS::EC2::Instance. Use the device name shown for the AMI in the EC2 console if it differs from /dev/xvda.

What does not work

Fast snapshot restore is enabled on the snapshot owner's account and does not carry over to the copy of the snapshot that your account receives from the Marketplace.

Questions or a measurement that differs from ours: contact us. See also launching an instance.