AWS Snowball Edge
Developer Guide
AWS Snowball Edge Developer Guide
Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.
Table of Contents
What Is Snowball Edge? ... 1
AWS Snowball Edge Features ... 1
Prerequisites for Using Snowball Edge ... 2
Related Services ... 2
Accessing the Service ... 3
Accessing an AWS Snowball Edge Device ... 3
Pricing for the AWS Snowball Edge ... 3
Are You a First-Time AWS Snowball User? ... 3
Device Differences ... 4
Snowball Edge Device Options ... 4
Use Case Differences ... 5
Tool Differences ... 6
How Snowball Edge Works ... 8
How Import Jobs Work ... 9
How Export Jobs Work ... 9
How Local Compute and Storage Jobs Work ... 10
How a Clustered Local Compute and Storage Job Works ... 10
Snowball Edge Videos and Blogs ... 10
Device Specifications ... 11
Snowball Edge Storage Optimized (for Data Transfer) Specifications ... 11
Snowball Edge Storage Optimized (with EC2) Specifications ... 12
Snowball Edge Compute Optimized Specifications ... 13
Supported Network Hardware ... 15
Setting Up ... 18
Sign Up for AWS ... 18
Create an IAM User ... 18
Next Step ... 20
Before You Order a Device ... 21
About the Local Environment ... 21
Working with Special Characters ... 22
Using Amazon EC2 ... 22
Using Compute Instances on Clusters ... 23
Pricing for Compute Instances on Snowball Edge ... 23
Prerequisites ... 23
Creating a Linux AMI from an Instance ... 24
Creating a Linux AMI from a Snapshot ... 24
Using Amazon S3 ... 26
How Import Works ... 26
How Export Works ... 26
Amazon S3 Encryption with AWS KMS ... 27
Amazon S3 Encryption with Server-Side Encryption ... 29
Snowball Edge Clusters ... 30
Snowball Edge Cluster Quorums ... 30
Cluster Job Considerations ... 31
Getting Started ... 32
Creating a Snowball Edge Job ... 32
Step 1: Plan Your Job ... 33
Step 2: Choose Your Shipping Preferences ... 34
Step 3: Choose Your Job Details ... 34
Step 4: Choose Your Security Preferences ... 35
Step 5: Choose Your Notification Preferences ... 39
Step 6: Download AWS OpsHub ... 39
Step 7: Review and Create Your Job ... 40
Receiving the Snowball Edge ... 40
Connecting to Your Local Network ... 41
Getting Your Credentials and Tools ... 42
Downloading and Installing the Snowball Edge client ... 43
Unlocking the Snowball Edge ... 43
Setting Up Local Users ... 44
Using Your Snowball Edge ... 45
Powering Off the Snowball Edge ... 46
Returning the Device ... 46
Disconnecting the Snowball Edge ... 46
Monitoring the Import Status ... 47
Getting Your Job Completion Report and Logs ... 47
Where Do I Go from Here? ... 48
Using AWS OpsHub to Manage Devices ... 49
Unlocking a device ... 49
Verifying the signature of AWS OpsHub (optional) ... 50
Managing AWS services ... 52
Using Compute Instances Locally ... 53
Managing clusters ... 58
Managing S3 storage ... 59
Using NFS file share to upload files ... 60
Using AWS IoT Greengrass on EC2 instances ... 63
Setting up your Amazon EC2 instance ... 63
Managing Your Devices ... 64
Rebooting Your Device ... 65
Editing Your Device Alias ... 65
Getting Updates ... 65
Managing Profiles ... 66
Automating Your Management Tasks ... 66
Creating and Starting a Task ... 67
Viewing Details of a Task ... 69
Deleting a Task ... 69
Setting the NTP time servers for your device ... 69
Using a Snowball Edge Device ... 71
Using the Snowball Edge Client ... 72
Downloading and Installing the Snowball Edge Client ... 72
Commands for the Snowball Edge Client ... 72
Transferring Files Using the S3 Interface ... 89
Downloading and Installing the AWS CLI Version 1.16.14 ... 89
Using the AWS CLI and API Operations on Snowball Edge ... 90
Getting and Using Local Amazon S3 Credentials ... 91
Unsupported Amazon S3 Features for Snowball Edge ... 92
Batching Small Files ... 92
Supported CLI Commands ... 94
Supported REST API Actions ... 96
Transferring Files Using the File Interface ... 98
Overview of the File Interface ... 99
Starting the File Interface ... 100
Mounting a Bucket with the File Interface ... 101
Monitoring the File Interface ... 104
Using NFS for Offline Data Transfer ... 105
Troubleshooting NFS Issues ... 106
Using an AWS Snowball Edge device with a Tape Gateway ... 107
Ordering a Snowball Edge device with a Tape Gateway ... 108
Deploying a Snowball Edge device with a Tape Gateway ... 108
Troubleshooting and best practices for a Snowball Edge device with a Tape Gateway ... 109
Using the AWS Snow Family API with a Snowball Edge device with a Tape Gateway ... 110
Using AWS Lambda ... 112
Before You Start ... 113
Getting Started with Lambda ... 114
Using Amazon EC2 ... 118
Overview ... 119
Compute Instances on Clusters ... 23
Pricing for Compute Instances on Snowball Edge ... 23
Using AMIs on Your Device ... 120
Importing an AMI to Your Device ... 123
Using the AWS CLI and API Operations ... 133
Quotas for Compute Instances ... 133
Creating a Compute Job ... 135
Network Configuration for Compute Instances ... 136
Using SSH to Connect to a Compute Instance ... 140
Transferring Data from Compute Instances to Buckets on the Same Device ... 141
Snowball Edge Client Commands for Compute Instances ... 141
Using the Amazon EC2 Endpoint ... 145
Autostarting EC2 Instances ... 156
Using Block Storage with EC2 Instances ... 157
Security Groups ... 157
Supported Instance Metadata and User Data ... 158
Stopping EC2 Instances ... 159
Troubleshooting Compute Instances ... 159
Using IAM Locally ... 160
Using the AWS CLI and API Operations ... 161
Supported IAM AWS CLI Commands ... 161
IAM Policy Examples ... 164
TrustPolicy Example ... 167
Using AWS STS ... 167
Using the AWS CLI and API Operations on Snowball Edge ... 167
Supported AWS STSAWS CLI Commands on a Snowball Edge ... 168
Supported AWS STS API Operations ... 168
Ports Required to Use AWS Services ... 169
Using a Snowball Edge Cluster ... 170
Clustering Overview ... 170
Snowball Edge Cluster Quorums ... 170
Cluster Job Considerations ... 171
Related Topics ... 171
Administering a Cluster ... 172
Reading and Writing Data to a Cluster ... 172
Reconnecting an Unavailable Cluster Node ... 172
Removing an Unhealthy Node from a Cluster ... 173
Adding or Replacing a Node in a Cluster ... 173
Understanding AWS Snowball Edge Jobs ... 175
Job Details ... 175
Job Statuses ... 177
Cluster Statuses ... 178
Importing Jobs into Amazon S3 ... 179
Exporting Jobs from Amazon S3 ... 180
Using Export Ranges ... 180
Export Jobs Best Practices ... 182
Local Compute and Storage Only Jobs ... 182
Local Compute Jobs ... 183
Local Storage Jobs ... 183
Local Cluster Option ... 183
Cloning a Job in the Console ... 183
Canceling Jobs in the Console ... 184
Best Practices ... 185
Security ... 185
Resource Management ... 186
Performance ... 186
Performance Recommendations ... 187
Speeding Up Data Transfer ... 187
Transferring Petabytes of Data ... 187
Planning Your Large Transfer ... 188
Calibrating a Large Transfer ... 189
Updating a Snowball Edge ... 191
Prerequisites ... 191
Downloading Updates ... 191
Installing Updates ... 192
Shipping Considerations ... 194
Preparing an AWS Snowball Edge for Shipping ... 194
Region-Based Shipping Restrictions ... 195
Shipping an AWS Snowball Edge ... 195
Shipping Carriers ... 195
Security ... 200
Data Protection ... 200
Protecting Data in the Cloud ... 201
Protecting Data On Your Device ... 203
Identity and Access Management ... 205
Access Control for Console and Jobs ... 205
Logging and Monitoring ... 219
Compliance Validation ... 219
Resilience ... 220
Infrastructure Security ... 220
Data Validation ... 221
Checksum Validation of Transferred Data ... 221
Local Inventory Creation During Snowball Transfer ... 221
Common Validation Errors ... 221
Manual Data Validation for Snowball Edge After Import into Amazon S3 ... 222
Notifications ... 223
Logging with AWS CloudTrail ... 224
AWS Snowball Edge Information in CloudTrail ... 224
Understanding Log File Entries for AWS Snowball Edge ... 225
Quotas ... 226
Region Availability for AWS Snowball Edge ... 226
Limitations for AWS Snowball Edge Jobs ... 227
Limitations on Transferring On-Premises Data with a Snowball Edge Device ... 227
Limitations for Lambda Powered by AWS IoT Greengrass ... 227
Limitations on Shipping a Snowball Edge ... 228
Limitations on Processing Your Returned Snowball Edge for Import ... 228
Troubleshooting ... 229
Identify Your Device ... 230
Connection Problems ... 230
Manifest File Problems ... 230
Credentials Problems ... 230
Unable to Locate AWS CLI Credentials ... 230
Error Message: Check Your Secret Access Key and Signing ... 231
Data Transfer Problems ... 231
Troubleshooting Problems with Transferring Data Using the File Interface ... 231
AWS CLI Problems ... 232
AWS CLI Error Message: "Profile Cannot Be Null" ... 232
Null Pointer Error When Transferring Data with the AWS CLI ... 232
Import Job Problems ... 233
Export Job Problems ... 233
API Reference ... 234 Document History ... 235 AWS glossary ... 241
AWS Snowball Edge Features
What Is AWS Snowball Edge?
AWS Snowball Edge is a type of Snowball device with on-board storage and compute power for select AWS capabilities. Snowball Edge can do local processing and edge-computing workloads in addition to transferring data between your local environment and the AWS Cloud.
Each Snowball Edge device can transport data at speeds faster than the internet. This transport is done by shipping the data in the appliances through a regional carrier. The appliances are rugged, complete with E Ink shipping labels.
Snowball Edge devices have three options for device configurations—Storage Optimized, Compute Optimized, and Compute Optimized with GPU. When this guide refers to Snowball Edge devices, it's referring to all options of the device. When specific information applies only to one or more optional configurations of devices (such as how the Snowball Edge with GPU has an on-board GPU), it is called out specifically. For more information, see Snowball Edge Device Options (p. 4).
Topics
• AWS Snowball Edge Features (p. 1)
• Prerequisites for Using Snowball Edge (p. 2)
• Services Related to the AWS Snowball Edge (p. 2)
• Accessing the Service (p. 3)
• Pricing for the AWS Snowball Edge (p. 3)
• Are You a First-Time AWS Snowball User? (p. 3)
• AWS Snowball Edge Device Differences (p. 4)
AWS Snowball Edge Features
Snowball Edge devices have the following features:
• Large amounts of storage capacity or compute functionality for devices. This depends on the options you choose when you create your job.
• Network adapters with transfer speeds of up to 100 Gbit/second.
• Encryption is enforced, protecting your data at rest and in physical transit.
• You can import or export data between your local environments and Amazon S3, and physically transport the data with one or more devices without using the internet.
• Snowball Edge devices are their own rugged box. The built-in E Ink display changes to show your shipping label when the device is ready to ship.
• Snowball Edge devices come with an on-board LCD display that can be used to manage network connections and get service status information.
• You can cluster Snowball Edge devices for local storage and compute jobs to achieve data durability across 5–10 devices and locally grow or shrink storage on demand.
• You can use the file interface to read and write data to an AWS Snowball Edge device through a file share or Network File System (NFS) mount point.
Prerequisites for Using Snowball Edge
• You can write Python-language Lambda functions and associate them with Amazon S3 buckets when you create an AWS Snowball Edge device job. Each function triggers when a local Amazon S3 PUT object action is run on the associated bucket on the device.
• Snowball Edge devices have Amazon S3 and Amazon EC2 compatible endpoints available, enabling programmatic use cases.
• Snowball Edge devices support the new sbe1, sbe-c, and sbe-g instance types, which you can use to run compute instances on the device using Amazon Machine Images (AMIs).
Prerequisites for Using Snowball Edge
Before creating your first job, keep the following in mind.
For jobs that import data into Amazon S3, follow these steps:
• Create an AWS account with AWS Identity and Access Management (IAM) administrator-level permissions. For more information, see Setting Up Your AWS Access for AWS Snowball Edge (p. 18).
• Confirm that the files and folders to transfer are named according to the object key naming guidelines for Amazon S3. Any files or folders with names that don't meet these guidelines aren't imported into Amazon S3.
• Plan what data you want to import into Amazon S3. For more information, see Transferring Petabytes of Data Efficiently (p. 187).
Before exporting data from Amazon S3, follow these steps:
• Understand what data is exported when you create your job. For more information, see Using Export Ranges (p. 180).
• For any files with a colon (:) in the file name, change the file names in Amazon S3 before you create the export job to get these files. Files with a colon in the file name fail export to Microsoft Windows Server.
For jobs using compute instances:
• Before you can add any AMIs to your job, you must have an AMI in your AWS account and it must be a supported image type. Currently, supported AMIs are based on the Amazon Linux 2, CentOS 7 (x86_64) - with Updates HVM, or Ubuntu 16.04 LTS - Xenial (HVM) images. You can get these images from the AWS Marketplace.
• If you're using SSH to connect to the instances running on a Snowball Edge, you must already have the key pair for connecting to the instance.
• For information specific to using compute instances on a device, see Using Amazon EC2 Compute Instances (p. 118).
Services Related to the AWS Snowball Edge
You can use an AWS Snowball Edge device with the following related AWS services:
• Amazon S3 – Transfer data to an AWS Snowball Edge device using the Amazon S3 API for Snowball Edge, which supports a subset of the Amazon S3 API operations. You can do this in a single Snowball Edge device or in a cluster of devices for increased data durability.
You can also import data that is hosted on an AWS Snowball Edge device to Amazon S3 and your local environment through a shipped Snowball Edge device. For more information, see the Amazon Simple Storage Service User Guide.
Accessing the Service
• Amazon EC2 – Run compute instances on a Snowball Edge device using the Amazon EC2 compatible endpoint, which supports a subset of the Amazon EC2 API operations. For more information about using Amazon EC2 in AWS, see Getting started with Amazon EC2 Linux instances.
• AWS Lambda powered by AWS IoT Greengrass – Invoke Lambda functions based on Amazon S3 storage actions made on an AWS Snowball Edge device. These Lambda functions are associated with an AWS Snowball Edge device during job creation. For more information about using Lambda, see the AWS Lambda Developer Guide.
• Amazon Elastic Block Store (Amazon EBS) – Provide block-level storage volumes for use with EC2 instances. For more information, see Amazon Elastic Block Store (Amazon EBS).
• AWS Identity and Access Management (IAM) – Use this service to securely control access to AWS resources. For more information, see What is IAM?
• AWS Security Token Service (AWS STS) – Request temporary, limited-privilege credentials for IAM users or for users that you authenticate (federated users). For more information, see Temporary security credentials in IAM.
• Amazon EC2 Systems Manager – Use this service to view and control your infrastructure on AWS. For more information, see What is AWS Systems Manager?
Accessing the Service
You can either use the AWS Snow Family Management Console or the job management API to create and manage jobs. For information about the job management API, see Job Management API Reference for AWS Snowball.
Accessing an AWS Snowball Edge Device
After your Snowball Edge device or devices are onsite, you can access them in several different ways. You can use the LCD display (used only for network configuration) that's built into each device, the Amazon S3 and Amazon EC2 compatible endpoints, or the available file interface. For more information, see Using an AWS Snowball Edge Device (p. 71).
Pricing for the AWS Snowball Edge
For information about the pricing and fees associated with the service and its devices, see AWS Snowball Edge Pricing.
Are You a First-Time AWS Snowball User?
If you are a first-time user of the AWS Snow Family service, we recommend that you read the following sections in order:
1. For information about device types and options, see AWS Snowball Edge Device Differences (p. 4).
2. To learn more about the types of jobs, see Understanding AWS Snowball Edge Jobs (p. 175).
3. For an end-to-end overview of how to use an AWS Snowball Edge device, see How AWS Snowball Edge Works (p. 8).
4. When you're ready to get started, see Getting Started (p. 32).
5. For information about using compute instances on a device, see Using Amazon EC2 Compute Instances (p. 118).
Device Differences
AWS Snowball Edge Device Differences
This guide contains documentation for the Snowball Edge devices. You can use these devices to move huge amounts of data into and out of Amazon S3. You can order them using the job management API or the AWS Snow Family console. For frequently asked questions and pricing information, see AWS Snowball.
Topics
• Snowball Edge Device Options (p. 4)
• AWS Snow Family Use Case Differences (p. 5)
• AWS Snow Family Tool Differences (p. 6)
Snowball Edge Device Options
Snowball Edge devices have the following options for device configurations:
• Snowball Edge Storage Optimized (for data transfer) – This Snowball Edge device option has a 100 TB (80 TB usable) storage capacity.
• Snowball Edge Storage Optimized (with EC2 compute functionality) – This Snowball Edge device option has up to 80 TB of usable storage space, 24 vCPUs, and 32 GiB of memory for compute functionality. It also comes with 1 TB of additional SSD storage space for block volumes attached to Amazon EC2 AMIs.
• Snowball Edge Compute Optimized – This Snowball Edge device option has the most compute functionality, with 52 vCPUs, 208 GiB of memory, and 42 TB (39.5 usable) plus 7.68 TB of dedicated NVMe SSD for compute instances for block storage volumes for EC2 compute instances, and 42 TB of HDD capacity for either object storage or block storage volumes.
• Snowball Edge Compute Optimized with GPU – This Snowball Edge device option is identical to the Compute Optimized option, except for an installed GPU, equivalent to the one available in the P3 Amazon EC2 instance type. It has a storage capacity of 42 TB (39.5 TB of HDD storage that can be used for a combination of Amazon S3 compatible object storage and Amazon EBS compatible block storage volumes) plus 7.68 TB of dedicated NVMe SSD for compute instances.
For more information about the compute functionality of these three options, see Using Amazon EC2 Compute Instances (p. 118).
Note
When this guide refers to Snowball Edge devices, it's referring to all optional variants of the device. Whenever specific information applies only to one or more optional configurations (such as how the Snowball Edge Compute Optimized with GPU option has an on-board GPU peripheral), it is mentioned explicitly.The following table summarizes the differences between the various device options. For hardware specification information, see AWS Snowball Edge Specifications (p. 11).
Snowball Edge Storage
Optimized (for data transfer)
Snowball Edge Storage Optimized (with EC2 compute functionality)
Snowball Edge Compute Optimized
CPU AMD Naples, 32 cores,
3.4Ghz Intel Xeon D processor,
16 cores, 1.8Ghz AMD Naples, 32 cores, 3.4Ghz
vCPUs 24 52
Use Case Differences
Snowball Edge Storage
Optimized (for data transfer)
Snowball Edge Storage Optimized (with EC2 compute functionality)
Snowball Edge Compute Optimized
Usable memory 32 GB 208 GB
Security card Yes Yes Yes
GPU (optional) None None None (NVidia V100)
SSD 1 TB SATA 7.68 TB NVMe
Usable HDD 80 TB plus 1 TB of dedicated SATA SSD for compute instances.
80 TB 39.5 TB
Network interfaces • 2x 10 Gbit – RJ45
• 1x 25 Gbit – SFP28
• 1x 100 Gbit – QSFP28
• 1x 10 Gbit – RJ45
• 1x 25 Gbit – SFP28
• 1x 40 Gbit – QSFP+
• 2x 10 Gbit – RJ45
• 1x 25 Gbit – SFP28
• 1x 100 Gbit – QSFP28 Physical security
features • Hidden magnetic
screws
• Intrusion switches
• NFC tags
• Anti-tamper inserts
• Android app for tamper detection
• GPS and cellular
• Conformal coating
• Hidden magnetic screws
• Anti-tamper inserts
• Conformal coating
• Hidden magnetic screws
• Intrusion switches
• NFC tags
• Anti-tamper inserts
• Android app for tamper detection
• GPS and cellular
• Conformal coating
AWS Snow Family Use Case Differences
The following table shows the different use cases for the different AWS Snow Family devices.
Use case Snowball Edge AWS Snowcone
Import data into
Amazon S3 ✓ ✓
Export from Amazon S3 ✓
Durable local storage ✓
Local compute with
AWS Lambda ✓
Local compute
instances ✓ ✓
Durable Amazon S3 storage in a cluster of devices
✓
Use with AWS IoT
Greengrass (IoT) ✓
Tool Differences
Use case Snowball Edge AWS Snowcone
Transfer files through
NFS with a GUI ✓ ✓
GPU workloads ✓
Note
Workloads that need GPU support require the Snowball Edge Compute Optimized with GPU option.AWS Snow Family Tool Differences
The following outlines the different tools used with the Snow Family devices, and how they are used.
Snowball Edge Tools
AWS OpsHub for Snow Family
• The Snow Family devices now offer a user-friendly tool, AWS OpsHub for Snow Family, that you can use to manage your devices and local AWS services. You use AWS OpsHub on a client computer to perform tasks such as unlocking and configuring single or clustered devices, transferring files, and launching and managing instances running on Snow Family devices. For more information, see Using AWS OpsHub for Snow Family to Manage Snowball Devices.
Snowball Edge client with Snowball Edge
• Download the Snowball Edge client from the AWS Snowball Edge Resources page and install it on your own computer.
• Use the Snowball Edge client to unlock the Snowball Edge or the cluster of Snowball Edge devices. For more information, see Using the Snowball Edge Client (p. 72).
• The Snowball Edge client doesn't transfer data.
Amazon S3 interface with Snowball Edge
• Is already installed on the Snowball Edge by default. It does not need to be downloaded or installed.
• Can transfer data to or from the Snowball Edge. For more information, see Transferring Files Using the Amazon S3 Interface (p. 89).
• Encrypts data on the Snowball Edge while the data is transferred to the device.
File interface with Snowball Edge
• Is already installed on the Snowball Edge by default. It does not need to be downloaded or installed.
• Can transfer data by dragging and dropping files up to 150 GB in size from your computer to the buckets on the Snowball Edge through an easy-to-configure NFS mount point. For more information, see Transferring Files to AWS Snowball Edge Using the File Interface (p. 98).
• Encrypts data on the Snowball Edge while the data is transferred to the device.
AWS IoT Greengrass console with Snowball Edge
• With a Snowball Edge, you can use the AWS IoT Greengrass console to update your AWS IoT Greengrass group and the core running on the Snowball Edge.
Tool Differences
Items Provided for Snowball Edge
The following outlines the network adapters, cables used, and cables provided for the Snowball Edge device.
Network interface Snowball Edge support Cables provided with device
RJ45 ✓ Not provided.
SFP28 ✓ Not provided.
SFP28 (with optic
connector) ✓ No cables provided.
No optic connector provided for Snowball Edge devices.
QSFP ✓ No cables or optics
provided.
For more information about the network interfaces, cables, and connectors, see Supported Network Hardware (p. 15).
How AWS Snowball Edge Works
AWS Snowball Edge devices are owned by AWS, and they reside at your on-premises location while they're in use.
There are three job types you can use with an AWS Snowball Edge device. Although the job types differ in their use cases, every job type has the same workflow for how you order, receive, and return devices.
Regardless of the job type, every job follows a data erasure of the National Institute of Standards and Technology (NIST) 800-88 standard after the job completes.
The shared workflow
1.Create the job – Each job is created in the AWS Snow Family Management Console or
programmatically through the job management API. The status for a job can be tracked in the console or through the API.
2.A device is prepared for your job – We prepare an AWS Snowball Edge device for your job, and the status of your job is now Preparing Snowball.
3.A device is shipped to you by your region's carrier – The carrier takes over from here, and the status of your job is now In transit to you. You can find your tracking number and a link to the tracking website on the console or with the job management API. For information about who your region's carrier is, see Shipping Considerations for AWS Snowball (p. 194).
4.Receive the device – A few days later, your region's carrier delivers the AWS Snowball Edge device to the address that you provided when you created the job, and the status of your job changes to Delivered to you. When it arrives, you’ll notice that it didn’t arrive in a box, because the device is its own shipping container.
5.Get your credentials and download the Snowball Edge client – Get ready to start transferring data by getting your credentials, your job manifest, and the manifest's unlock code, and then downloading the Snowball Edge client.
• The 8 client is the tool that you use to manage the flow of data from the device to your on-premises data destination.
You can download and install the Snowball Edge client from the AWS Snowball resources page.
You must download the Snowball Edge client from the AWS Snowball Edge Resources page and install on a powerful workstation that you own.
• The manifest is used to authenticate your access to the device, and it is encrypted so that only the unlock code can decrypt it. You can get the manifest from the console or with the job management API when the device is on-premises at your location.
• The unlock code is a 29-character code used to decrypt the manifest. You can get the unlock code from the console or with the job management API. We recommend that you keep the unlock code saved somewhere separate from the manifest to prevent unauthorized access to the device while it’s at your facility.
6.Position the hardware – Move the device into your data center and open it following the instructions on the case. Connect the device to power and your local network.
7.Power on the device – Next, power on the device by pressing the power button above the LCD display.
Wait a few minutes, and the Ready screen appears.
8.Get the IP address for the device – The LCD display has a CONNECTION tab on it. Tap this tab and get the IP address for the AWS Snowball Edge device.
9.Use the Snowball Edge client to unlock the device – When you use the Snowball Edge client to unlock the AWS Snowball Edge device, enter the IP address of the device, the path to your manifest,
How Import Jobs Work
and the unlock code. The Snowball Edge client decrypts the manifest and uses it to authenticate your access to the device.
10.Use the device – The device is up and running. You can use it to transfer data or for local compute and storage. You can read and write data with the Amazon S3 interface or the Network File System (NFS) mount point.
11.Prepare the device for its return trip – After you're done with the device in your on-premises location and the file interface status is Complete, press the power button above the LCD display. It takes about 20 seconds or so for the device to power off. Unplug the device and its power cables into the cable nook on top of the device, and shut all three of the device's doors. The device is now ready to be returned.
12.Your region's carrier returns the device to AWS – When the carrier has the AWS Snowball Edge device, the status for the job becomes In transit to AWS.
Note
There are additional steps for export and cluster jobs. For more information, see How Export Jobs Work (p. 9) and How a Clustered Local Compute and Storage Job Works (p. 10).Topics
• How Import Jobs Work (p. 9)
• How Export Jobs Work (p. 9)
• How Local Compute and Storage Jobs Work (p. 10)
• Snowball Edge Videos and Blogs (p. 10)
How Import Jobs Work
Each import job uses a single Snowball appliance. After you create a job in the AWS Snow Family Management Console or the job management API, we ship a Snowball to you. When it arrives in a few days, you connect the Snowball Edge device to your network and transfer the data that you want imported into Amazon S3 onto the device. When you’re done transferring data, ship the Snowball back to AWS, and we import your data into Amazon S3.
How Export Jobs Work
Each export job can use any number of AWS Snowball Edge devices. If the listing contains more data than can fit on a single device, multiple devices are provided to you. Each job part has exactly one device associated with it. After your job parts are created, your first job part enters the Preparing Snowball status.
Note
The listing operation used to split your job into parts is a function of Amazon S3, and you are billed for it the same way as any Amazon S3 operation.Soon after that, we start exporting your data onto a device. Typically, exporting data takes one business day. However, this process can take longer depending on the amount and type of data. When the export is done, AWS gets the device ready for pickup by your region's carrier. When it arrives, you connect the AWS AWS Snowball Edge device to your network and transfer the data that you want to import from Amazon S3 onto the device.
When you’re done transferring data, ship the device back to AWS. When we receive the device for your export job part, we erase it completely. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards. This step marks the completion of that particular job part.
• For keylisting
How Local Compute and Storage Jobs Work
Before we export the objects in the S3 bucket, we scan the bucket. If the bucket is altered after the scan, the job could encounter delays because we scan for missing or altered objects.
• For S3 Glacier Flexible Retrieval
It is important to note that AWS Snowball cannot export objects in the S3 Glacier Flexible Retrieval storage class. These objects must be restored before AWS Snowball can successfully export the objects in the bucket.
How Local Compute and Storage Jobs Work
You can use the local compute and storage functionality of an AWS Snowball Edge device with all job types in AWS Regions that support Lambda. The compute functionality is named AWS Lambda powered by AWS IoT Greengrass, where Python-language AWS Lambda functions can be triggered by Amazon S3 PUT object actions on buckets specified when you created the job. For more information, see Local Compute and Storage Only Jobs (p. 182).
How a Clustered Local Compute and Storage Job Works
A cluster job is a special kind of job for local storage and compute only. It is for those workloads that require increased data durability and storage capacity. For more information, see Local Cluster Option (p. 183).
Note
Like standalone local storage and compute jobs, the data stored in a cluster can't be imported into Amazon S3 without ordering additional devices as a part of separate import jobs. If you order these devices, you can transfer the data from the cluster to the devices and import the data when you return the devices for the import jobs.Clusters have 5–10 AWS Snowball Edge devices, called nodes. When you receive the nodes from your regional carrier, connect all the nodes to power and your network to obtain their IP addresses. You use these IP addresses to unlock all the nodes of the cluster at once with a single unlock command, using the IP address of one of the nodes. For more information, see Using the Snowball Edge Client (p. 72).
You can write data to an unlocked cluster by using the Amazon S3 interface or the NFS mount point through the leader node and the data distributed among the other nodes.
When you’re done with your cluster, ship all the nodes back to AWS. When we receive the cluster node, we perform a complete erasure of the Snowball. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards.
Snowball Edge Videos and Blogs
• AWS Snowball Edge Data Migration
• AWS OpsHub for Snow Family
• Novetta delivers IoT and Machine Learning to the edge for disaster response
• Enable large-scale database migrations with DMS and AWS Snowball
• Data Migration Best Practices with AWS Snowball Edge
• AWS Snowball resources
Snowball Edge Storage Optimized (for Data Transfer) Specifications
AWS Snowball Edge Specifications
In this section, you can find hardware specifications for Snowball Edge devices.
Topics
• Snowball Edge Storage Optimized (for Data Transfer) Specifications (p. 11)
• Snowball Edge Storage Optimized (with EC2) Specifications (p. 12)
• Snowball Edge Compute Optimized Specifications (p. 13)
• Supported Network Hardware (p. 15)
Snowball Edge Storage Optimized (for Data Transfer) Specifications
The following table contains hardware specifications for Snowball Edge Storage Optimized devices.
Item Snowball Edge Storage Optimized (for Data
Transfer) specifications
Storage specifications
HDD storage capacity 80 TB of usable
Power supply specifications
Power In AWS Regions in the US: NEMA 5–15p 100–
220 volts. In all AWS Regions, a power cable is included
Power consumption 304 watts for an average use case, though the
power supply is rated for 1200 watts.
Voltage 100 – 240V AC
Frequency 47/63 Hz
Data and network connections 2x 10 Gbit – RJ45 1x 25 Gbit – SFP28 1x 100 Gbit – QSFP28
Cables Each AWS Snowball Edge device ships country-
specific power cables. No other cables or optics are provided. For more information, see Supported Network Hardware (p. 15).
Thermal requirements AWS Snowball Edge devices are designed for office operations, and are ideal for data center operations.
Decibel output On average, an AWS Snowball Edge device
produces 68 decibels of sound, typically quieter than a vacuum cleaner or living-room music.
Snowball Edge Storage Optimized (with EC2) Specifications
Item Snowball Edge Storage Optimized (for Data
Transfer) specifications Dimensions and weight specifications
Weight 49.7 pounds (22.54 Kg)
Height 15.5 inches (394 mm)
Width 10.6 inches (265 mm)
Length 28.3 inches (718 mm)
Environment specifications
Vibration Non-operational use equivalent to ASTM D4169
Truck level I 0.73 GRMS
Shock Operational use equivalent to 70G (MIL-S-901)
Non-operational use equivalent to 50G (ISTA-3A)
Altitude Operational use equivalent to 0–3,000 meters (0–
10,000 feet)
Non-operational use equivalent to 0–12,000 meters
Temperature range 0–45°C (operational)
Snowball Edge Storage Optimized (with EC2) Specifications
The following table contains hardware specifications for Snowball Edge Storage Optimized (with EC2) devices.
Item Snowball Edge Storage Optimized (with EC2) specifications Compute and memory
specifications
CPU 24 vCPUs
RAM 32 GB RAM
Storage specifications
HDD storage capacity 80 TB usable (for object and block storage) SSD storage capacity 1 TB usable SATA SSD storage (for block storage) Power supply
specifications
Power In AWS Regions in the US: NEMA 5–15p 100–220 volts. In all AWS Regions, a power cable is included
Snowball Edge Compute Optimized Specifications
Item Snowball Edge Storage Optimized (with EC2) specifications
Power consumption 304 watts for an average use case, though the power supply is rated for 1200 watts
Voltage 100 – 240V AC
Frequency 47/63 Hz
Data and network
connections 1x 10 Gbit – RJ45 1x 25 Gbit – SFP28 1x 40 Gbit – QSFP28
Cables Each AWS Snowball Edge device ships country-specific power cables. No other cables or optics are provided. For more information, see Supported Network Hardware (p. 15).
Thermal requirements AWS Snowball Edge devices are designed for office operations, and are ideal for data center operations.
Decibel output On average, an AWS Snowball Edge device produces 68 decibels of sound, typically quieter than a vacuum cleaner or living-room music.
Dimensions and weight
specifications
Weight 49.7 pounds (22.45 Kg)
Height 15.5 inches (394 mm)
Width 10.6 inches (265 mm)
Length 28.3 inches (718 mm)
Environment
specifications
Vibration Non-operational use equivalent to ASTM D4169 Truck level I 0.73 GRMS Shock Operational use equivalent to 70G (MIL-S-901)
Non-operational use equivalent to 50G (ISTA-3A)
Altitude Operational use equivalent to 0–3,000 meters (0–10,000 feet) Non-operational use equivalent to 0–12,000 meters
Temperature range 0–45°C (operational)
Snowball Edge Compute Optimized Specifications
The following table contains hardware specifications for Snowball Edge Compute Optimized and Compute Optimized with GPU devices.
Item Snowball Edge Compute Optimized specifications
Compute and memory specifications
Snowball Edge Compute Optimized Specifications
Item Snowball Edge Compute Optimized specifications
CPU 52 vCPUs
RAM 256 GB RAM (208 GB RAM - Customer usable)
GPU nVidia V100 (available in Compute Optimized with GPU
configuration)
Storage specifications
HDD storage capacity 42 TB usable (for object and block storage) SSD storage capacity 7.68 TB usable NVMe SSD (for block storage) Power supply specifications
Power In AWS Regions in the US: NEMA 5–15p 100–220 volts. In all
AWS Regions, a power cable is included
Power consumption 304 watts for an average use case, though the power supply is rated for 1200 watts
Voltage 100 – 240V AC
Frequency 47/63 Hz
Data and network connections 2x 10 Gbit – RJ45 1x 25 Gbit – SFP28 1x 100 Gbit – QSFP28
Cables Each AWS Snowball Edge device ships country-
specific power cables. No other cables or optics are provided. For more information, see Supported Network Hardware (p. 15).
Thermal requirements AWS Snowball Edge devices are designed for office operations, and are ideal for data center operations.
Decibel output On average, an AWS Snowball Edge device produces 68 decibels of sound, typically quieter than a vacuum cleaner or living-room music.
Dimensions and weight specifications
Weight 49.7 pounds (22.45 Kg)
Height 15.5 inches (394 mm)
Width 10.6 inches (265 mm)
Length 28.3 inches (718 mm)
Environment specifications
Vibration Non-operational use equivalent to ASTM D4169 Truck level I 0.73 GRMS
Supported Network Hardware
Item Snowball Edge Compute Optimized specifications
Shock Operational use equivalent to 70G (MIL-S-901)
Non-operational use equivalent to 50G (ISTA-3A)
Altitude Operational use equivalent to 0–3,000 meters (0–10,000
feet)
Non-operational use equivalent to 0–12,000 meters
Temperature range 0–45°C (operational)
Supported Network Hardware
To use the AWS Snowball Edge device, you need your own network cables. For RJ45 cables, there are no specific recommendations. SFP28 and QSFP28 cables and modules from Mellanox and Finisar have been verified to be compatible with the device.
After you open the back panel of the AWS Snowball Edge device, you see the network ports shown in the following photograph.
Only one network interface on the AWS Snowball Edge device can be used at a time. Hence use any one of the ports to support the following network hardware.
SFP
This port provides a 10G/25G SFP28 interface compatible with SFP28 and SFP+ transceiver modules and direct-attach copper (DAC) cables. You must provide your own transceivers or DAC cables.
• For 10G operation, you can use any SFP+ option. Examples include:
• 10Gbase-LR (single mode fiber) transceiver
• 10Gbase-SR (multi-mode fiber) transceiver
• SFP+ DAC cable
• For 25G operation, you can use any SFP28 option. Examples include:
• 25Gbase-LR (single mode fiber) transceiver
• 25Gbase-SR (multi-mode fiber) transceiver
• SFP28 DAC cable
Supported Network Hardware
QSFP
This port provides a 40G QSFP28 interface on storage-optimized devices and a 40/50/100G QSFP28 interface on compute-optimized devices. Both are compatible with QSFP+ transceiver modules and DAC cables. You must provide your own transceivers or DAC cables. Examples include the following:
• 40Gbase-LR4 (single mode fiber) transceiver
• 40Gbase-SR4 (multi-mode fiber) transceiver
• QSFP28 DAC
RJ45
This port provides 1Gbase-TX/10Gbase-TX operation. It is connected via UTP cable terminated with an RJ45 connector. Compute-optimized devices have two RJ45 ports.
1G operation is indicated by a blinking amber light. 1G operation is not recommended for large-scale data transfers to the Snowball Edge device, as it dramatically increases the time it takes to transfer data.
10G operation is indicated by a blinking green light. It requires a Cat6A UTP cable with a maximum operating distance of 180 feet (55 meters).
Supported Network Hardware
Sign Up for AWS
Setting Up Your AWS Access for AWS Snowball Edge
Before you use AWS Snowball Edge for the first time, you need to complete the following tasks:
1.Sign Up for AWS (p. 18).
Note
In the Asia Pacific (Mumbai) AWS Region service is provided by Amaz on Internet Services Private Limited (AISPL). For information on signing up for Amazon Web Services in the Asia Pacific (Mumbai) AWS Region, see Signing Up for AISPL.2.Create an IAM User (p. 18).
Sign Up for AWS
When you sign up for Amazon Web Services (AWS), your AWS account is automatically signed up for all services in AWS, including AWS Snow Family. You are charged only for the services that you use. For more information about pricing and fees, see AWS Snowball Edge Pricing. AWS Snowball Edge is not free to use. For more information on what AWS services are free, see AWS Free Usage Tier.
If you have an AWS account already, skip to the next task. If you don't have an AWS account, use the following procedure to create one.
To create an AWS account
1. Open https://portal.aws.amazon.com/billing/signup.
2. Follow the online instructions.
Part of the sign-up procedure involves receiving a phone call and entering a verification code on the phone keypad.
Note your AWS account number, because you'll need it for the next task.
Create an IAM User
Services in AWS, such as AWS Snowball Edge, require that you provide credentials when you access them, so that the service can determine whether you have permission to access its resources. AWS recommends not using the root credentials of your AWS account to make requests. Instead, create an AWS Identity and Access Management (IAM) user, and grant that user full access. We refer to these users as IAM users with administrator-level credentials.
You can use the administrator user credentials, instead of root credentials of your account, to interact with AWS and perform tasks, such as to create an Amazon S3 bucket, create users, and grant them permissions. For more information, see Root Account Credentials vs. IAM User Credentials in the AWS General Reference and IAM Best Practices in IAM User Guide.
If you signed up for AWS but have not created an IAM user for yourself, you can create one using the IAM console.
Create an IAM User
To create an administrator user for yourself and add the user to an administrators group (console)
1. Sign in to the IAM console as the account owner by choosing Root user and entering your AWS account email address. On the next page, enter your password.
Note
We strongly recommend that you adhere to the best practice of using the Administrator IAM user that follows and securely lock away the root user credentials. Sign in as the root user only to perform a few account and service management tasks.2. In the navigation pane, choose Users and then choose Add user.
3. For User name, enter Administrator.
4. Select the check box next to AWS Management Console access. Then select Custom password, and then enter your new password in the text box.
5. (Optional) By default, AWS requires the new user to create a new password when first signing in. You can clear the check box next to User must create a new password at next sign-in to allow the new user to reset their password after they sign in.
6. Choose Next: Permissions.
7. Under Set permissions, choose Add user to group.
8. Choose Create group.
9. In the Create group dialog box, for Group name enter Administrators.
10. Choose Filter policies, and then select AWS managed - job function to filter the table contents.
11. In the policy list, select the check box for AdministratorAccess. Then choose Create group.
Note
You must activate IAM user and role access to Billing before you can use the
AdministratorAccess permissions to access the AWS Billing and Cost Management console. To do this, follow the instructions in step 1 of the tutorial about delegating access to the billing console.
12. Back in the list of groups, select the check box for your new group. Choose Refresh if necessary to see the group in the list.
13. Choose Next: Tags.
14. (Optional) Add metadata to the user by attaching tags as key-value pairs. For more information about using tags in IAM, see Tagging IAM entities in the IAM User Guide.
15. Choose Next: Review to see the list of group memberships to be added to the new user. When you are ready to proceed, choose Create user.
You can use this same process to create more groups and users and to give your users access to your AWS account resources. To learn about using policies that restrict user permissions to specific AWS resources, see Access management and Example policies.
To sign in as this new IAM user, sign out of the AWS Management Console, then use the following URL, where your_aws_account_id is your AWS account number without the hyphens (for example, if your AWS account number is 1234-5678-9012, your AWS account ID is 123456789012).
https://your_aws_account_id.signin.aws.amazon.com/console/
Type the IAM user name and password that you just created. When you're signed in, the navigation bar displays "your_user_name @ your_aws_account_id".
If you don't want the URL for your sign-in page to contain your AWS account ID, you can create an account alias. From the IAM dashboard, choose Create Account Alias and type an alias, such as your company name. To sign in after you create an account alias, use the following URL.
Next Step
https://your_account_alias.signin.aws.amazon.com/console/
To verify the sign-in link for IAM users for your account, open the IAM console and check under AWS account Alias on the dashboard.
If you're going to create AWS Snowball Edge jobs through an IAM user that is not an administrator user, that user needs certain permissions to use the AWS Snow Family Management Console effectively.
For more information on those permissions, see Permissions Required to Use the AWS Snowball Console (p. 211).
Next Step
Getting Started (p. 32)
About the Local Environment
Before You Order a Snowball Edge device
AWS Snowball Edge is a region-specific service. So before you plan your job, be sure that the service is available in your region. Ensure that your location and Amazon S3 bucket are within the same AWS Region or the same country because it will impact your ability to order the device.
As part of the order process, you create an AWS Identity and Access Management (IAM) role and an AWS Key Management Service (AWS KMS) key. The KMS key is used for encrypting the data during transit and at rest on the Snowball Edge device. For more information about creating IAM roles and KMS keys, see Creating an AWS Snowball Edge Job.
Topics
• Questions about the Local Environment (p. 21)
• Working with Files That Contain Special Characters (p. 22)
• Using Amazon EC2 on Snowball (p. 22)
• Using Amazon S3 on Snowball (p. 26)
• Snowball Edge Clusters (p. 30)
Questions about the Local Environment
Understanding your dataset and how the local environment is set up will help you complete your data transfer. Consider the following before placing your order.
What data are you transferring?
Transferring a large number of small files does not work well with AWS Snowball Edge. This is because Snowball Edge encrypts each individual object. Small files include files under 1 MB in size.
We recommend that you zip them up before transferring them onto the AWS Snowball Edge device.
We also recommend that you have no more than 500,000 files or directories within each directory.
Will the data be accessed during the transfer?
It is important to have a static dataset, (that is, no users or systems are accessing the data during transfer). If not, the file transfer can fail due to a checksum mismatch. The files won't be transferred and the files will be marked as Failed.
We recommend that if you are using the file interface, you only use one method of transferring data to the AWS Snowball Edge. Copying data with both the file interface and the Amazon S3 interface can result in read/write conflicts.
To prevent corrupting your data, don't disconnect an AWS Snowball Edge device or change its network settings while transferring data. Files should be in a static state while being written to the device. Files that are modified while they are being written to the device can result in read/write conflicts.
Will the network support AWS Snowball data transfer?
Snowball Edge supports the RJ45, SFP+, or QSFP+ networking adapters. Verify that your switch is a gigabit switch. Depending on the brand of switch, it might say gigabit or 10/100/1000. Snowball Edge devices do not support a megabit switch, or 10/100 switch.
Working with Special Characters
Working with Files That Contain Special Characters
It's important to note that if your objects contain special characters, you might encounter errors.
Although Amazon S3 allows special characters, we highly recommend that you avoid the following characters:
• Backslash ("\")
• Left curly brace ("{")
• Right curly brace ("}")
• Left square bracket ("[")
• Right square bracket ("]")
• 'Less Than' symbol ("<")
• 'Greater Than' symbol (">")
• Non-printable ASCII characters (128–255 decimal characters)
• Caret ("^")
• Percent character ("%")
• Grave accent / back tick ("`")
• Quotation marks
• Tilde ("~")
• 'Pound' character ("#")
• Vertical bar / pipe ("|")
If your files have one or more of these characters, rename them before you copy them to the AWS Snowball Edge device. Windows users who have spaces in their file names should be careful when copying individual objects or running a recursive command. Surround individual objects that have spacing in the name with quotation marks. The following are examples of such files.
Operating system File name: test file.txt
Windows “C:\Users\<username>\desktop\test file.txt”
iOS /Users/<username>/test\ file.txt
Linux /home/<username>/test\ file.txt
Note
The only object metadata that is transferred is the object name and size. If you want additional metadata to be copied, you can use the file interface or other tools to copy the data to Amazon S3.Using Amazon EC2 on Snowball
This section provides an overview of using Amazon EC2 compute instances on an AWS Snowball Edge device. It includes conceptual information, procedures, and examples.
Note
These Amazon EC2 features on AWS Snowball are not supported in the Asia Pacific (Mumbai) and Europe (Paris) AWS Regions.You can run Amazon EC2 compute instances hosted on an AWS Snowball Edge with the sbe1, sbe-c, and sbe-g instance types:
Using Compute Instances on Clusters
• The sbe1 instance type works on devices with the Snowball Edge Storage Optimized option.
• The sbe-c instance type works on devices with the Snowball Edge Compute Optimized option.
• Both the sbe-c and sbe-g instance types work on devices with the Snowball Edge Compute Optimized with GPU option.
All the compute instance types supported on Snowball Edge device options are unique to AWS Snowball Edge devices. Like their cloud-based counterparts, these instances require Amazon Machine Images (AMIs) to launch. You choose the AMI for an instance before you create your Snowball Edge job.
To use a compute instance on a Snowball Edge, create a job and specify your AMIs. You can do this using the AWS Snowball Management Console, the AWS Command Line Interface (AWS CLI), or one of the AWS SDKs. Typically, to use your instances, there are some housekeeping prerequisites that you must perform before creating your job.
After your device arrives, you can start managing your AMIs and instances. You can manage your compute instances on a Snowball Edge through an Amazon EC2-compatible endpoint. This type of endpoint supports many of the Amazon EC2 CLI commands and actions for the AWS SDKs. You can't use the AWS Management Console on the Snowball Edge to manage your AMIs and compute instances.
When you're done with your device, return it to AWS. If the device was used in an import job, the data transferred using the Amazon S3 interface or the file interface is imported into Amazon S3. Otherwise, we perform a complete erasure of the device when it is returned to AWS. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards.
Important
Data in compute instances running on a Snowball Edge isn't imported into AWS.
Using Compute Instances on Clusters
You can use compute instances on clusters of Snowball Edge devices. The procedures and guidance for doing so are the same as for using compute instances on a standalone device.
When you create a cluster job with AMIs, a copy of each AMI exists on each node in the cluster. You can have only 10 AMIs associated with a cluster of devices regardless of the number of nodes on the cluster.
When you launch an instance in a cluster, you declare the node to host the instance in your command and the instance runs on a single node.
Clusters must be either compute-optimized or storage-optimized. You can have a cluster of compute- optimized nodes, and some number of them can have GPUs. You can have a cluster made entirely of storage-optimized nodes. A cluster can't be made of a combination of compute-optimized nodes and storage-optimized nodes.
Pricing for Compute Instances on Snowball Edge
There are additional costs associated with using compute instances. For more information, see AWS Snowball Edge Pricing.
Prerequisites
Before creating your job, keep the following information in mind:
• Before you add any AMIs to your job request, make sure that you have created an AMI that is
supported in your AWS account. Currently, supported AMIs are based on the CentOS 7 (x86_64) - with Updates HVM and Ubuntu 16.04 LTS - Xenial (HVM) images. You can get these images from the AWS Marketplace website.
Creating a Linux AMI from an Instance
• All AMIs must be based on Amazon Elastic Block Store (Amazon EBS), with a single volume.
• If you are connecting to a compute instance running on a Snowball Edge, you must use Secure Shell (SSH). To do so, you first add the key pair. For more information, see Configuring an AMI to Use SSH to Connect to Compute Instances Launched on the Device (p. 135).
Creating a Linux AMI from an Instance
You can create an AMI using the AWS Management Console or the command line. Start with an existing AMI, launch an instance, customize it, create a new AMI from it, and finally, launch an instance of your new AMI.
To create an AMI from an instance using the console
1. Select an appropriate EBS-backed AMI as a starting point for your new AMI, and configure it as needed before launch. For more information, see Launching an instance using the Launch Instance Wizard in the Amazon EC2 User Guide for Linux Instances.
2. Choose Launch to launch an instance of the EBS-backed AMI that you selected. Accept the default values as you step through the wizard. For more information, see Launching an instance using the Launch Instance Wizard.
3. While the instance is running, connect to it. You can perform the following actions on your instance to customize it for your needs:
• Install software and applications.
• Copy data.
• Reduce start time by deleting temporary files, defragmenting your hard drive, and zeroing out free space.
• Attach additional Amazon EBS volumes.
4. (Optional) Create snapshots of all the volumes attached to your instance. For more information about creating snapshots, see Creating Amazon EBS snapshots in the Amazon EC2 User Guide for Linux Instances.
5. In the navigation pane, choose Instances, and choose your instance. Choose Actions, choose Image, and then choose Create image.
Tip
If this option isn't available, your instance isn't an Amazon EBS-backed instance.6. In the Create Image dialog box, specify the following information, and then choose Create image.
• Image name - A unique name for the image.
• Image description - An optional description of the image, up to 255 characters.
• No reboot - This option is not selected by default. Amazon EC2 shuts down the instance, takes snapshots of any attached volumes, creates and registers the AMI, and then reboots the instance.
Select No reboot to avoid having your instance shut down.
Warning
If you select No reboot, we can't guarantee the file system integrity of the created image.
• Instance Volumes - The fields in this section enable you to modify the root volume, and add more Amazon EBS and instance store volumes. For information about each field, pause on the i icon next to each field to display field tooltips. Some important points are listed following:
• To change the size of the root volume, locate Root in the Volume Type column. For Size (GiB), enter the required value.
• If you select Delete on Termination, when you terminate the instance created from this AMI, the Amazon EBS volume is deleted. If you clear Delete on Termination, when you terminate the instance, the Amazon EBS volume is not deleted. For more information, see Preserving Amazon EBS volumes on instance termination in the Amazon EC2 User Guide for Linux Instances.
Creating a Linux AMI from a Snapshot
• To add an Amazon EBS volume, choose Add New Volume (which adds a new row). For Volume Type, choose EBS, and fill in the fields in the row. When you launch an instance from your new AMI, additional volumes are automatically attached to the instance. Empty volumes must be formatted and mounted. Volumes based on a snapshot must be mounted.
• To add an instance store volume, see Adding instance store volumes to an AMI in the Amazon EC2 User Guide for Linux Instances. When you launch an instance from your new AMI, additional volumes are automatically initialized and mounted. These volumes don't contain data from the instance store volumes of the running instance on which you based your AMI.
7. To view the status of your AMI while it is being created, in the navigation pane, choose AMIs.
Initially, the status is pending but should change to available after a few minutes.
(Optional) To view the snapshot that was created for the new AMI, choose Snapshots. When you launch an instance from this AMI, we use this snapshot to create its root device volume.
8. Launch an instance from your new AMI. For more information, see Launching an instance using the Launch Instance Wizard in the Amazon EC2 User Guide for Linux Instances.
9. The new running instance contains all of the customizations that you applied in previous steps.
To Create an AMI from an Instance Using the Command Line
You can use one of the following commands. For more information about these command line interfaces, see Accessing Amazon EC2 in the Amazon EC2 User Guide for Linux Instances.
• create-image (AWS CLI)
• New-EC2Image (AWS Tools for Windows PowerShell)
Creating a Linux AMI from a Snapshot
If you have a snapshot of the root device volume of an instance, you can create an AMI from this snapshot using the AWS Management Console or the command line.
To create an AMI from a snapshot using the console
1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
2. In the navigation pane, under Elastic Block Store, choose Snapshots.
3. Choose the snapshot, choose Actions, and then choose Create image.
4. In the Create image from EBS snapshot dialog box, complete the fields to create your AMI. Then choose Create. If you're re-creating a parent instance, choose the same options as the parent instance.
• Architecture – Choose i386 for 32-bit or x86_64 for 64-bit.
• Root device name – Enter the appropriate name for the root volume. For more information, see Device naming on Linux instances in the Amazon EC2 User Guide for Linux Instances.
• Virtualization type – Choose whether instances launched from this AMI use paravirtual (PV) or hardware virtual machine (HVM) virtualization. For more information, see Linux AMI virtualization types.
• (PV virtualization type only) Kernel ID and RAM disk ID – Choose the AKI and ARI from the lists.
If you choose the default AKI, or you don't choose an AKI, you must specify an AKI every time you launch an instance using this AMI. In addition, your instance might fail the health checks if the default AKI is incompatible with the instance.
• (Optional) Block Device Mappings – Add volumes or expand the default size of the root volume for the AMI. For more information about resizing the file system on your instance for a larger
Using Amazon S3
volume, see Extending a Linux File system after resizing a volume in the Amazon EC2 User Guide for Linux Instances.
To Create an AMI from a Snapshot Using the Command Line
To create an AMI from a snapshot, you can use one of the following commands. For more information about these command line interfaces, see Accessing Amazon EC2 in the Amazon EC2 User Guide for Linux Instances.
• register-image (AWS CLI)
• Register-EC2Image (AWS Tools for Windows PowerShell)
Using Amazon S3 on Snowball
As part of the order process, you are asked to create an AWS Identity and Access Management (IAM) role and AWS Key Management Service (AWS KMS) key. The KMS key is used for encrypting the data during transit and at rest on the Snowball Edge device. For more information about creating IAM roles and KMS keys, see Creating an AWSAWS Snowball Edge Job.
Important
If the imported data must be encrypted in the S3 bucket using Server-Side Encryption with keys stored in AWS KMS (SSE-KMS), see Amazon S3 Encryption with AWS KMS (p. 27).
If the imported data must be encrypted in the S3 bucket using Server-Side Encryption with Amazon S3 managed keys (SSE-S3), see Amazon S3 Encryption with Server-Side Encryption (p. 29).
How Import Works
Each import job uses a single Snowball Edge device. After you create a job, we ship a Snowball Edge device to you. When it arrives, you connect the Snowball Edge device to your network and transfer the data that you want to import to Amazon S3 onto that Snowball Edge. When you’re done transferring data, ship the Snowball Edge back to AWS. We then import your data into Amazon S3.
Important
Snowball Edge cannot write to buckets if you have turned on S3 Object Lock. We also cannot write to your bucket if IAM policies on the bucket prevent writing to the bucket.
How Export Works
Each export job can use any number of AWS Snowball Edge devices. After you create a job, a listing operation starts in Amazon S3. This listing operation splits your job into parts. Each job part has exactly one device associated with it. After your job parts are created, your first job part enters the Preparing Snowball status.
Note
The listing operation to split your job into parts is a function of Amazon S3, and you are billed the same as Amazon S3 operation.We then start exporting your data onto a device. Typically, exporting data takes one business day.
However, this process can take longer. When the export is done, AWS gets the device ready for your regional carrier to pick up.
When the device arrives at your site, you connect it to your network and transfer the data that you want to import into Amazon S3 onto the device. When you’re done transferring the data, ship the device back to AWS. When we receive the returned device, we erase it completely. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards.
Amazon S3 Encryption with AWS KMS
This step marks the completion of that particular job part. If there are more job parts, the next job part now is prepared for shipping.
Important
Snowball Edge is unable to export files that are in S3 Glacier storage class. These objects must be restored before we can export the files. If we encounter files in S3 Glacier storage class, we contact you to let you know, but this might add delays to your export job.
Amazon S3 Encryption with AWS KMS
You can use the default AWS managed or customer managed encryption keys to protect your data when importing or exporting data.
Using Amazon S3 Default Bucket Encryption with AWS KMS Managed Keys
To enable AWS managed encryption with AWS KMS
1. Open the Amazon S3 console at https://console.aws.amazon.com/s3/.
2. Choose the Amazon S3 bucket that you want to encrypt.
3. In the wizard that appears on the right side, choose Properties.
4. In the Default encryption box, choose Disabled (this option is grayed out) to enable default encryption.
5. Choose AWS-KMS as the encryption method, and then choose the KMS key that you want to use.
This key is used to encrypt objects that are PUT into the bucket.
6. Choose Save.
After the Snowball Edge job is created, and before the data is imported, add a statement to the existing IAM role policy. This is the role you created during the ordering process. Depending on the job type, the default role name looks similar to Snowball-import-s3-only-role or Snowball-export-s3- only-role.
The following are examples of such a statement.
For importing data
If you use server-side encryption with AWS KMS managed keys (SSE-KMS) to encrypt the Amazon S3 buckets associated with your import job, you also need to add the following statement to your IAM role.
Example Example: Snowball import IAM role
{ "Effect": "Allow", "Action": [
"kms: GenerateDataKey", "kms: Decrypt"
],
"Resource":"arn:aws:kms:us-west-2:123456789012:key/abc123a1-abcd-1234- efgh-111111111111"
}
For exporting data
If you use server-side encryption with AWS KMS managed keys to encrypt the Amazon S3 buckets associated with your export job, you also must add the following statement to your IAM role.