• 沒有找到結果。

AWS Snowball Edge Developer Guide

N/A
N/A
Protected

Academic year: 2022

Share "AWS Snowball Edge Developer Guide"

Copied!
248
0
0

加載中.... (立即查看全文)

全文

(1)

AWS Snowball Edge

Developer Guide

(2)

AWS Snowball Edge Developer Guide

Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved.

Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.

(3)

Table of Contents

What Is Snowball Edge? ... 1

AWS Snowball Edge Features ... 1

Prerequisites for Using Snowball Edge ... 2

Related Services ... 2

Accessing the Service ... 3

Accessing an AWS Snowball Edge Device ... 3

Pricing for the AWS Snowball Edge ... 3

Are You a First-Time AWS Snowball User? ... 3

Device Differences ... 4

Snowball Edge Device Options ... 4

Use Case Differences ... 5

Tool Differences ... 6

How Snowball Edge Works ... 8

How Import Jobs Work ... 9

How Export Jobs Work ... 9

How Local Compute and Storage Jobs Work ... 10

How a Clustered Local Compute and Storage Job Works ... 10

Snowball Edge Videos and Blogs ... 10

Device Specifications ... 11

Snowball Edge Storage Optimized (for Data Transfer) Specifications ... 11

Snowball Edge Storage Optimized (with EC2) Specifications ... 12

Snowball Edge Compute Optimized Specifications ... 13

Supported Network Hardware ... 15

Setting Up ... 18

Sign Up for AWS ... 18

Create an IAM User ... 18

Next Step ... 20

Before You Order a Device ... 21

About the Local Environment ... 21

Working with Special Characters ... 22

Using Amazon EC2 ... 22

Using Compute Instances on Clusters ... 23

Pricing for Compute Instances on Snowball Edge ... 23

Prerequisites ... 23

Creating a Linux AMI from an Instance ... 24

Creating a Linux AMI from a Snapshot ... 24

Using Amazon S3 ... 26

How Import Works ... 26

How Export Works ... 26

Amazon S3 Encryption with AWS KMS ... 27

Amazon S3 Encryption with Server-Side Encryption ... 29

Snowball Edge Clusters ... 30

Snowball Edge Cluster Quorums ... 30

Cluster Job Considerations ... 31

Getting Started ... 32

Creating a Snowball Edge Job ... 32

Step 1: Plan Your Job ... 33

Step 2: Choose Your Shipping Preferences ... 34

Step 3: Choose Your Job Details ... 34

Step 4: Choose Your Security Preferences ... 35

Step 5: Choose Your Notification Preferences ... 39

Step 6: Download AWS OpsHub ... 39

Step 7: Review and Create Your Job ... 40

Receiving the Snowball Edge ... 40

(4)

Connecting to Your Local Network ... 41

Getting Your Credentials and Tools ... 42

Downloading and Installing the Snowball Edge client ... 43

Unlocking the Snowball Edge ... 43

Setting Up Local Users ... 44

Using Your Snowball Edge ... 45

Powering Off the Snowball Edge ... 46

Returning the Device ... 46

Disconnecting the Snowball Edge ... 46

Monitoring the Import Status ... 47

Getting Your Job Completion Report and Logs ... 47

Where Do I Go from Here? ... 48

Using AWS OpsHub to Manage Devices ... 49

Unlocking a device ... 49

Verifying the signature of AWS OpsHub (optional) ... 50

Managing AWS services ... 52

Using Compute Instances Locally ... 53

Managing clusters ... 58

Managing S3 storage ... 59

Using NFS file share to upload files ... 60

Using AWS IoT Greengrass on EC2 instances ... 63

Setting up your Amazon EC2 instance ... 63

Managing Your Devices ... 64

Rebooting Your Device ... 65

Editing Your Device Alias ... 65

Getting Updates ... 65

Managing Profiles ... 66

Automating Your Management Tasks ... 66

Creating and Starting a Task ... 67

Viewing Details of a Task ... 69

Deleting a Task ... 69

Setting the NTP time servers for your device ... 69

Using a Snowball Edge Device ... 71

Using the Snowball Edge Client ... 72

Downloading and Installing the Snowball Edge Client ... 72

Commands for the Snowball Edge Client ... 72

Transferring Files Using the S3 Interface ... 89

Downloading and Installing the AWS CLI Version 1.16.14 ... 89

Using the AWS CLI and API Operations on Snowball Edge ... 90

Getting and Using Local Amazon S3 Credentials ... 91

Unsupported Amazon S3 Features for Snowball Edge ... 92

Batching Small Files ... 92

Supported CLI Commands ... 94

Supported REST API Actions ... 96

Transferring Files Using the File Interface ... 98

Overview of the File Interface ... 99

Starting the File Interface ... 100

Mounting a Bucket with the File Interface ... 101

Monitoring the File Interface ... 104

Using NFS for Offline Data Transfer ... 105

Troubleshooting NFS Issues ... 106

Using an AWS Snowball Edge device with a Tape Gateway ... 107

Ordering a Snowball Edge device with a Tape Gateway ... 108

Deploying a Snowball Edge device with a Tape Gateway ... 108

Troubleshooting and best practices for a Snowball Edge device with a Tape Gateway ... 109

Using the AWS Snow Family API with a Snowball Edge device with a Tape Gateway ... 110

Using AWS Lambda ... 112

(5)

Before You Start ... 113

Getting Started with Lambda ... 114

Using Amazon EC2 ... 118

Overview ... 119

Compute Instances on Clusters ... 23

Pricing for Compute Instances on Snowball Edge ... 23

Using AMIs on Your Device ... 120

Importing an AMI to Your Device ... 123

Using the AWS CLI and API Operations ... 133

Quotas for Compute Instances ... 133

Creating a Compute Job ... 135

Network Configuration for Compute Instances ... 136

Using SSH to Connect to a Compute Instance ... 140

Transferring Data from Compute Instances to Buckets on the Same Device ... 141

Snowball Edge Client Commands for Compute Instances ... 141

Using the Amazon EC2 Endpoint ... 145

Autostarting EC2 Instances ... 156

Using Block Storage with EC2 Instances ... 157

Security Groups ... 157

Supported Instance Metadata and User Data ... 158

Stopping EC2 Instances ... 159

Troubleshooting Compute Instances ... 159

Using IAM Locally ... 160

Using the AWS CLI and API Operations ... 161

Supported IAM AWS CLI Commands ... 161

IAM Policy Examples ... 164

TrustPolicy Example ... 167

Using AWS STS ... 167

Using the AWS CLI and API Operations on Snowball Edge ... 167

Supported AWS STSAWS CLI Commands on a Snowball Edge ... 168

Supported AWS STS API Operations ... 168

Ports Required to Use AWS Services ... 169

Using a Snowball Edge Cluster ... 170

Clustering Overview ... 170

Snowball Edge Cluster Quorums ... 170

Cluster Job Considerations ... 171

Related Topics ... 171

Administering a Cluster ... 172

Reading and Writing Data to a Cluster ... 172

Reconnecting an Unavailable Cluster Node ... 172

Removing an Unhealthy Node from a Cluster ... 173

Adding or Replacing a Node in a Cluster ... 173

Understanding AWS Snowball Edge Jobs ... 175

Job Details ... 175

Job Statuses ... 177

Cluster Statuses ... 178

Importing Jobs into Amazon S3 ... 179

Exporting Jobs from Amazon S3 ... 180

Using Export Ranges ... 180

Export Jobs Best Practices ... 182

Local Compute and Storage Only Jobs ... 182

Local Compute Jobs ... 183

Local Storage Jobs ... 183

Local Cluster Option ... 183

Cloning a Job in the Console ... 183

Canceling Jobs in the Console ... 184

Best Practices ... 185

(6)

Security ... 185

Resource Management ... 186

Performance ... 186

Performance Recommendations ... 187

Speeding Up Data Transfer ... 187

Transferring Petabytes of Data ... 187

Planning Your Large Transfer ... 188

Calibrating a Large Transfer ... 189

Updating a Snowball Edge ... 191

Prerequisites ... 191

Downloading Updates ... 191

Installing Updates ... 192

Shipping Considerations ... 194

Preparing an AWS Snowball Edge for Shipping ... 194

Region-Based Shipping Restrictions ... 195

Shipping an AWS Snowball Edge ... 195

Shipping Carriers ... 195

Security ... 200

Data Protection ... 200

Protecting Data in the Cloud ... 201

Protecting Data On Your Device ... 203

Identity and Access Management ... 205

Access Control for Console and Jobs ... 205

Logging and Monitoring ... 219

Compliance Validation ... 219

Resilience ... 220

Infrastructure Security ... 220

Data Validation ... 221

Checksum Validation of Transferred Data ... 221

Local Inventory Creation During Snowball Transfer ... 221

Common Validation Errors ... 221

Manual Data Validation for Snowball Edge After Import into Amazon S3 ... 222

Notifications ... 223

Logging with AWS CloudTrail ... 224

AWS Snowball Edge Information in CloudTrail ... 224

Understanding Log File Entries for AWS Snowball Edge ... 225

Quotas ... 226

Region Availability for AWS Snowball Edge ... 226

Limitations for AWS Snowball Edge Jobs ... 227

Limitations on Transferring On-Premises Data with a Snowball Edge Device ... 227

Limitations for Lambda Powered by AWS IoT Greengrass ... 227

Limitations on Shipping a Snowball Edge ... 228

Limitations on Processing Your Returned Snowball Edge for Import ... 228

Troubleshooting ... 229

Identify Your Device ... 230

Connection Problems ... 230

Manifest File Problems ... 230

Credentials Problems ... 230

Unable to Locate AWS CLI Credentials ... 230

Error Message: Check Your Secret Access Key and Signing ... 231

Data Transfer Problems ... 231

Troubleshooting Problems with Transferring Data Using the File Interface ... 231

AWS CLI Problems ... 232

AWS CLI Error Message: "Profile Cannot Be Null" ... 232

Null Pointer Error When Transferring Data with the AWS CLI ... 232

Import Job Problems ... 233

Export Job Problems ... 233

(7)

API Reference ... 234 Document History ... 235 AWS glossary ... 241

(8)

AWS Snowball Edge Features

What Is AWS Snowball Edge?

AWS Snowball Edge is a type of Snowball device with on-board storage and compute power for select AWS capabilities. Snowball Edge can do local processing and edge-computing workloads in addition to transferring data between your local environment and the AWS Cloud.

Each Snowball Edge device can transport data at speeds faster than the internet. This transport is done by shipping the data in the appliances through a regional carrier. The appliances are rugged, complete with E Ink shipping labels.

Snowball Edge devices have three options for device configurations—Storage Optimized, Compute Optimized, and Compute Optimized with GPU. When this guide refers to Snowball Edge devices, it's referring to all options of the device. When specific information applies only to one or more optional configurations of devices (such as how the Snowball Edge with GPU has an on-board GPU), it is called out specifically. For more information, see Snowball Edge Device Options (p. 4).

Topics

• AWS Snowball Edge Features (p. 1)

• Prerequisites for Using Snowball Edge (p. 2)

• Services Related to the AWS Snowball Edge (p. 2)

• Accessing the Service (p. 3)

• Pricing for the AWS Snowball Edge (p. 3)

• Are You a First-Time AWS Snowball User? (p. 3)

• AWS Snowball Edge Device Differences (p. 4)

AWS Snowball Edge Features

Snowball Edge devices have the following features:

• Large amounts of storage capacity or compute functionality for devices. This depends on the options you choose when you create your job.

• Network adapters with transfer speeds of up to 100 Gbit/second.

• Encryption is enforced, protecting your data at rest and in physical transit.

• You can import or export data between your local environments and Amazon S3, and physically transport the data with one or more devices without using the internet.

• Snowball Edge devices are their own rugged box. The built-in E Ink display changes to show your shipping label when the device is ready to ship.

• Snowball Edge devices come with an on-board LCD display that can be used to manage network connections and get service status information.

• You can cluster Snowball Edge devices for local storage and compute jobs to achieve data durability across 5–10 devices and locally grow or shrink storage on demand.

• You can use the file interface to read and write data to an AWS Snowball Edge device through a file share or Network File System (NFS) mount point.

(9)

Prerequisites for Using Snowball Edge

• You can write Python-language Lambda functions and associate them with Amazon S3 buckets when you create an AWS Snowball Edge device job. Each function triggers when a local Amazon S3 PUT object action is run on the associated bucket on the device.

• Snowball Edge devices have Amazon S3 and Amazon EC2 compatible endpoints available, enabling programmatic use cases.

• Snowball Edge devices support the new sbe1, sbe-c, and sbe-g instance types, which you can use to run compute instances on the device using Amazon Machine Images (AMIs).

Prerequisites for Using Snowball Edge

Before creating your first job, keep the following in mind.

For jobs that import data into Amazon S3, follow these steps:

• Create an AWS account with AWS Identity and Access Management (IAM) administrator-level permissions. For more information, see Setting Up Your AWS Access for AWS Snowball Edge (p. 18).

• Confirm that the files and folders to transfer are named according to the object key naming guidelines for Amazon S3. Any files or folders with names that don't meet these guidelines aren't imported into Amazon S3.

• Plan what data you want to import into Amazon S3. For more information, see Transferring Petabytes of Data Efficiently (p. 187).

Before exporting data from Amazon S3, follow these steps:

• Understand what data is exported when you create your job. For more information, see Using Export Ranges (p. 180).

• For any files with a colon (:) in the file name, change the file names in Amazon S3 before you create the export job to get these files. Files with a colon in the file name fail export to Microsoft Windows Server.

For jobs using compute instances:

• Before you can add any AMIs to your job, you must have an AMI in your AWS account and it must be a supported image type. Currently, supported AMIs are based on the Amazon Linux 2, CentOS 7 (x86_64) - with Updates HVM, or Ubuntu 16.04 LTS - Xenial (HVM) images. You can get these images from the AWS Marketplace.

• If you're using SSH to connect to the instances running on a Snowball Edge, you must already have the key pair for connecting to the instance.

• For information specific to using compute instances on a device, see Using Amazon EC2 Compute Instances (p. 118).

Services Related to the AWS Snowball Edge

You can use an AWS Snowball Edge device with the following related AWS services:

Amazon S3 – Transfer data to an AWS Snowball Edge device using the Amazon S3 API for Snowball Edge, which supports a subset of the Amazon S3 API operations. You can do this in a single Snowball Edge device or in a cluster of devices for increased data durability.

You can also import data that is hosted on an AWS Snowball Edge device to Amazon S3 and your local environment through a shipped Snowball Edge device. For more information, see the Amazon Simple Storage Service User Guide.

(10)

Accessing the Service

Amazon EC2 – Run compute instances on a Snowball Edge device using the Amazon EC2 compatible endpoint, which supports a subset of the Amazon EC2 API operations. For more information about using Amazon EC2 in AWS, see Getting started with Amazon EC2 Linux instances.

AWS Lambda powered by AWS IoT Greengrass – Invoke Lambda functions based on Amazon S3 storage actions made on an AWS Snowball Edge device. These Lambda functions are associated with an AWS Snowball Edge device during job creation. For more information about using Lambda, see the AWS Lambda Developer Guide.

Amazon Elastic Block Store (Amazon EBS) – Provide block-level storage volumes for use with EC2 instances. For more information, see Amazon Elastic Block Store (Amazon EBS).

AWS Identity and Access Management (IAM) – Use this service to securely control access to AWS resources. For more information, see What is IAM?

AWS Security Token Service (AWS STS) – Request temporary, limited-privilege credentials for IAM users or for users that you authenticate (federated users). For more information, see Temporary security credentials in IAM.

Amazon EC2 Systems Manager – Use this service to view and control your infrastructure on AWS. For more information, see What is AWS Systems Manager?

Accessing the Service

You can either use the AWS Snow Family Management Console or the job management API to create and manage jobs. For information about the job management API, see Job Management API Reference for AWS Snowball.

Accessing an AWS Snowball Edge Device

After your Snowball Edge device or devices are onsite, you can access them in several different ways. You can use the LCD display (used only for network configuration) that's built into each device, the Amazon S3 and Amazon EC2 compatible endpoints, or the available file interface. For more information, see Using an AWS Snowball Edge Device (p. 71).

Pricing for the AWS Snowball Edge

For information about the pricing and fees associated with the service and its devices, see AWS Snowball Edge Pricing.

Are You a First-Time AWS Snowball User?

If you are a first-time user of the AWS Snow Family service, we recommend that you read the following sections in order:

1. For information about device types and options, see AWS Snowball Edge Device Differences (p. 4).

2. To learn more about the types of jobs, see Understanding AWS Snowball Edge Jobs (p. 175).

3. For an end-to-end overview of how to use an AWS Snowball Edge device, see How AWS Snowball Edge Works (p. 8).

4. When you're ready to get started, see Getting Started (p. 32).

5. For information about using compute instances on a device, see Using Amazon EC2 Compute Instances (p. 118).

(11)

Device Differences

AWS Snowball Edge Device Differences

This guide contains documentation for the Snowball Edge devices. You can use these devices to move huge amounts of data into and out of Amazon S3. You can order them using the job management API or the AWS Snow Family console. For frequently asked questions and pricing information, see AWS Snowball.

Topics

• Snowball Edge Device Options (p. 4)

• AWS Snow Family Use Case Differences (p. 5)

• AWS Snow Family Tool Differences (p. 6)

Snowball Edge Device Options

Snowball Edge devices have the following options for device configurations:

Snowball Edge Storage Optimized (for data transfer) – This Snowball Edge device option has a 100 TB (80 TB usable) storage capacity.

Snowball Edge Storage Optimized (with EC2 compute functionality) – This Snowball Edge device option has up to 80 TB of usable storage space, 24 vCPUs, and 32 GiB of memory for compute functionality. It also comes with 1 TB of additional SSD storage space for block volumes attached to Amazon EC2 AMIs.

Snowball Edge Compute Optimized – This Snowball Edge device option has the most compute functionality, with 52 vCPUs, 208 GiB of memory, and 42 TB (39.5 usable) plus 7.68 TB of dedicated NVMe SSD for compute instances for block storage volumes for EC2 compute instances, and 42 TB of HDD capacity for either object storage or block storage volumes.

Snowball Edge Compute Optimized with GPU – This Snowball Edge device option is identical to the Compute Optimized option, except for an installed GPU, equivalent to the one available in the P3 Amazon EC2 instance type. It has a storage capacity of 42 TB (39.5 TB of HDD storage that can be used for a combination of Amazon S3 compatible object storage and Amazon EBS compatible block storage volumes) plus 7.68 TB of dedicated NVMe SSD for compute instances.

For more information about the compute functionality of these three options, see Using Amazon EC2 Compute Instances (p. 118).

Note

When this guide refers to Snowball Edge devices, it's referring to all optional variants of the device. Whenever specific information applies only to one or more optional configurations (such as how the Snowball Edge Compute Optimized with GPU option has an on-board GPU peripheral), it is mentioned explicitly.

The following table summarizes the differences between the various device options. For hardware specification information, see AWS Snowball Edge Specifications (p. 11).

  Snowball Edge Storage

Optimized (for data transfer)

Snowball Edge Storage Optimized (with EC2 compute functionality)

Snowball Edge Compute Optimized

CPU AMD Naples, 32 cores,

3.4Ghz Intel Xeon D processor,

16 cores, 1.8Ghz AMD Naples, 32 cores, 3.4Ghz

vCPUs   24 52

(12)

Use Case Differences

  Snowball Edge Storage

Optimized (for data transfer)

Snowball Edge Storage Optimized (with EC2 compute functionality)

Snowball Edge Compute Optimized

Usable memory   32 GB 208 GB

Security card Yes Yes Yes

GPU (optional) None None None (NVidia V100)

SSD   1 TB SATA 7.68 TB NVMe

Usable HDD 80 TB plus 1 TB of dedicated SATA SSD for compute instances.

80 TB 39.5 TB

Network interfaces • 2x 10 Gbit – RJ45

• 1x 25 Gbit – SFP28

• 1x 100 Gbit – QSFP28

• 1x 10 Gbit – RJ45

• 1x 25 Gbit – SFP28

• 1x 40 Gbit – QSFP+

• 2x 10 Gbit – RJ45

• 1x 25 Gbit – SFP28

• 1x 100 Gbit – QSFP28 Physical security

features • Hidden magnetic

screws

• Intrusion switches

• NFC tags

• Anti-tamper inserts

• Android app for tamper detection

• GPS and cellular

• Conformal coating

• Hidden magnetic screws

• Anti-tamper inserts

• Conformal coating

• Hidden magnetic screws

• Intrusion switches

• NFC tags

• Anti-tamper inserts

• Android app for tamper detection

• GPS and cellular

• Conformal coating

AWS Snow Family Use Case Differences

The following table shows the different use cases for the different AWS Snow Family devices.

Use case Snowball Edge AWS Snowcone

Import data into

Amazon S3 ✓ ✓

Export from Amazon S3 ✓  

Durable local storage ✓  

Local compute with

AWS Lambda ✓  

Local compute

instances ✓ ✓

Durable Amazon S3 storage in a cluster of devices

✓  

Use with AWS IoT

Greengrass (IoT) ✓  

(13)

Tool Differences

Use case Snowball Edge AWS Snowcone

Transfer files through

NFS with a GUI ✓ ✓

GPU workloads ✓  

Note

Workloads that need GPU support require the Snowball Edge Compute Optimized with GPU option.

AWS Snow Family Tool Differences

The following outlines the different tools used with the Snow Family devices, and how they are used.

Snowball Edge Tools

AWS OpsHub for Snow Family

• The Snow Family devices now offer a user-friendly tool, AWS OpsHub for Snow Family, that you can use to manage your devices and local AWS services. You use AWS OpsHub on a client computer to perform tasks such as unlocking and configuring single or clustered devices, transferring files, and launching and managing instances running on Snow Family devices. For more information, see Using AWS OpsHub for Snow Family to Manage Snowball Devices.

Snowball Edge client with Snowball Edge

• Download the Snowball Edge client from the AWS Snowball Edge Resources page and install it on your own computer.

• Use the Snowball Edge client to unlock the Snowball Edge or the cluster of Snowball Edge devices. For more information, see Using the Snowball Edge Client (p. 72).

• The Snowball Edge client doesn't transfer data.

Amazon S3 interface with Snowball Edge

• Is already installed on the Snowball Edge by default. It does not need to be downloaded or installed.

• Can transfer data to or from the Snowball Edge. For more information, see Transferring Files Using the Amazon S3 Interface (p. 89).

• Encrypts data on the Snowball Edge while the data is transferred to the device.

File interface with Snowball Edge

• Is already installed on the Snowball Edge by default. It does not need to be downloaded or installed.

• Can transfer data by dragging and dropping files up to 150 GB in size from your computer to the buckets on the Snowball Edge through an easy-to-configure NFS mount point. For more information, see Transferring Files to AWS Snowball Edge Using the File Interface (p. 98).

• Encrypts data on the Snowball Edge while the data is transferred to the device.

AWS IoT Greengrass console with Snowball Edge

• With a Snowball Edge, you can use the AWS IoT Greengrass console to update your AWS IoT Greengrass group and the core running on the Snowball Edge.

(14)

Tool Differences

Items Provided for Snowball Edge

The following outlines the network adapters, cables used, and cables provided for the Snowball Edge device.

Network interface Snowball Edge support Cables provided with device

RJ45 ✓ Not provided.

SFP28 ✓ Not provided.

SFP28 (with optic

connector) ✓ No cables provided.

No optic connector provided for Snowball Edge devices.

QSFP ✓ No cables or optics

provided.

For more information about the network interfaces, cables, and connectors, see Supported Network Hardware (p. 15).

(15)

How AWS Snowball Edge Works

AWS Snowball Edge devices are owned by AWS, and they reside at your on-premises location while they're in use.

There are three job types you can use with an AWS Snowball Edge device. Although the job types differ in their use cases, every job type has the same workflow for how you order, receive, and return devices.

Regardless of the job type, every job follows a data erasure of the National Institute of Standards and Technology (NIST) 800-88 standard after the job completes.

The shared workflow

1.Create the job – Each job is created in the AWS Snow Family Management Console or

programmatically through the job management API. The status for a job can be tracked in the console or through the API.

2.A device is prepared for your job – We prepare an AWS Snowball Edge device for your job, and the status of your job is now Preparing Snowball.

3.A device is shipped to you by your region's carrier – The carrier takes over from here, and the status of your job is now In transit to you. You can find your tracking number and a link to the tracking website on the console or with the job management API. For information about who your region's carrier is, see Shipping Considerations for AWS Snowball (p. 194).

4.Receive the device – A few days later, your region's carrier delivers the AWS Snowball Edge device to the address that you provided when you created the job, and the status of your job changes to Delivered to you. When it arrives, you’ll notice that it didn’t arrive in a box, because the device is its own shipping container.

5.Get your credentials and download the Snowball Edge client – Get ready to start transferring data by getting your credentials, your job manifest, and the manifest's unlock code, and then downloading the Snowball Edge client.

• The 8 client is the tool that you use to manage the flow of data from the device to your on-premises data destination.

You can download and install the Snowball Edge client from the AWS Snowball resources page.

You must download the Snowball Edge client from the AWS Snowball Edge Resources page and install on a powerful workstation that you own.

• The manifest is used to authenticate your access to the device, and it is encrypted so that only the unlock code can decrypt it. You can get the manifest from the console or with the job management API when the device is on-premises at your location.

• The unlock code is a 29-character code used to decrypt the manifest. You can get the unlock code from the console or with the job management API. We recommend that you keep the unlock code saved somewhere separate from the manifest to prevent unauthorized access to the device while it’s at your facility.

6.Position the hardware – Move the device into your data center and open it following the instructions on the case. Connect the device to power and your local network.

7.Power on the device – Next, power on the device by pressing the power button above the LCD display.

Wait a few minutes, and the Ready screen appears.

8.Get the IP address for the device – The LCD display has a CONNECTION tab on it. Tap this tab and get the IP address for the AWS Snowball Edge device.

9.Use the Snowball Edge client to unlock the device – When you use the Snowball Edge client to unlock the AWS Snowball Edge device, enter the IP address of the device, the path to your manifest,

(16)

How Import Jobs Work

and the unlock code. The Snowball Edge client decrypts the manifest and uses it to authenticate your access to the device.

10.Use the device – The device is up and running. You can use it to transfer data or for local compute and storage. You can read and write data with the Amazon S3 interface or the Network File System (NFS) mount point.

11.Prepare the device for its return trip – After you're done with the device in your on-premises location and the file interface status is Complete, press the power button above the LCD display. It takes about 20 seconds or so for the device to power off. Unplug the device and its power cables into the cable nook on top of the device, and shut all three of the device's doors. The device is now ready to be returned.

12.Your region's carrier returns the device to AWS – When the carrier has the AWS Snowball Edge device, the status for the job becomes In transit to AWS.

Note

There are additional steps for export and cluster jobs. For more information, see How Export Jobs Work (p. 9) and How a Clustered Local Compute and Storage Job Works (p. 10).

Topics

• How Import Jobs Work (p. 9)

• How Export Jobs Work (p. 9)

• How Local Compute and Storage Jobs Work (p. 10)

• Snowball Edge Videos and Blogs (p. 10)

How Import Jobs Work

Each import job uses a single Snowball appliance. After you create a job in the AWS Snow Family Management Console or the job management API, we ship a Snowball to you. When it arrives in a few days, you connect the Snowball Edge device to your network and transfer the data that you want imported into Amazon S3 onto the device. When you’re done transferring data, ship the Snowball back to AWS, and we import your data into Amazon S3.

How Export Jobs Work

Each export job can use any number of AWS Snowball Edge devices. If the listing contains more data than can fit on a single device, multiple devices are provided to you. Each job part has exactly one device associated with it. After your job parts are created, your first job part enters the Preparing Snowball status.

Note

The listing operation used to split your job into parts is a function of Amazon S3, and you are billed for it the same way as any Amazon S3 operation.

Soon after that, we start exporting your data onto a device. Typically, exporting data takes one business day. However, this process can take longer depending on the amount and type of data. When the export is done, AWS gets the device ready for pickup by your region's carrier. When it arrives, you connect the AWS AWS Snowball Edge device to your network and transfer the data that you want to import from Amazon S3 onto the device.

When you’re done transferring data, ship the device back to AWS. When we receive the device for your export job part, we erase it completely. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards. This step marks the completion of that particular job part.

• For keylisting

(17)

How Local Compute and Storage Jobs Work

Before we export the objects in the S3 bucket, we scan the bucket. If the bucket is altered after the scan, the job could encounter delays because we scan for missing or altered objects.

• For S3 Glacier Flexible Retrieval

It is important to note that AWS Snowball cannot export objects in the S3 Glacier Flexible Retrieval storage class. These objects must be restored before AWS Snowball can successfully export the objects in the bucket.

How Local Compute and Storage Jobs Work

You can use the local compute and storage functionality of an AWS Snowball Edge device with all job types in AWS Regions that support Lambda. The compute functionality is named AWS Lambda powered by AWS IoT Greengrass, where Python-language AWS Lambda functions can be triggered by Amazon S3 PUT object actions on buckets specified when you created the job. For more information, see Local Compute and Storage Only Jobs (p. 182).

How a Clustered Local Compute and Storage Job Works

A cluster job is a special kind of job for local storage and compute only. It is for those workloads that require increased data durability and storage capacity. For more information, see Local Cluster Option (p. 183).

Note

Like standalone local storage and compute jobs, the data stored in a cluster can't be imported into Amazon S3 without ordering additional devices as a part of separate import jobs. If you order these devices, you can transfer the data from the cluster to the devices and import the data when you return the devices for the import jobs.

Clusters have 5–10 AWS Snowball Edge devices, called nodes. When you receive the nodes from your regional carrier, connect all the nodes to power and your network to obtain their IP addresses. You use these IP addresses to unlock all the nodes of the cluster at once with a single unlock command, using the IP address of one of the nodes. For more information, see Using the Snowball Edge Client (p. 72).

You can write data to an unlocked cluster by using the Amazon S3 interface or the NFS mount point through the leader node and the data distributed among the other nodes.

When you’re done with your cluster, ship all the nodes back to AWS. When we receive the cluster node, we perform a complete erasure of the Snowball. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards.

Snowball Edge Videos and Blogs

• AWS Snowball Edge Data Migration

• AWS OpsHub for Snow Family

• Novetta delivers IoT and Machine Learning to the edge for disaster response

• Enable large-scale database migrations with DMS and AWS Snowball

• Data Migration Best Practices with AWS Snowball Edge

• AWS Snowball resources

(18)

Snowball Edge Storage Optimized (for Data Transfer) Specifications

AWS Snowball Edge Specifications

In this section, you can find hardware specifications for Snowball Edge devices.

Topics

• Snowball Edge Storage Optimized (for Data Transfer) Specifications (p. 11)

• Snowball Edge Storage Optimized (with EC2) Specifications (p. 12)

• Snowball Edge Compute Optimized Specifications (p. 13)

• Supported Network Hardware (p. 15)

Snowball Edge Storage Optimized (for Data Transfer) Specifications

The following table contains hardware specifications for Snowball Edge Storage Optimized devices.

Item Snowball Edge Storage Optimized (for Data

Transfer) specifications

Storage specifications  

HDD storage capacity 80 TB of usable

Power supply specifications  

Power In AWS Regions in the US: NEMA 5–15p 100–

220 volts. In all AWS Regions, a power cable is included

Power consumption 304 watts for an average use case, though the

power supply is rated for 1200 watts.

Voltage 100 – 240V AC

Frequency 47/63 Hz

Data and network connections 2x 10 Gbit – RJ45 1x 25 Gbit – SFP28 1x 100 Gbit – QSFP28

Cables Each AWS Snowball Edge device ships country-

specific power cables. No other cables or optics are provided. For more information, see Supported Network Hardware (p. 15).

Thermal requirements AWS Snowball Edge devices are designed for office operations, and are ideal for data center operations.

Decibel output On average, an AWS Snowball Edge device

produces 68 decibels of sound, typically quieter than a vacuum cleaner or living-room music.

(19)

Snowball Edge Storage Optimized (with EC2) Specifications

Item Snowball Edge Storage Optimized (for Data

Transfer) specifications Dimensions and weight specifications  

Weight 49.7 pounds (22.54 Kg)

Height 15.5 inches (394 mm)

Width 10.6 inches (265 mm)

Length 28.3 inches (718 mm)

Environment specifications  

Vibration Non-operational use equivalent to ASTM D4169

Truck level I 0.73 GRMS

Shock Operational use equivalent to 70G (MIL-S-901)

Non-operational use equivalent to 50G (ISTA-3A)

Altitude Operational use equivalent to 0–3,000 meters (0–

10,000 feet)

Non-operational use equivalent to 0–12,000 meters

Temperature range 0–45°C (operational)

Snowball Edge Storage Optimized (with EC2) Specifications

The following table contains hardware specifications for Snowball Edge Storage Optimized (with EC2) devices.

Item Snowball Edge Storage Optimized (with EC2) specifications Compute and memory

specifications  

CPU 24 vCPUs

RAM 32 GB RAM

Storage specifications  

HDD storage capacity 80 TB usable (for object and block storage) SSD storage capacity 1 TB usable SATA SSD storage (for block storage) Power supply

specifications  

Power In AWS Regions in the US: NEMA 5–15p 100–220 volts. In all AWS Regions, a power cable is included

(20)

Snowball Edge Compute Optimized Specifications

Item Snowball Edge Storage Optimized (with EC2) specifications

Power consumption 304 watts for an average use case, though the power supply is rated for 1200 watts

Voltage 100 – 240V AC

Frequency 47/63 Hz

Data and network

connections 1x 10 Gbit – RJ45 1x 25 Gbit – SFP28 1x 40 Gbit – QSFP28

Cables Each AWS Snowball Edge device ships country-specific power cables. No other cables or optics are provided. For more information, see Supported Network Hardware (p. 15).

Thermal requirements AWS Snowball Edge devices are designed for office operations, and are ideal for data center operations.

Decibel output On average, an AWS Snowball Edge device produces 68 decibels of sound, typically quieter than a vacuum cleaner or living-room music.

Dimensions and weight

specifications  

Weight 49.7 pounds (22.45 Kg)

Height 15.5 inches (394 mm)

Width 10.6 inches (265 mm)

Length 28.3 inches (718 mm)

Environment

specifications  

Vibration Non-operational use equivalent to ASTM D4169 Truck level I 0.73 GRMS Shock Operational use equivalent to 70G (MIL-S-901)

Non-operational use equivalent to 50G (ISTA-3A)

Altitude Operational use equivalent to 0–3,000 meters (0–10,000 feet) Non-operational use equivalent to 0–12,000 meters

Temperature range 0–45°C (operational)

Snowball Edge Compute Optimized Specifications

The following table contains hardware specifications for Snowball Edge Compute Optimized and Compute Optimized with GPU devices.

Item Snowball Edge Compute Optimized specifications

Compute and memory specifications  

(21)

Snowball Edge Compute Optimized Specifications

Item Snowball Edge Compute Optimized specifications

CPU 52 vCPUs

RAM 256 GB RAM (208 GB RAM - Customer usable)

GPU nVidia V100 (available in Compute Optimized with GPU

configuration)

Storage specifications  

HDD storage capacity 42 TB usable (for object and block storage) SSD storage capacity 7.68 TB usable NVMe SSD (for block storage) Power supply specifications  

Power In AWS Regions in the US: NEMA 5–15p 100–220 volts. In all

AWS Regions, a power cable is included

Power consumption 304 watts for an average use case, though the power supply is rated for 1200 watts

Voltage 100 – 240V AC

Frequency 47/63 Hz

Data and network connections 2x 10 Gbit – RJ45 1x 25 Gbit – SFP28 1x 100 Gbit – QSFP28

Cables Each AWS Snowball Edge device ships country-

specific power cables. No other cables or optics are provided. For more information, see Supported Network Hardware (p. 15).

Thermal requirements AWS Snowball Edge devices are designed for office operations, and are ideal for data center operations.

Decibel output On average, an AWS Snowball Edge device produces 68 decibels of sound, typically quieter than a vacuum cleaner or living-room music.

Dimensions and weight specifications  

Weight 49.7 pounds (22.45 Kg)

Height 15.5 inches (394 mm)

Width 10.6 inches (265 mm)

Length 28.3 inches (718 mm)

Environment specifications  

Vibration Non-operational use equivalent to ASTM D4169 Truck level I 0.73 GRMS

(22)

Supported Network Hardware

Item Snowball Edge Compute Optimized specifications

Shock Operational use equivalent to 70G (MIL-S-901)

Non-operational use equivalent to 50G (ISTA-3A)

Altitude Operational use equivalent to 0–3,000 meters (0–10,000

feet)

Non-operational use equivalent to 0–12,000 meters

Temperature range 0–45°C (operational)

Supported Network Hardware

To use the AWS Snowball Edge device, you need your own network cables. For RJ45 cables, there are no specific recommendations. SFP28 and QSFP28 cables and modules from Mellanox and Finisar have been verified to be compatible with the device.

After you open the back panel of the AWS Snowball Edge device, you see the network ports shown in the following photograph.

Only one network interface on the AWS Snowball Edge device can be used at a time. Hence use any one of the ports to support the following network hardware.

SFP

This port provides a 10G/25G SFP28 interface compatible with SFP28 and SFP+ transceiver modules and direct-attach copper (DAC) cables. You must provide your own transceivers or DAC cables.

• For 10G operation, you can use any SFP+ option. Examples include:

• 10Gbase-LR (single mode fiber) transceiver

• 10Gbase-SR (multi-mode fiber) transceiver

• SFP+ DAC cable

• For 25G operation, you can use any SFP28 option. Examples include:

• 25Gbase-LR (single mode fiber) transceiver

• 25Gbase-SR (multi-mode fiber) transceiver

• SFP28 DAC cable

(23)

Supported Network Hardware

QSFP

This port provides a 40G QSFP28 interface on storage-optimized devices and a 40/50/100G QSFP28 interface on compute-optimized devices. Both are compatible with QSFP+ transceiver modules and DAC cables. You must provide your own transceivers or DAC cables. Examples include the following:

• 40Gbase-LR4 (single mode fiber) transceiver

• 40Gbase-SR4 (multi-mode fiber) transceiver

• QSFP28 DAC

RJ45

This port provides 1Gbase-TX/10Gbase-TX operation. It is connected via UTP cable terminated with an RJ45 connector. Compute-optimized devices have two RJ45 ports.

1G operation is indicated by a blinking amber light. 1G operation is not recommended for large-scale data transfers to the Snowball Edge device, as it dramatically increases the time it takes to transfer data.

10G operation is indicated by a blinking green light. It requires a Cat6A UTP cable with a maximum operating distance of 180 feet (55 meters).

(24)

Supported Network Hardware

(25)

Sign Up for AWS

Setting Up Your AWS Access for AWS Snowball Edge

Before you use AWS Snowball Edge for the first time, you need to complete the following tasks:

1.Sign Up for AWS (p. 18).

Note

In the Asia Pacific (Mumbai) AWS Region service is provided by Amaz on Internet Services Private Limited (AISPL). For information on signing up for Amazon Web Services in the Asia Pacific (Mumbai) AWS Region, see Signing Up for AISPL.

2.Create an IAM User (p. 18).

Sign Up for AWS

When you sign up for Amazon Web Services (AWS), your AWS account is automatically signed up for all services in AWS, including AWS Snow Family. You are charged only for the services that you use. For more information about pricing and fees, see AWS Snowball Edge Pricing. AWS Snowball Edge is not free to use. For more information on what AWS services are free, see AWS Free Usage Tier.

If you have an AWS account already, skip to the next task. If you don't have an AWS account, use the following procedure to create one.

To create an AWS account

1. Open https://portal.aws.amazon.com/billing/signup.

2. Follow the online instructions.

Part of the sign-up procedure involves receiving a phone call and entering a verification code on the phone keypad.

Note your AWS account number, because you'll need it for the next task.

Create an IAM User

Services in AWS, such as AWS Snowball Edge, require that you provide credentials when you access them, so that the service can determine whether you have permission to access its resources. AWS recommends not using the root credentials of your AWS account to make requests. Instead, create an AWS Identity and Access Management (IAM) user, and grant that user full access. We refer to these users as IAM users with administrator-level credentials.

You can use the administrator user credentials, instead of root credentials of your account, to interact with AWS and perform tasks, such as to create an Amazon S3 bucket, create users, and grant them permissions. For more information, see Root Account Credentials vs. IAM User Credentials in the AWS General Reference and IAM Best Practices in IAM User Guide.

If you signed up for AWS but have not created an IAM user for yourself, you can create one using the IAM console.

(26)

Create an IAM User

To create an administrator user for yourself and add the user to an administrators group (console)

1. Sign in to the IAM console as the account owner by choosing Root user and entering your AWS account email address. On the next page, enter your password.

Note

We strongly recommend that you adhere to the best practice of using the Administrator IAM user that follows and securely lock away the root user credentials. Sign in as the root user only to perform a few account and service management tasks.

2. In the navigation pane, choose Users and then choose Add user.

3. For User name, enter Administrator.

4. Select the check box next to AWS Management Console access. Then select Custom password, and then enter your new password in the text box.

5. (Optional) By default, AWS requires the new user to create a new password when first signing in. You can clear the check box next to User must create a new password at next sign-in to allow the new user to reset their password after they sign in.

6. Choose Next: Permissions.

7. Under Set permissions, choose Add user to group.

8. Choose Create group.

9. In the Create group dialog box, for Group name enter Administrators.

10. Choose Filter policies, and then select AWS managed - job function to filter the table contents.

11. In the policy list, select the check box for AdministratorAccess. Then choose Create group.

Note

You must activate IAM user and role access to Billing before you can use the

AdministratorAccess permissions to access the AWS Billing and Cost Management console. To do this, follow the instructions in step 1 of the tutorial about delegating access to the billing console.

12. Back in the list of groups, select the check box for your new group. Choose Refresh if necessary to see the group in the list.

13. Choose Next: Tags.

14. (Optional) Add metadata to the user by attaching tags as key-value pairs. For more information about using tags in IAM, see Tagging IAM entities in the IAM User Guide.

15. Choose Next: Review to see the list of group memberships to be added to the new user. When you are ready to proceed, choose Create user.

You can use this same process to create more groups and users and to give your users access to your AWS account resources. To learn about using policies that restrict user permissions to specific AWS resources, see Access management and Example policies.

To sign in as this new IAM user, sign out of the AWS Management Console, then use the following URL, where your_aws_account_id is your AWS account number without the hyphens (for example, if your AWS account number is 1234-5678-9012, your AWS account ID is 123456789012).

https://your_aws_account_id.signin.aws.amazon.com/console/

Type the IAM user name and password that you just created. When you're signed in, the navigation bar displays "your_user_name @ your_aws_account_id".

If you don't want the URL for your sign-in page to contain your AWS account ID, you can create an account alias. From the IAM dashboard, choose Create Account Alias and type an alias, such as your company name. To sign in after you create an account alias, use the following URL.

(27)

Next Step

https://your_account_alias.signin.aws.amazon.com/console/

To verify the sign-in link for IAM users for your account, open the IAM console and check under AWS account Alias on the dashboard.

If you're going to create AWS Snowball Edge jobs through an IAM user that is not an administrator user, that user needs certain permissions to use the AWS Snow Family Management Console effectively.

For more information on those permissions, see Permissions Required to Use the AWS Snowball Console (p. 211).

Next Step

Getting Started (p. 32)

(28)

About the Local Environment

Before You Order a Snowball Edge device

AWS Snowball Edge is a region-specific service. So before you plan your job, be sure that the service is available in your region. Ensure that your location and Amazon S3 bucket are within the same AWS Region or the same country because it will impact your ability to order the device.

As part of the order process, you create an AWS Identity and Access Management (IAM) role and an AWS Key Management Service (AWS KMS) key. The KMS key is used for encrypting the data during transit and at rest on the Snowball Edge device. For more information about creating IAM roles and KMS keys, see Creating an AWS Snowball Edge Job.

Topics

• Questions about the Local Environment (p. 21)

• Working with Files That Contain Special Characters (p. 22)

• Using Amazon EC2 on Snowball (p. 22)

• Using Amazon S3 on Snowball (p. 26)

• Snowball Edge Clusters (p. 30)

Questions about the Local Environment

Understanding your dataset and how the local environment is set up will help you complete your data transfer. Consider the following before placing your order.

What data are you transferring?

Transferring a large number of small files does not work well with AWS Snowball Edge. This is because Snowball Edge encrypts each individual object. Small files include files under 1 MB in size.

We recommend that you zip them up before transferring them onto the AWS Snowball Edge device.

We also recommend that you have no more than 500,000 files or directories within each directory.

Will the data be accessed during the transfer?

It is important to have a static dataset, (that is, no users or systems are accessing the data during transfer). If not, the file transfer can fail due to a checksum mismatch. The files won't be transferred and the files will be marked as Failed.

We recommend that if you are using the file interface, you only use one method of transferring data to the AWS Snowball Edge. Copying data with both the file interface and the Amazon S3 interface can result in read/write conflicts.

To prevent corrupting your data, don't disconnect an AWS Snowball Edge device or change its network settings while transferring data. Files should be in a static state while being written to the device. Files that are modified while they are being written to the device can result in read/write conflicts.

Will the network support AWS Snowball data transfer?

Snowball Edge supports the RJ45, SFP+, or QSFP+ networking adapters. Verify that your switch is a gigabit switch. Depending on the brand of switch, it might say gigabit or 10/100/1000. Snowball Edge devices do not support a megabit switch, or 10/100 switch.

(29)

Working with Special Characters

Working with Files That Contain Special Characters

It's important to note that if your objects contain special characters, you might encounter errors.

Although Amazon S3 allows special characters, we highly recommend that you avoid the following characters:

• Backslash ("\")

• Left curly brace ("{")

• Right curly brace ("}")

• Left square bracket ("[")

• Right square bracket ("]")

• 'Less Than' symbol ("<")

• 'Greater Than' symbol (">")

• Non-printable ASCII characters (128–255 decimal characters)

• Caret ("^")

• Percent character ("%")

• Grave accent / back tick ("`")

• Quotation marks

• Tilde ("~")

• 'Pound' character ("#")

• Vertical bar / pipe ("|")

If your files have one or more of these characters, rename them before you copy them to the AWS Snowball Edge device. Windows users who have spaces in their file names should be careful when copying individual objects or running a recursive command. Surround individual objects that have spacing in the name with quotation marks. The following are examples of such files.

Operating system File name: test file.txt

Windows “C:\Users\<username>\desktop\test file.txt”

iOS /Users/<username>/test\ file.txt

Linux /home/<username>/test\ file.txt

Note

The only object metadata that is transferred is the object name and size. If you want additional metadata to be copied, you can use the file interface or other tools to copy the data to Amazon S3.

Using Amazon EC2 on Snowball

This section provides an overview of using Amazon EC2 compute instances on an AWS Snowball Edge device. It includes conceptual information, procedures, and examples.

Note

These Amazon EC2 features on AWS Snowball are not supported in the Asia Pacific (Mumbai) and Europe (Paris) AWS Regions.

You can run Amazon EC2 compute instances hosted on an AWS Snowball Edge with the sbe1, sbe-c, and sbe-g instance types:

(30)

Using Compute Instances on Clusters

• The sbe1 instance type works on devices with the Snowball Edge Storage Optimized option.

• The sbe-c instance type works on devices with the Snowball Edge Compute Optimized option.

• Both the sbe-c and sbe-g instance types work on devices with the Snowball Edge Compute Optimized with GPU option.

All the compute instance types supported on Snowball Edge device options are unique to AWS Snowball Edge devices. Like their cloud-based counterparts, these instances require Amazon Machine Images (AMIs) to launch. You choose the AMI for an instance before you create your Snowball Edge job.

To use a compute instance on a Snowball Edge, create a job and specify your AMIs. You can do this using the AWS Snowball Management Console, the AWS Command Line Interface (AWS CLI), or one of the AWS SDKs. Typically, to use your instances, there are some housekeeping prerequisites that you must perform before creating your job.

After your device arrives, you can start managing your AMIs and instances. You can manage your compute instances on a Snowball Edge through an Amazon EC2-compatible endpoint. This type of endpoint supports many of the Amazon EC2 CLI commands and actions for the AWS SDKs. You can't use the AWS Management Console on the Snowball Edge to manage your AMIs and compute instances.

When you're done with your device, return it to AWS. If the device was used in an import job, the data transferred using the Amazon S3 interface or the file interface is imported into Amazon S3. Otherwise, we perform a complete erasure of the device when it is returned to AWS. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards.

Important

Data in compute instances running on a Snowball Edge isn't imported into AWS.

Using Compute Instances on Clusters

You can use compute instances on clusters of Snowball Edge devices. The procedures and guidance for doing so are the same as for using compute instances on a standalone device.

When you create a cluster job with AMIs, a copy of each AMI exists on each node in the cluster. You can have only 10 AMIs associated with a cluster of devices regardless of the number of nodes on the cluster.

When you launch an instance in a cluster, you declare the node to host the instance in your command and the instance runs on a single node.

Clusters must be either compute-optimized or storage-optimized. You can have a cluster of compute- optimized nodes, and some number of them can have GPUs. You can have a cluster made entirely of storage-optimized nodes. A cluster can't be made of a combination of compute-optimized nodes and storage-optimized nodes.

Pricing for Compute Instances on Snowball Edge

There are additional costs associated with using compute instances. For more information, see AWS Snowball Edge Pricing.

Prerequisites

Before creating your job, keep the following information in mind:

• Before you add any AMIs to your job request, make sure that you have created an AMI that is

supported in your AWS account. Currently, supported AMIs are based on the CentOS 7 (x86_64) - with Updates HVM and Ubuntu 16.04 LTS - Xenial (HVM) images. You can get these images from the AWS Marketplace website.

(31)

Creating a Linux AMI from an Instance

• All AMIs must be based on Amazon Elastic Block Store (Amazon EBS), with a single volume.

• If you are connecting to a compute instance running on a Snowball Edge, you must use Secure Shell (SSH). To do so, you first add the key pair. For more information, see Configuring an AMI to Use SSH to Connect to Compute Instances Launched on the Device (p. 135).

Creating a Linux AMI from an Instance

You can create an AMI using the AWS Management Console or the command line. Start with an existing AMI, launch an instance, customize it, create a new AMI from it, and finally, launch an instance of your new AMI.

To create an AMI from an instance using the console

1. Select an appropriate EBS-backed AMI as a starting point for your new AMI, and configure it as needed before launch. For more information, see Launching an instance using the Launch Instance Wizard in the Amazon EC2 User Guide for Linux Instances.

2. Choose Launch to launch an instance of the EBS-backed AMI that you selected. Accept the default values as you step through the wizard. For more information, see Launching an instance using the Launch Instance Wizard.

3. While the instance is running, connect to it. You can perform the following actions on your instance to customize it for your needs:

• Install software and applications.

• Copy data.

• Reduce start time by deleting temporary files, defragmenting your hard drive, and zeroing out free space.

• Attach additional Amazon EBS volumes.

4. (Optional) Create snapshots of all the volumes attached to your instance. For more information about creating snapshots, see Creating Amazon EBS snapshots in the Amazon EC2 User Guide for Linux Instances.

5. In the navigation pane, choose Instances, and choose your instance. Choose Actions, choose Image, and then choose Create image.

Tip

If this option isn't available, your instance isn't an Amazon EBS-backed instance.

6. In the Create Image dialog box, specify the following information, and then choose Create image.

Image name - A unique name for the image.

Image description - An optional description of the image, up to 255 characters.

No reboot - This option is not selected by default. Amazon EC2 shuts down the instance, takes snapshots of any attached volumes, creates and registers the AMI, and then reboots the instance.

Select No reboot to avoid having your instance shut down.

Warning

If you select No reboot, we can't guarantee the file system integrity of the created image.

Instance Volumes - The fields in this section enable you to modify the root volume, and add more Amazon EBS and instance store volumes. For information about each field, pause on the i icon next to each field to display field tooltips. Some important points are listed following:

• To change the size of the root volume, locate Root in the Volume Type column. For Size (GiB), enter the required value.

• If you select Delete on Termination, when you terminate the instance created from this AMI, the Amazon EBS volume is deleted. If you clear Delete on Termination, when you terminate the instance, the Amazon EBS volume is not deleted. For more information, see Preserving Amazon EBS volumes on instance termination in the Amazon EC2 User Guide for Linux Instances.

(32)

Creating a Linux AMI from a Snapshot

• To add an Amazon EBS volume, choose Add New Volume (which adds a new row). For Volume Type, choose EBS, and fill in the fields in the row. When you launch an instance from your new AMI, additional volumes are automatically attached to the instance. Empty volumes must be formatted and mounted. Volumes based on a snapshot must be mounted.

• To add an instance store volume, see Adding instance store volumes to an AMI in the Amazon EC2 User Guide for Linux Instances. When you launch an instance from your new AMI, additional volumes are automatically initialized and mounted. These volumes don't contain data from the instance store volumes of the running instance on which you based your AMI.

7. To view the status of your AMI while it is being created, in the navigation pane, choose AMIs.

Initially, the status is pending but should change to available after a few minutes.

(Optional) To view the snapshot that was created for the new AMI, choose Snapshots. When you launch an instance from this AMI, we use this snapshot to create its root device volume.

8. Launch an instance from your new AMI. For more information, see Launching an instance using the Launch Instance Wizard in the Amazon EC2 User Guide for Linux Instances.

9. The new running instance contains all of the customizations that you applied in previous steps.

To Create an AMI from an Instance Using the Command Line

You can use one of the following commands. For more information about these command line interfaces, see Accessing Amazon EC2 in the Amazon EC2 User Guide for Linux Instances.

• create-image (AWS CLI)

• New-EC2Image (AWS Tools for Windows PowerShell)

Creating a Linux AMI from a Snapshot

If you have a snapshot of the root device volume of an instance, you can create an AMI from this snapshot using the AWS Management Console or the command line.

To create an AMI from a snapshot using the console

1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

2. In the navigation pane, under Elastic Block Store, choose Snapshots.

3. Choose the snapshot, choose Actions, and then choose Create image.

4. In the Create image from EBS snapshot dialog box, complete the fields to create your AMI. Then choose Create. If you're re-creating a parent instance, choose the same options as the parent instance.

Architecture – Choose i386 for 32-bit or x86_64 for 64-bit.

Root device name – Enter the appropriate name for the root volume. For more information, see Device naming on Linux instances in the Amazon EC2 User Guide for Linux Instances.

Virtualization type – Choose whether instances launched from this AMI use paravirtual (PV) or hardware virtual machine (HVM) virtualization. For more information, see Linux AMI virtualization types.

• (PV virtualization type only) Kernel ID and RAM disk ID – Choose the AKI and ARI from the lists.

If you choose the default AKI, or you don't choose an AKI, you must specify an AKI every time you launch an instance using this AMI. In addition, your instance might fail the health checks if the default AKI is incompatible with the instance.

• (Optional) Block Device Mappings – Add volumes or expand the default size of the root volume for the AMI. For more information about resizing the file system on your instance for a larger

(33)

Using Amazon S3

volume, see Extending a Linux File system after resizing a volume in the Amazon EC2 User Guide for Linux Instances.

To Create an AMI from a Snapshot Using the Command Line

To create an AMI from a snapshot, you can use one of the following commands. For more information about these command line interfaces, see Accessing Amazon EC2 in the Amazon EC2 User Guide for Linux Instances.

• register-image (AWS CLI)

• Register-EC2Image (AWS Tools for Windows PowerShell)

Using Amazon S3 on Snowball

As part of the order process, you are asked to create an AWS Identity and Access Management (IAM) role and AWS Key Management Service (AWS KMS) key. The KMS key is used for encrypting the data during transit and at rest on the Snowball Edge device. For more information about creating IAM roles and KMS keys, see Creating an AWSAWS Snowball Edge Job.

Important

If the imported data must be encrypted in the S3 bucket using Server-Side Encryption with keys stored in AWS KMS (SSE-KMS), see Amazon S3 Encryption with AWS KMS (p. 27).

If the imported data must be encrypted in the S3 bucket using Server-Side Encryption with Amazon S3 managed keys (SSE-S3), see Amazon S3 Encryption with Server-Side Encryption (p. 29).

How Import Works

Each import job uses a single Snowball Edge device. After you create a job, we ship a Snowball Edge device to you. When it arrives, you connect the Snowball Edge device to your network and transfer the data that you want to import to Amazon S3 onto that Snowball Edge. When you’re done transferring data, ship the Snowball Edge back to AWS. We then import your data into Amazon S3.

Important

Snowball Edge cannot write to buckets if you have turned on S3 Object Lock. We also cannot write to your bucket if IAM policies on the bucket prevent writing to the bucket.

How Export Works

Each export job can use any number of AWS Snowball Edge devices. After you create a job, a listing operation starts in Amazon S3. This listing operation splits your job into parts. Each job part has exactly one device associated with it. After your job parts are created, your first job part enters the Preparing Snowball status.

Note

The listing operation to split your job into parts is a function of Amazon S3, and you are billed the same as Amazon S3 operation.

We then start exporting your data onto a device. Typically, exporting data takes one business day.

However, this process can take longer. When the export is done, AWS gets the device ready for your regional carrier to pick up.

When the device arrives at your site, you connect it to your network and transfer the data that you want to import into Amazon S3 onto the device. When you’re done transferring the data, ship the device back to AWS. When we receive the returned device, we erase it completely. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards.

(34)

Amazon S3 Encryption with AWS KMS

This step marks the completion of that particular job part. If there are more job parts, the next job part now is prepared for shipping.

Important

Snowball Edge is unable to export files that are in S3 Glacier storage class. These objects must be restored before we can export the files. If we encounter files in S3 Glacier storage class, we contact you to let you know, but this might add delays to your export job.

Amazon S3 Encryption with AWS KMS

You can use the default AWS managed or customer managed encryption keys to protect your data when importing or exporting data.

Using Amazon S3 Default Bucket Encryption with AWS KMS Managed Keys

To enable AWS managed encryption with AWS KMS

1. Open the Amazon S3 console at https://console.aws.amazon.com/s3/.

2. Choose the Amazon S3 bucket that you want to encrypt.

3. In the wizard that appears on the right side, choose Properties.

4. In the Default encryption box, choose Disabled (this option is grayed out) to enable default encryption.

5. Choose AWS-KMS as the encryption method, and then choose the KMS key that you want to use.

This key is used to encrypt objects that are PUT into the bucket.

6. Choose Save.

After the Snowball Edge job is created, and before the data is imported, add a statement to the existing IAM role policy. This is the role you created during the ordering process. Depending on the job type, the default role name looks similar to Snowball-import-s3-only-role or Snowball-export-s3- only-role.

The following are examples of such a statement.

For importing data

If you use server-side encryption with AWS KMS managed keys (SSE-KMS) to encrypt the Amazon S3 buckets associated with your import job, you also need to add the following statement to your IAM role.

Example Example: Snowball import IAM role

{ "Effect": "Allow", "Action": [

"kms: GenerateDataKey", "kms: Decrypt"

],

"Resource":"arn:aws:kms:us-west-2:123456789012:key/abc123a1-abcd-1234- efgh-111111111111"

}

For exporting data

If you use server-side encryption with AWS KMS managed keys to encrypt the Amazon S3 buckets associated with your export job, you also must add the following statement to your IAM role.

參考文獻

相關文件

 Calculating the expected total edge number for one left path started at one problem with m’ edges.  Evaluating the total edge number for all right sub-problems #

 Following these simple rules will ensure you gain the confidence and respect of your trip ( including host family) , and help to ensure a pleasant and rewarding experience...

2.1.1 The pre-primary educator must have specialised knowledge about the characteristics of child development before they can be responsive to the needs of children, set

Understanding and inferring information, ideas, feelings and opinions in a range of texts with some degree of complexity, using and integrating a small range of reading

Writing texts to convey information, ideas, personal experiences and opinions on familiar topics with elaboration. Writing texts to convey information, ideas, personal

 Promote project learning, mathematical modeling, and problem-based learning to strengthen the ability to integrate and apply knowledge and skills, and make. calculated

Now, nearly all of the current flows through wire S since it has a much lower resistance than the light bulb. The light bulb does not glow because the current flowing through it

During early childhood, developing proficiency in the mother-tongue is of primary importance. Cantonese is most Hong Kong children’s mother-tongue and should also be the medium