AWS SDK for Go (version 1)
Developer Guide
AWS SDK for Go (version 1): Developer Guide
Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.
Table of Contents
What is the AWS SDK for Go ... 1
Using the AWS SDK for Go with AWS Cloud9 ... 1
More Info ... 1
Maintenance and support for SDK major versions ... 1
Maintenance and support for SDK major versions ... 1
Getting Started ... 3
Get an Amazon Account ... 3
Install the AWS SDK for Go ... 3
Get your AWS access keys ... 3
To get your access key ID and secret access key ... 3
Import Packages ... 4
Configuring the SDK ... 5
Creating a Session ... 5
Specifying the AWS Region ... 5
Specifying Credentials ... 6
IAM Roles for Tasks ... 7
IAM Roles for Amazon EC2 Instances ... 7
Shared Credentials File ... 7
Environment Variables ... 8
Hard-Coded Credentials in an Application (Not Recommended) ... 9
Other Credentials Providers ... 9
Configuring a Proxy ... 9
Logging Service Calls ... 9
Creating a Custom Endpoint ... 10
Custom HTTP Client ... 10
Dialer.KeepAlive ... 10
Dialer.Timeout ... 10
Transport.ExpectContinueTimeout ... 10
Transport.IdleConnTimeout ... 11
Transport.MaxIdleConns ... 11
Transport.MaxIdleConnsPerHost ... 11
Transport.ResponseHeaderTimeout ... 11
Transport.TLSHandshakeTimeout ... 12
Create Import Statement ... 12
Creating a Timeout Struct ... 12
Creating a Function to Create a Custom HTTP Client ... 13
Using a Custom HTTP Client ... 13
Using Cloud9 with the SDK ... 15
Step 1: Set up Your AWS Account to Use AWS Cloud9 ... 15
Step 2: Set up Your AWS Cloud9 Development Environment ... 15
Step 3: Set up the AWS SDK for Go ... 15
Step 4: Download Example Code ... 16
Step 5: Run Example Code ... 17
Using Sessions ... 18
Concurrency ... 18
Sessions with a Shared Configuration File ... 18
Creating Sessions ... 18
Create Sessions with Option Overrides ... 19
Deprecated New ... 19
Shared Configuration Fields ... 20
Environment Variables ... 8
Adding Request Handlers ... 20
Copying a Session ... 20
Using AWS Services ... 21
Constructing a Service ... 21
Tagging Service Resources ... 21
Getting the HTTP Request and Response with Each Service Call ... 23
Service Operation Calls ... 24
Calling Operations ... 24
Calling Operations with the Request Form ... 25
Handling Operation Response Body ... 25
Concurrently Using Service Clients ... 26
Using Pagination Methods ... 26
Using Waiters ... 27
Handling Errors ... 29
Handling Specific Service Error Codes ... 29
Additional Error Information ... 30
Specific Error Interfaces ... 30
Code Examples ... 31
SDK Request Examples ... 31
Using context.Context with SDK Requests ... 31
Using API Field Setters with SDK Requests ... 32
AWS CloudTrail Examples ... 32
Listing the CloudTrail Trails ... 33
Creating a CloudTrail Trail ... 33
Listing CloudTrail Trail Events ... 36
Deleting a CloudTrail Trail ... 37
Amazon CloudWatch Examples ... 38
Describing CloudWatch Alarms ... 38
Using Alarms and Alarm Actions in CloudWatch ... 39
Getting Metrics from CloudWatch ... 42
Sending Events to Amazon CloudWatch Events ... 45
Getting Log Events from CloudWatch ... 49
AWS CodeBuild Examples ... 50
Getting Information about All AWS CodeBuild Projects ... 50
Building an AWS CodeBuild Project ... 51
Listing Your AWS CodeBuild Project Builds ... 52
Amazon DynamoDB Examples ... 53
Listing all Amazon DynamoDB Tables ... 53
Creating an Amazon DynamoDB Table ... 55
Creating an Amazon DynamoDB Table Item ... 56
Creating Amazon DynamoDB Table Items from a JSON File ... 57
Reading an Amazon DynamoDB Table Item ... 59
Getting Amazon DynamoDB Table Items Using Expression Builder ... 60
Updating an Amazon DynamoDB Table Item ... 62
Deleting an Amazon DynamoDB Table Item ... 63
Amazon EC2 Examples ... 64
Creating Amazon EC2 Instances with Tags or without Block Devices ... 65
Managing Amazon EC2 Instances ... 67
Working with Amazon EC2 Key Pairs ... 72
Using Regions and Availability Zones with Amazon EC2 ... 76
Working with Security Groups in Amazon EC2 ... 77
Using Elastic IP Addresses in Amazon EC2 ... 82
Amazon Glacier Examples ... 86
The Scenario ... 87
Prerequisites ... 87
Create a Vault ... 87
Upload an Archive ... 88
IAM Examples ... 88
Managing IAM Users ... 89
Managing IAM Access Keys ... 96
Managing IAM Account Aliases ... 100
Working with IAM Policies ... 103
Working with IAM Server Certificates ... 108
AWS KMS Examples ... 112
Creating a CMK in AWS Key Management Service ... 112
Encrypting Data with AWS Key Management Service ... 113
Decrypting a Data Blob in AWS Key Management Service ... 114
Re-encrypting a Data Blob in AWS Key Management Service ... 114
AWS Lambda Examples ... 115
Displaying Information about All Lambda Functions ... 116
Creating a Lambda Function ... 116
Running a Lambda Function ... 118
Configuring a Lambda Function to Receive Notifications ... 120
Amazon Polly Examples ... 121
Getting a List of Voices ... 121
Getting a List of Lexicons ... 122
Synthesizing Speech ... 122
Amazon S3 Examples ... 124
Performing Basic Amazon S3 Bucket Operations ... 124
Creating Pre-Signed URLs for Amazon S3 Buckets ... 136
Using an Amazon S3 Bucket as a Static Web Host ... 139
Working with Amazon S3 CORS Permissions ... 142
Working with Amazon S3 Bucket Policies ... 144
Working with Amazon S3 Bucket ACLs ... 147
Encrypting Amazon S3 Bucket Items ... 155
Amazon SES Examples ... 159
Listing Valid Amazon SES Email Addresses ... 159
Verifying an Email Address in Amazon SES ... 160
Sending a Message to an Email Address in Amazon SES ... 161
Deleting an Email Address in Amazon SES ... 163
Getting Amazon SES Statistics ... 164
Amazon SNS Examples ... 165
Listing Your Amazon SNS Topics ... 165
Creating an Amazon SNS Topic ... 166
List Your Amazon SNS Subscriptions ... 166
Subscribe to an Amazon SNS Topic ... 167
Sending a Message to All Amazon SNS Topic Subscribers ... 168
Amazon SQS Examples ... 169
Using Amazon SQS Queues ... 169
Sending and Receiving Messages in Amazon SQS ... 173
Managing Visibility Timeout in Amazon SQS Queues ... 176
Enabling Long Polling in Amazon SQS Queues ... 178
Using Dead Letter Queues in Amazon SQS ... 182
Amazon WorkDocs Examples ... 184
Listing Users ... 184
Listing User Docs ... 186
SDK Utilities ... 188
Amazon CloudFront URL Signer ... 188
Amazon DynamoDB Attributes Converter ... 188
Amazon Elastic Compute Cloud Metadata ... 189
Retrieving an Instance’s Region ... 189
Amazon S3 Transfer Managers ... 190
Upload Manager ... 190
Download Manager ... 193
Security ... 196
Data Protection ... 196
Identity and Access Management ... 197
Compliance Validation ... 197
Resilience ... 198
Infrastructure Security ... 198
Enforcing TLS 1.2 ... 199
How do I set my TLS version? ... 199
S3 Encryption Client Migration ... 200
Migration Overview ... 200
Update Existing Clients to Read New Formats ... 200
Migrate Encryption and Decryption Clients to V2 ... 201
Document History ... 206
Using the AWS SDK for Go with AWS Cloud9
What is the AWS SDK for Go
The AWS SDK for Go provides APIs and utilities that developers can use to build Go applications that use AWS services, such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).
Note
This document is for version 1 of the AWS SDK for Go. If you're looking for version 2 of the SDK, see the version 2 developer guide and the version 2 SDK API reference.The SDK removes the complexity of coding directly against a web service interface. It hides a lot of the lower-level plumbing, such as authentication, request retries, and error handling.
The SDK also includes helpful utilities. For example, the Amazon S3 download and upload manager can automatically break up large objects into multiple parts and transfer them in parallel.
Use the AWS SDK for Go Developer Guide to help you install, configure, and use the SDK. The guide provides configuration information, sample code, and an introduction to the SDK utilities.
Using the AWS SDK for Go with AWS Cloud9
AWS Cloud9 is a web-based integrated development environment (IDE) that contains a collection of tools that you use to code, build, run, test, debug, and release software in the cloud.
See Using AWS Cloud9 with the AWS SDK for Go (p. 15) for information on using AWS Cloud9 with the AWS SDK for Go.
More Info
• To learn about everything you need before you can start using the AWS SDK for Go, see Getting Started with the AWS SDK for Go (p. 3).
• For code examples, see AWS SDK for Go Code Examples (p. 31).
• You can browse the AWS SDK for Go examples in the aws-doc-sdk-examples repo on GitHub.
• To learn about the SDK utilities, see Using the AWS SDK for Go Utilities (p. 188).
• For learn about the types and functionality that the library provides, see the AWS SDK for Go API Reference.
• To view a video introduction of the SDK and a sample application demonstration, see AWS SDK for Go:
Gophers Get Going with AWS from AWS re:Invent 2015.
Maintenance and support for SDK major versions
Maintenance and support for SDK major versions
For information about maintenance and support for SDK major versions and their underlying dependencies, see the following in the AWS SDKs and Tools Reference Guide:
• AWS SDKs and tools maintenance policy
Maintenance and support for SDK major versions
• AWS SDKs and tools version support matrix
Get an Amazon Account
Getting Started with the AWS SDK for Go
The AWS SDK for Go requires Go 1.5 or later. You can view your current version of Go by running the go version command. For information about installing or upgrading your version of Go, see https://
golang.org/doc/install.
Get an Amazon Account
Before you can use the AWS SDK for Go, you must have an Amazon account. See How do I create and activate a new Amazon Web Services account? for details.
Install the AWS SDK for Go
To install the SDK and its dependencies, run the following Go command.
go get -u github.com/aws/aws-sdk-go/...
If you set the Go vendor experiment environment variable to 1, you can use the following command to get the SDK. The SDK's runtime dependencies are vendored in the vendor/ folder.
go get -u github.com/aws/aws-sdk-go
Get your AWS access keys
Access keys consist of an access key ID and secret access key, which are used to sign programmatic requests that you make to AWS. If you don't have access keys, you can create them by using the AWS Management Console. We recommend that you use IAM access keys instead of AWS root account access keys. IAM lets you securely control access to AWS services and resources in your AWS account.
Note
To create access keys, you must have permissions to perform the required IAM actions. For more information, see Granting IAM User Permission to Manage Password Policy and Credentials in the IAM User Guide.To get your access key ID and secret access key
1. Open the IAM console.
2. On the navigation menu, choose Users.
3. Choose your IAM user name (not the check box).
4. Open the Security credentials tab, and then choose Create access key.
5. To see the new access key, choose Show. Your credentials resemble the following:
• Access key ID: AKIAIOSFODNN7EXAMPLE
Import Packages
• Secret access key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY 6. To download the key pair, choose Download .csv file. Store the keys
in a secure location.
Important
Keep the keys confidential to protect your AWS account, and never email them. Do not share them outside your organization, even if an inquiry appears to come from AWS or Amazon.com.
No one who legitimately represents Amazon will ever ask you for your secret key.
Related topics
• What Is IAM? in IAM User Guide.
• AWS Security Credentials in Amazon Web Services General Reference.
Import Packages
After you have installed the SDK, you import AWS packages into your Go applications to use the SDK, as shown in the following example, which imports the AWS, Session, and Amazon S3 libraries:
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
Creating a Session
Configuring the AWS SDK for Go
In the AWS SDK for Go, you can configure settings for service clients, such as the log level and maximum number of retries. Most settings are optional. However, for each service client, you must specify an AWS Region and your credentials. The SDK uses these values to send requests to the correct Region and sign requests with the correct credentials. You can specify these values as part of a session or as environment variables.
Creating a Session
Before you can create a service client you must create a session, which is part of the github.com/aws/
aws-sdk-go/aws/session package.
There are a number of ways of configuring a session but the following are the most common.
Create a session using the default Region and credentials:
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
) // ...
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable, }))
Create a session in us-west-2:
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
) // ...
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2")}, )
See session for additional information.
Specifying the AWS Region
When you specify the Region, you specify where to send requests, such as us-west-2 or us-east-2.
For a list of Regions for each service, see Service endpoints and quotas in the Amazon Web Services General Reference.
The SDK does not have a default Region. To specify a Region:
• Set the AWS_REGION environment variable to the default Region
Specifying Credentials
• Set the AWS_SDK_LOAD_CONFIG environment variable to true to get the Region value from the config file in the .aws/ folder in your home directory
• Set the NewSessionWithOptions method argument SharedConfigState to SharedConfigEnable when you create a session to get the Region value from the config file in the .aws/ folder in your home directory
• Set the Region explicitly when you create a session
If you set a Region using all of these techniques, the SDK uses the Region you explicitly specified in the session.
The following examples show you how to configure the environment variable.
Linux, OS X, or Unix
$ export AWS_REGION=us-west-2
Windows
> set AWS_REGION=us-west-2
The following snippet specifies the Region in a session:
sess, err := session.NewSession(&aws.Config{Region: aws.String("us-west-2")})
Specifying Credentials
The AWS SDK for Go requires credentials (an access key and secret access key) to sign requests to AWS.
You can specify your credentials in several different locations, depending on your particular use case. For information about obtaining credentials, see Setting Up (p. 3).
When you initialize a new service client without providing any credential arguments, the SDK uses the default credential provider chain to find AWS credentials. The SDK uses the first provider in the chain that returns credentials without an error. The default provider chain looks for credentials in the following order:
1. Environment variables.
2. Shared credentials file.
3. If your application uses an ECS task definition or RunTask API operation, IAM role for tasks.
4. If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.
The SDK detects and uses the built-in providers automatically, without requiring manual configurations.
For example, if you use IAM roles for Amazon EC2 instances, your applications automatically use the instance’s credentials. You don’t need to manually configure credentials in your application.
As a best practice, AWS recommends that you specify credentials in the following order:
1. Use IAM roles for tasks if your application uses an ECS task definition or RunTask API operation.
2. Use IAM roles for Amazon EC2 (if your application is running on an Amazon EC2 instance).
IAM roles provide applications on the instance temporary security credentials to make AWS calls. IAM roles provide an easy way to distribute and manage credentials on multiple Amazon EC2 instances.
IAM Roles for Tasks
3. Use a shared credentials file.
This credentials file is the same one used by other SDKs and the AWS CLI. If you’re already using a shared credentials file, you can also use it for this purpose.
4. Use environment variables.
Setting environment variables is useful if you’re doing development work on a machine other than an Amazon EC2 instance.
IAM Roles for Tasks
If your application uses an Amazon ECS task definition or RunTask operation, use IAM Roles for Tasks to specify an IAM role that can be used by the containers in a task.
IAM Roles for Amazon EC2 Instances
If you are running your application on an Amazon EC2 instance, use the instance’s IAM role to get temporary security credentials to make calls to AWS.
If you have configured your instance to use IAM roles, the SDK uses these credentials for your application automatically. You don’t need to manually specify these credentials.
Shared Credentials File
A credential file is a plaintext file that contains your access keys. The file must be on the same machine on which you’re running your application. The file must be named credentials and located in the .aws/ folder in your home directory. The home directory can vary by operating system. In Windows, you can refer to your home directory by using the environment variable %UserProfile%. In Unix-like systems, you can use the environment variable $HOME or ~ (tilde).
If you already use this file for other SDKs and tools (like the AWS CLI), you don’t need to change anything to use the files in this SDK. If you use different credentials for different tools or applications, you can use profiles to configure multiple access keys in the same configuration file.
Creating the Credentials File
If you don’t have a shared credentials file (.aws/credentials), you can use any text editor to create one in your home directory. Add the following content to your credentials file, replacing
<YOUR_ACCESS_KEY_ID> and <YOUR_SECRET_ACCESS_KEY> with your credentials.
[default]
aws_access_key_id = <YOUR_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_SECRET_ACCESS_KEY>
The [default] heading defines credentials for the default profile, which the SDK will use unless you configure it to use another profile.
You can also use temporary security credentials by adding the session tokens to your profile, as shown in the following example:
[temp]
aws_access_key_id = <YOUR_TEMP_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_TEMP_SECRET_ACCESS_KEY>
aws_session_token = <YOUR_SESSION_TOKEN>
Environment Variables
Specifying Profiles
You can include multiple access keys in the same configuration file by associating each set of access keys with a profile. For example, in your credentials file, you can declare multiple profiles, as follows.
[default]
aws_access_key_id = <YOUR_DEFAULT_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_DEFAULT_SECRET_ACCESS_KEY>
[test-account]
aws_access_key_id = <YOUR_TEST_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_TEST_SECRET_ACCESS_KEY>
[prod-account]
; work profile
aws_access_key_id = <YOUR_PROD_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_PROD_SECRET_ACCESS_KEY>
By default, the SDK checks the AWS_PROFILE environment variable to determine which profile to use. If no AWS_PROFILE variable is set, the SDK uses the default profile.
If you have an application named myapp that uses the SDK, you can run it with the test credentials by setting the variable to test-account myapp, as shown in the following command.
$ AWS_PROFILE=test-account myapp
You can also use the SDK to select a profile by specifying os.Setenv("AWS_PROFILE", test- account) before constructing any service clients or by manually setting the credential provider, as shown in the following example.
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2"),
Credentials: credentials.NewSharedCredentials("", "test-account"), })
In addition, checking if your credentials have been found is fairly easy.
_, err := sess.Config.Credentials.Get()
If ChainProvider is being used, set CredentialsChainVerboseErrors to true in the session config.
Note
If you specify credentials in environment variables, the SDK will always use those credentials, no matter which profile you specify.Environment Variables
By default, the SDK detects AWS credentials set in your environment and uses them to sign requests to AWS. That way you don’t need to manage credentials in your applications.
The SDK looks for credentials in the following environment variables:
• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN (optional)
Hard-Coded Credentials in an Application (Not Recommended)
The following examples show how you configure the environment variables.
Linux, OS X, or Unix
$ export AWS_ACCESS_KEY_ID=YOUR_AKID
$ export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY
$ export AWS_SESSION_TOKEN=TOKEN
Windows
> set AWS_ACCESS_KEY_ID=YOUR_AKID
> set AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY
> set AWS_SESSION_TOKEN=TOKEN
Hard-Coded Credentials in an Application (Not Recommended)
Warning
Do not embed credentials inside an application. Use this method only for testing purposes.
You can hard-code credentials in your application by passing the access keys to a configuration instance, as shown in the following snippet.
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2"),
Credentials: credentials.NewStaticCredentials("AKID", "SECRET_KEY", "TOKEN"), })
Other Credentials Providers
The SDK provides other methods for retrieving credentials in the aws/credentials package. For example, you can retrieve temporary security credentials from AWS Security Token Service or credentials from encrypted storage. For more information, see Credentials.
Configuring a Proxy
If you cannot directly connect to the internet, you can use Go-supported environment variables
(HTTP_PROXY) or create a custom HTTP client to configure your proxy. Use the Config.HTTPClient struct to specify a custom HTTP client. For more information about how to create an HTTP client to use a proxy, see the Transport struct in the Go http package.
Logging Service Calls
You can enable logging in a client by setting the LogLevel in a configuration instance, as shown in the following snippet, which sets the log level to LogDebugWithHTTPBody for a new DynamoDB client.
svc := dynamodb.New(sess, aws.NewConfig().WithLogLevel(aws.LogDebugWithHTTPBody))
See LogLevelType for the different log level values.
Creating a Custom Endpoint
Creating a Custom Endpoint
In most cases you use the endpoint that is pre-configured for a service. However, you can specify a custom endpoint, such as for pre-release versions of the service, as shown in the following snippet, which sets the Endpoint to https://test.us-west-2.amazonaws.com for a new DynamoDB client.
svc := dynamodb.New(sess, &aws.Config{Endpoint: aws.String("https://test.us- west-2.amazonaws.com")})
See aws.Config for details.
Creating a Custom HTTP Client
The AWS SDK for Go uses a default HTTP client with default configuration values. Although you can change some of these configuration values, the default HTTP client and transport are not sufficiently configurable for customers using the AWS SDK for Go in an environment with high throughput and low latency requirements. This section describes how to create a custom HTTP client, and use that client to create AWS SDK for Go calls.
To assist you in creating a custom HTTP client, this section describes how to create a structure to encapsulate the custom settings, create a function to create a custom HTTP client based on those settings, and use that custom HTTP client to call an AWS SDK for Go service client.
Let’s define what we want to customize.
Dialer.KeepAlive
This setting represents the keep-alive period for an active network connection.
Set to a negative value to disable keep-alives.
Set to 0 to enable keep-alives if supported by the protocol and operating system.
Network protocols or operating systems that do not support keep-alives ignore this field. By default, TCP enables keep alive.
See https://golang.org/pkg/net/#Dialer.KeepAlive We’ll call this ConnKeepAlive as time.Duration.
Dialer.Timeout
This setting represents the maximum amount of time a dial to wait for a connection to be created.
Default is 30 seconds.
See https://golang.org/pkg/net/#Dialer.Timeout We’ll call this Connect as time.Duration.
Transport.ExpectContinueTimeout
This setting represents the maximum amount of time to wait for a server’s first response headers after fully writing the request headers, if the request has an “Expect: 100-continue” header. This time does
Transport.IdleConnTimeout
not include the time to send the request header. The HTTP client sends its payload after this timeout is exhausted.
Default 1 second.
Set to 0 for no timeout and send request payload without waiting. One use case is when you run into issues with proxies or third party services that take a session similar to the use of Amazon S3 in the function shown later.
See https://golang.org/pkg/net/http/#Transport.ExpectContinueTimeout We’ll call this ExpectContinue as time.Duration.
Transport.IdleConnTimeout
This setting represents the maximum amount of time to keep an idle network connection alive between HTTP requests.
Set to 0 for no limit.
See https://golang.org/pkg/net/http/#Transport.IdleConnTimeout We’ll call this IdleConn as time.Duration.
Transport.MaxIdleConns
This setting represents the maximum number of idle (keep-alive) connections across all hosts. One use case for increasing this value is when you are seeing many connections in a short period from the same clients
0 means no limit.
See https://golang.org/pkg/net/http/#Transport.MaxIdleConns We’ll call this MaxAllIdleConns as int.
Transport.MaxIdleConnsPerHost
This setting represents the maximum number of idle (keep-alive) connections to keep per-host. One use case for increasing this value is when you are seeing many connections in a short period from the same clients
Default is two idle connections per host.
Set to 0 to use DefaultMaxIdleConnsPerHost (2).
See https://golang.org/pkg/net/http/#Transport.MaxIdleConnsPerHost We’ll call this MaxHostIdleConns as int.
Transport.ResponseHeaderTimeout
This setting represents the maximum amount of time to wait for a client to read the response header.
If the client isn’t able to read the response’s header within this duration, the request fails with a timeout error.
Transport.TLSHandshakeTimeout
Be careful setting this value when using long-running Lambda functions, as the operation does not return any response headers until the Lambda function has finished or timed out. However, you can still use this option with the InvokeAsync API operation.
Default is no timeout; wait forever.
See https://golang.org/pkg/net/http/#Transport.ResponseHeaderTimeout We’ll call this ResponseHeader as time.Duration.
Transport.TLSHandshakeTimeout
This setting represents the maximum amount of time waiting for a TLS handshake to be completed.
Default is 10 seconds.
Zero means no timeout.
See https://golang.org/pkg/net/http/#Transport.TLSHandshakeTimeout We’ll call this TLSHandshake as time.Duration.
Create Import Statement
The complete example imports the following Go packages.
import ( "bytes"
"context"
"flag"
"fmt"
"io"
"net"
"net/http"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"golang.org/x/net/http2"
)
Creating a Timeout Struct
Let’s create a struct to hold the timeout values we want to be able to set on our HTTP client.
type HTTPClientSettings struct { Connect time.Duration ConnKeepAlive time.Duration ExpectContinue time.Duration IdleConn time.Duration MaxAllIdleConns int
MaxHostIdleConns int
ResponseHeader time.Duration TLSHandshake time.Duration }
Creating a Function to Create a Custom HTTP Client
Creating a Function to Create a Custom HTTP Client
Next let’s create a function that takes a ClientTimeout struct and creates a custom HTTP client based on those timeout values.
func NewHTTPClientWithSettings(httpSettings HTTPClientSettings) (*http.Client, error) { var client http.Client
tr := &http.Transport{
ResponseHeaderTimeout: httpSettings.ResponseHeader, Proxy: http.ProxyFromEnvironment, DialContext: (&net.Dialer{
KeepAlive: httpSettings.ConnKeepAlive, DualStack: true,
Timeout: httpSettings.Connect, }).DialContext,
MaxIdleConns: httpSettings.MaxAllIdleConns, IdleConnTimeout: httpSettings.IdleConn, TLSHandshakeTimeout: httpSettings.TLSHandshake, MaxIdleConnsPerHost: httpSettings.MaxHostIdleConns, ExpectContinueTimeout: httpSettings.ExpectContinue, }
// So client makes HTTP/2 requests err := http2.ConfigureTransport(tr) if err != nil {
return &client, err }
return &http.Client{
Transport: tr, }, nil
}
Using a Custom HTTP Client
Let’s create a custom HTTP client and use it to create an Amazon S3 client.
The following example creates an http.Client that is configured to have:
• a five second TCP connection timeout
• a five second TLS handshake timeout
• a five second wait for the HTTP response headers
httpClient, err := NewHTTPClientWithSettings(HTTPClientSettings{
Connect: 5 * time.Second, ExpectContinue: 1 * time.Second, IdleConn: 90 * time.Second, ConnKeepAlive: 30 * time.Second, MaxAllIdleConns: 100,
MaxHostIdleConns: 10,
ResponseHeader: 5 * time.Second, TLSHandshake: 5 * time.Second, })
if err != nil {
fmt.Println("Got an error creating custom HTTP client:") fmt.Println(err)
return }
Using a Custom HTTP Client
sess := session.Must(session.NewSession(&aws.Config{
HTTPClient: httpClient, }))
svc := s3.New(sess)
All of these settings give the client approximately 15 seconds create a connection, do a TLS handshake, and receive the response headers from the service. The time that the client takes to read the response body is not covered by these timeouts. To specify a total timeout for the request to include reading the response body, use the AWS SDK for Go client’s WithContext API operation methods, such as the Amazon S3 operation PutObjectWithContext with a context.Withtimeout.
The following example uses a timeout context to limit the total time an API request can be active to a maximum of 20 seconds. The SDK must be able to read the full HTTP response body (Object body) within the timeout or the SDK returns a timeout error. For API operations that return an io.ReadCloser in their response type, the Context’s timeout includes reading the content from the io.ReadCloser.
ctx, cancelFn := context.WithTimeout(context.TODO(), 20*time.Second) defer cancelFn()
resp, err := svc.GetObjectWithContext(ctx, &s3.GetObjectInput{
Bucket: bucket, Key: object, })
if err != nil { return body, err }
return resp.Body, nil
See the complete example on GitHub.
Step 1: Set up Your AWS Account to Use AWS Cloud9
Using AWS Cloud9 with the AWS SDK for Go
You can use AWS Cloud9 with the AWS SDK for Go to write and run your Go code using just a browser.
AWS Cloud9 includes tools such as a code editor and terminal. Because the AWS Cloud9 IDE is cloud based, you can work on your projects from your office, home, or anywhere using an internet-connected machine. For general information about AWS Cloud9, see the AWS Cloud9 User Guide.
Follow these instructions to set up AWS Cloud9 with the AWS SDK for Go:
• Step 1: Set up Your AWS Account to Use AWS Cloud9 (p. 15)
• Step 2: Set up Your AWS Cloud9 Development Environment (p. 15)
• Step 3: Set up the AWS SDK for Go (p. 15)
• Step 4: Download Example Code (p. 16)
• Step 5: Run Example Code (p. 17)
Step 1: Set up Your AWS Account to Use AWS Cloud9
Start to use AWS Cloud9 by signing in to the AWS Cloud9 console as an AWS Identity and Access
Management (IAM) entity (for example, an IAM user) in your AWS account who has access permissions for AWS Cloud9.
To set up an IAM entity in your AWS account to access AWS Cloud9, and to sign in to the AWS Cloud9 console, see Team Setup for AWS Cloud9 in the AWS Cloud9 User Guide.
Step 2: Set up Your AWS Cloud9 Development Environment
After you sign in to the AWS Cloud9 console, use the console to create an AWS Cloud9 development environment. After you create the environment, AWS Cloud9 opens the IDE for that environment.
See Creating an Environment in AWS Cloud9 in the AWS Cloud9 User Guide for details.
Note
As you create your environment in the console for the first time, we recommend that you choose the option to Create a new instance for environment (EC2). This option tells AWS Cloud9 to create an environment, launch an Amazon EC2 instance, and then connect the new instance to the new environment. This is the fastest way to begin using AWS Cloud9.Step 3: Set up the AWS SDK for Go
After AWS Cloud9 opens the IDE for your development environment, use the IDE to set up the AWS SDK for Go in your environment, as follows.
Step 4: Download Example Code
1. If the terminal isn’t already open in the IDE, open it. On the menu bar in the IDE, choose Window, New Terminal.
2. Set your GOPATH environment variable. To do this, add the following code to the end of your shell profile file (for example, ~/.bashrc in Amazon Linux, assuming you chose the option to Create a new instance for environment (EC2), earlier in this topic), and then save the file.
GOPATH=~/environment/go export GOPATH
After you save the file, source the ~/.bashrc file to finish setting your GOPATH environment variable. To do this, run the following command. (This command assumes you chose the option to Create a new instance for environment (EC2), earlier in this topic.)
. ~/.bashrc
3. Run the following command to install the AWS SDK for Go.
go get -u github.com/aws/aws-sdk-go/...
If the IDE can’t find Go, run the following commands, one at a time in this order, to install it. (These commands assume you chose the option to Create a new instance for environment (EC2), earlier in this topic. Also, these commands assume the latest stable version of Go at the time this topic was written; for more information, see Downloads on The Go Programming Language website.)
wget https://storage.googleapis.com/golang/go1.9.3.linux-amd64.tar.gz # Download the Go installer.
sudo tar -C /usr/local -xzf ./go1.9.3.linux-amd64.tar.gz # Install Go.
rm ./go1.9.3.linux-amd64.tar.gz # Delete the Go installer, as you no longer need it.
After you install Go, add the path to the Go binary to your PATH environment variable. To do this, add the following code to the end of your shell profile file (for example, ~/.bashrc in Amazon Linux, assuming you chose the option to Create a new instance for environment (EC2), earlier in this topic), and then save the file.
PATH=$PATH:/usr/local/go/bin
After you save the file, source the ~/.bashrc file so that the terminal can now find the Go binary you just referenced. To do this, run the following command. (This command assumes you chose the option to Create a new instance for environment (EC2), earlier in this topic.)
. ~/.bashrc
Step 4: Download Example Code
Use the terminal you opened in the previous step to download example code for the AWS SDK for Go into the AWS Cloud9 development environment.
To do this, run the following command. This command downloads a copy of all of the code examples used in the official AWS SDK documentation into your environment’s root directory.
Step 5: Run Example Code
git clone https://github.com/awsdocs/aws-doc-sdk-examples.git
To find code examples for the AWS SDK for Go, use the Environment window to open the ENVIRONMENT_NAME/aws-doc-sdk-examples/go/example_code directory, where ENVIRONMENT_NAME is the name of your development environment.
To learn how to work with these and other code examples, see AWS SDK for Go Code Examples (p. 31).
Step 5: Run Example Code
To run code in your AWS Cloud9 development environment, see Run Your Code in the AWS Cloud9 User Guide.
Concurrency
Using Sessions to Configure Service Clients in the AWS SDK for Go
In the AWS SDK for Go, a session is an object that contains configuration information for service clients (p. 21). which you use to interact with AWS services. For example, sessions can include
information about the region where requests will be sent, which credentials to use, or additional request handlers. Whenever you create a service client, you must specify a session. For more information about sessions, see the session package in the AWS SDK for Go API Reference.
Sessions can be shared across all service clients that share the same base configuration. The session is built from the SDK’s default configuration and request handlers.
You should cache sessions when possible. This is because creating a new session loads all configuration values from the environment and configuration files each time the session is created. Sharing the session value across all of your service clients ensures the configuration is loaded the fewest number of times.
Concurrency
Sessions are safe to use concurrently as long as the session isn’t being modified. The SDK doesn’t modify the session once the session is created. Creating service clients concurrently from a shared session is safe.
Sessions with a Shared Configuration File
Using the previous method, you can create sessions that load the additional configuration file only if the AWS_SDK_LOAD_CONFIG environment variable is set. Alternatively you can explicitly create a session with a shared configuration enabled. To do this, you can use NewSessionWithOptions to configure how the session is created. Using the NewSessionWithOptions with SharedConfigState set to SharedConfigEnable will create the session as if the AWS_SDK_LOAD_CONFIG environment variable was set.
Creating Sessions
When you create a session, you can pass in optional aws.Config values that override the default or that override the current configuration values. This allows you to provide additional or case-based configuration as needed.
By default NewSession only loads credentials from the shared credentials file (~/.aws/credentials).
If the AWS_SDK_LOAD_CONFIG environment variable is set to a truthy value, the session is created from the configuration values from the shared configuration (~/.aws/config) and shared credentials (~/.aws/credentials) files. See Sessions with a Shared Configuration File (p. 18) for more
information.
Create a session with the default configuration and request handlers. The following example creates a session with credentials, region, and profile values from either the environment variables or the shared credentials file. It requires that the AWS_PROFILE is set, or default is used.
Create Sessions with Option Overrides
sess, err := session.NewSession()
The SDK provides a default configuration that all sessions use, unless you override a field. For example, you can specify an AWS Region when you create a session by using the aws.Config struct. For more information about the fields you can specify, see the aws.Config in the AWS SDK for Go API Reference.
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-east-2")}, )
Create an Amazon S3 client instance from a session:
sess, err := session.NewSession() if err != nil {
// Handle Session creation error }svc := s3.New(sess)
Create Sessions with Option Overrides
In addition to NewSession, you can create sessions using NewSessionWithOptions. This function allows you to control and override how the session will be created through code, instead of being driven by environment variables only.
Use NewSessionWithOptions when you want to provide the config profile, or override the shared credentials state (AWS_SDK_LOAD_CONFIG).
// Equivalent to session.New
sess, err := session.NewSessionWithOptions(session.Options{}) // Specify profile to load for the session's config
sess, err := session.NewSessionWithOptions(session.Options{
Profile: "profile_name", })
// Specify profile for config and region for requests sess, err := session.NewSessionWithOptions(session.Options{
Config: aws.Config{Region: aws.String("us-east-2")}, Profile: "profile_name",
})
// Force enable Shared Config support
sess, err := session.NewSessionWithOptions(session.Options{
SharedConfigState: SharedConfigEnable, })
// Assume an IAM role with MFA prompting for token code on stdin sess := session.Must(session.NewSessionWithOptions(session.Options{
AssumeRoleTokenProvider: stscreds.StdinTokenProvider, SharedConfigState: SharedConfigEnable,
}))
Deprecated New
The New function has been deprecated because it doesn’t provide a good way to return errors that occur when loading the configuration files and values. Because of this, NewSession was created so errors can be retrieved when creating a session fails.
Shared Configuration Fields
Shared Configuration Fields
By default, the SDK loads credentials from the shared credentials file ~/.aws/credentials. Any other configuration values are provided by the environment variables, SDK defaults, and user-provided aws.config values.
If the AWS_SDK_LOAD_CONFIG environment variable is set, or the SharedConfigEnable option is used to create the session (as shown in the following example), additional configuration information is also loaded from the shared configuration file (~/.aws/config), if it exists. If any configuration setting value differs between the two files, the value from the shared credentials file (~/.aws/credentials) takes precedence.
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable, }))
See the session package’s documentation for more information on shared credentials setup.
Environment Variables
When a session is created, you can set several environment variables to adjust how the SDK functions, and what configuration data it loads when creating sessions. Environment values are optional. For credentials, you must set both an access key and a secret access key. Otherwise, Go ignores the one you’ve set. All environment variable values are strings unless otherwise noted.
See the session package’s documentation for more information on environment variable setup.
Adding Request Handlers
You can add handlers to a session for processing HTTP requests. All service clients that use the session inherit the handlers. For example, the following handler logs every request and its payload made by a service client.
// Create a session, and add additional handlers for all service // clients created with the Session to inherit. Adds logging handler.
sess, err := session.NewSession()
sess.Handlers.Send.PushFront(func(r *request.Request) { // Log every request made and its payload
logger.Println("Request: %s/%s, Payload: %s",
r.ClientInfo.ServiceName, r.Operation, r.Params) })
Copying a Session
You can use the Copy method to create copies of sessions. Copying sessions is useful when you want to create multiple sessions that have similar settings. Each time you copy a session, you can specify different values for any field. For example, the following snippet copies the sess session while overriding the Region field to us-east-2:
usEast2Sess := sess.Copy(&aws.Config{Region: aws.String("us-east-2")})
Constructing a Service
Using the AWS SDK for Go with AWS Services
To make calls to an AWS service, you must first construct a service client instance with a session. A service client provides low-level access to every API action for that service. For example, you create an Amazon S3 service client to make calls to Amazon S3.
When you call service operations, you pass in input parameters as a struct. A successful call usually results in an output struct that you can use. For example, after you successfully call an Amazon S3 create bucket action, the action returns an output struct with the bucket’s location.
For the list of service clients, including their methods and parameters, see the AWS SDK for Go API Reference.
Constructing a Service
To construct a service client instance, use the NewSession() function. The following example creates an Amazon S3 service client.
sess, err := session.NewSession() if err != nil {
fmt.Println("Error creating session ", err) return
}svc := s3.New(sess)
After you have a service client instance, you can use it to call service operations. For more information about configurations, see Configuring the AWS SDK for Go (p. 5).
When you create a service client, you can pass in custom configurations so that you don’t need to create a session for each configuration. The SDK merges the two configurations, overriding session values with your custom configuration. For example, in the following snippet, the Amazon S3 client uses the mySession session but overrides the Region field with a custom value (us-west-2):
svc := s3.New(mySession, aws.NewConfig().WithRegion("us-west-2"))
Tagging Service Resources
You can tag service resources, such as Amazon S3 buckets, so that you can determine the costs of your service resources at whatever level of granularity you require.
The following example shows how to tag the Amazon S3 bucket MyBucket with Cost Center tag with the value 123456 and Stack tag with the value MyTestStack.
package main import (
Tagging Service Resources
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"fmt"
)
// Tag S3 bucket MyBucket with cost center tag "123456" and stack tag "MyTestStack".
//
// See:
// http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-alloc-tags.html func main() {
// Pre-defined values bucket := "MyBucket"
tagName1 := "Cost Center"
tagValue1 := "123456"
tagName2 := "Stack"
tagValue2 := "MyTestStack"
// Initialize a session in us-west-2 that the SDK will use to load credentials // from the shared credentials file. (~/.aws/credentials).
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2")}, )
if err != nil {
fmt.Println(err.Error()) return
}
// Create S3 service client svc := s3.New(sess)
// Create input for PutBucket method putInput := &s3.PutBucketTaggingInput{
Bucket: aws.String(bucket), Tagging: &s3.Tagging{
TagSet: []*s3.Tag{
{
Key: aws.String(tagName1), Value: aws.String(tagValue1), },
{
Key: aws.String(tagName2), Value: aws.String(tagValue2), },
}, }, }
_, err = svc.PutBucketTagging(putInput) if err != nil {
fmt.Println(err.Error()) return
}
// Now show the tags
// Create input for GetBucket method getInput := &s3.GetBucketTaggingInput{
Bucket: aws.String(bucket), }
result, err := svc.GetBucketTagging(getInput) if err != nil {
fmt.Println(err.Error()) return
}
Getting the HTTP Request and Response with Each Service Call
numTags := len(result.TagSet) if numTags > 0 {
fmt.Println("Found", numTags, "Tag(s):") fmt.Println("")
for _, t := range result.TagSet { fmt.Println(" Key: ", *t.Key) fmt.Println(" Value:", *t.Value) fmt.Println("")
} } else {
fmt.Println("Did not find any tags") }
}
Note that if a tag of the same name already exists, its value is overwritten by the new value.
Getting the HTTP Request and Response with Each Service Call
You can direct the AWS SDK for Go to display the HTTP request and response it sends and receives for each call by including a configuration option when constructing the service client.
The following example uses the DynamoDBListTables operation to illustrate how to add a custom header to a service call.
package main import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/dynamodb"
"fmt"
"os"
)
func main() {
// Initialize a session in us-west-2 that the SDK will use to load credentials // from the shared config file. (~/.aws/credentials).
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2")}, )
if err != nil {
fmt.Println("Error getting session:") fmt.Println(err)
os.Exit(1) }
// Create DynamoDB client
// and expose HTTP requests/responses
svc := dynamodb.New(sess, aws.NewConfig().WithLogLevel(aws.LogDebugWithHTTPBody)) // Add "CustomHeader" header with value of 10
svc.Handlers.Send.PushFront(func(r *request.Request) {
r.HTTPRequest.Header.Set("CustomHeader", fmt.Sprintf("%d", 10)) })
Service Operation Calls
// Call ListTables just to see HTTP request/response // The request should have the CustomHeader set to 10 _, _ = svc.ListTables(&dynamodb.ListTablesInput{}) }
If you run this program, the output should be similar to the following, where ACCESS-KEY is the access key of the user and TABLE-1, through TABLE-N are the names of the tables.
2017/10/25 11:10:57 DEBUG: Request dynamodb/ListTables Details:
---[ REQUEST POST-SIGN ]--- POST / HTTP/1.1
Host: dynamodb.us-west-2.amazonaws.com
User-Agent: aws-sdk-go/1.10.34 (go1.8; windows; amd64) Content-Length: 2
Accept-Encoding: identity
Authorization: AWS4-HMAC-SHA256 Credential=ACCESS-KEY/20171025/us-west-2/dynamodb/
aws4_request, SignedHeaders=accept-encoding;content-length;content-type;host;x-amz-date;x- amz-target, Signature=9c92efe5d6c597cf29e4f7cc74de6dc2e39f8010a0d4957a397c59ef9cde21f2 Content-Type: application/x-amz-json-1.0
CustomHeader: 10
X-Amz-Date: 20171025T181057Z
X-Amz-Target: DynamoDB_20120810.ListTables
{}---
2017/10/25 11:10:58 DEBUG: Response dynamodb/ListTables Details:
---[ RESPONSE ]--- HTTP/1.1 200 OK
Content-Length: 177 Connection: keep-alive
Content-Type: application/x-amz-json-1.0 Date: Wed, 25 Oct 2017 18:10:58 GMT Server: Server
X-Amz-Crc32: 3023160996
X-Amzn-Requestid: M5B4BM4UU569MVBSDG5O2O9ITJVV4KQNSO5AEMVJF66Q9ASUAAJG
---
2017/10/25 11:10:58 {"TableNames":["TABLE-1","...","TABLE-N"]}
Service Operation Calls
You can call a service operation directly or with its request form. When you call a service operation, the SDK synchronously validates the input, builds the request, signs it with your credentials, sends it to AWS, and then gets a response or an error. In most cases, you can call service operations directly.
Calling Operations
Calling the operation will sync as the request is built, signed, sent, and the response is received. If an error occurs during the operation, it will be returned. The output or resulting structure won’t be valid.
For example, to call the Amazon S3 GET Object API, use the Amazon S3 service client instance and call its GetObject method:
result, err := s3Svc.GetObject(&s3.GetObjectInput{...}) // result is a *s3.GetObjectOutput struct pointer // err is a error which can be cast to awserr.Error.
Calling Operations with the Request Form
Passing Parameters to a Service Operation
When calling an operation on a service, you pass in input parameters as option values, similar to passing in a configuration. For example, to retrieve an object, you must specify a bucket and the object’s key by passing in the following parameters to the GetObject method:
svc := s3.New(session.New()) svc.GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucketName"), Key: aws.String("keyName"), })
Each service operation has an associated input struct and, usually, an output struct. The structs follow the naming pattern OperationName Input and OperationName Output.
For more information about the parameters of each method, see the service client documentation in the AWS SDK for Go API Reference.
Calling Operations with the Request Form
Calling the request form of a service operation, which follows the naming pattern OperationName Request, provides a simple way to control when a request is built, signed, and sent. Calling the request form immediately returns a request object. The request object output is a struct pointer that is not valid until the request is sent and returned successfully.
Calling the request form can be useful when you want to construct a number of pre-signed requests, such as pre-signed Amazon S3 URLs. You can also use the request form to modify how the SDK sends a request.
The following example calls the request form of the GetObject method. The Send method signs the request before sending it.
req, result := s3Svc.GetObjectRequest(&s3.GetObjectInput{...})
// result is a *s3.GetObjectOutput struct pointer, not populated until req.Send() returns // req is a *aws.Request struct pointer. Used to Send request.
if err := req.Send(); err != nil { // process error
return }
// Process result
Handling Operation Response Body
Some API operations return a response struct that contain a Body field that is an io.ReadCloser. If you’re making requests with these operations, always be sure to call Close on the field.
resp, err := s3svc.GetObject(&s3.GetObjectInput{...}) if err != nil {
// handle error return
}
// Make sure to always close the response Body when finished defer resp.Body.Close()
decoder := json.NewDecoder(resp.Body)
if err := decoder.Decode(&myStruct); err != nil { // handle error
return
Concurrently Using Service Clients
}
Concurrently Using Service Clients
You can create goroutines that concurrently use the same service client to send multiple requests. You can use a service client with as many goroutines as you want. However, you cannot concurrently modify the service client’s configuration and request handlers. If you do, the service client operations might encounter race conditions. Define service client settings before you concurrently use it.
In the following example, an Amazon S3 service client is used in multiple goroutines. The example concurrently outputs all objects in bucket1, bucket2, and bucket3, which are all in the same region.
To make sure all objects from the same bucket are printed together, the example uses a channel.
sess, err := session.NewSession() if err != nil {
fmt.Println("Error creating session ", err) }var wg sync.WaitGroup
keysCh := make(chan string, 10) svc := s3.New(sess)
buckets := []string{"bucket1", "bucket2", "bucket3"}
for _, bucket := range buckets { params := &s3.ListObjectsInput{
Bucket: aws.String(bucket), MaxKeys: aws.Int64(100), }
wg.Add(1)
go func(param *s3.ListObjectsInput) { defer wg.Done()
err = svc.ListObjectsPages(params,
func(page *s3.ListObjectsOutput, last bool) bool { // Add the objects to the channel for each page for _, object := range page.Contents {
keysCh <- fmt.Sprintf("%s:%s", *params.Bucket, *object.Key) }
return true },
)
if err != nil {
fmt.Println("Error listing", *params.Bucket, "objects:", err) }
}(params) }go func() { wg.Wait() close(keysCh)
}()for key := range keysCh {
// Print out each object key as its discovered fmt.Println(key)
}
Using Pagination Methods
Typically, when you retrieve a list of items, you might need to check the output for a token or marker to confirm whether AWS returned all results from your request. If present, you use the token or marker
Using Waiters
to request the next set of results. Instead of managing these tokens or markers, you can use pagination methods provided by the SDK.
Pagination methods iterate over a list operation until the method retrieves the last page of results or until the callback function returns false. The names of these methods use the following pattern:
OperationName Pages. For example, the pagination method for the Amazon S3 list objects operation (ListObjects) is ListObjectPages.
The following example uses the ListObjectPages pagination method to list up to three pages of object keys from the ListObject operation. Each page consists of up to 10 keys, which is defined by the MaxKeys field.
svc, err := s3.NewSession(sess) if err != nil {
fmt.Println("Error creating session ", err) }
inputparams := &s3.ListObjectsInput{
Bucket: aws.String("mybucket"), MaxKeys: aws.Int64(10),
}pageNum := 0
svc.ListObjectsPages(inputparams, func(page *s3.ListObjectsOutput, lastPage bool) bool { pageNum++
for _, value := range page.Contents { fmt.Println(*value.Key)
}
return pageNum < 3 })
Using Waiters
The SDK provides waiters that continuously check for completion of a job. For example, when you send a request to create an Amazon S3 bucket, you can use a waiter to check when the bucket has been successfully created. That way, subsequent operations on the bucket are done only after the bucket has been created.
The following example uses a waiter that waits until specific instances have stopped.
sess, err := session.NewSession(aws.NewConfig().WithRegion("us-west-2")) if err != nil {
fmt.Println("Error creating session ", err) }// Create an EC2 client
ec2client := ec2.New(sess) // Specify two instances to stop
instanceIDsToStop := aws.StringSlice([]string{"i-12345678", "i-23456789"}) // Send request to stop instances
_, err = ec2client.StopInstances(&ec2.StopInstancesInput{
InstanceIds: instanceIDsToStop, })
if err != nil { panic(err) }
// Use a waiter function to wait until the instances are stopped describeInstancesInput := &ec2.DescribeInstancesInput{
InstanceIds: instanceIDsToStop,
}if err := ec2client.WaitUntilInstanceStopped(describeInstancesInput); err != nil { panic(err)
}
Using Waiters
fmt.Println("Instances are stopped.")
Handling Specific Service Error Codes
Handling Errors in the AWS SDK for Go
The AWS SDK for Go returns errors that satisfy the Go error interface type and the Error interface in the aws/awserr package. You can use the Error() method to get a formatted string of the SDK error message without any special handling.
if err != nil {
if awsErr, ok := err.(awserr.Error); ok { // process SDK error
} }
Errors returned by the SDK are backed by a concrete type that will satisfy the awserr.Error interface.
The interface has the following methods, which provide classification and information about the error.
• Code returns the classification code by which related errors are grouped.
• Message returns a description of the error.
• OrigErr returns the original error of type error that is wrapped by the awserr.Error interface, such as a standard library error or a service error.
Handling Specific Service Error Codes
The following example demonstrates how to handle error codes that you encounter while using the AWS SDK for Go. The example assumes you have already set up and configured the SDK (that is, all required packages are imported and your credentials and region are set). For more information, see Getting Started with the AWS SDK for Go (p. 3) and Configuring the AWS SDK for Go (p. 5).
This example highlights how you can use the awserr.Error type to perform logic based on specific error codes returned by service API operations.
In this example the S3 GetObject API operation is used to request the contents of an object in S3. The example handles the NoSuchBucket and NoSuchKey error codes, printing custom messages to stderr. If any other error is received, a generic message is printed.
svc := s3.New(sess)
resp, err := svc.GetObject(&s3.GetObjectInput{
Bucket: aws.String(os.Args[1]), Key: aws.String(os.Args[2]), })
if err != nil {
// Casting to the awserr.Error type will allow you to inspect the error // code returned by the service in code. The error code can be used // to switch on context specific functionality. In this case a context // specific error message is printed to the user based on the bucket // and key existing.
//
// For information on other S3 API error codes see:
// http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html if aerr, ok := err.(awserr.Error); ok {
Additional Error Information
switch aerr.Code() {
case s3.ErrCodeNoSuchBucket:
exitErrorf("bucket %s does not exist", os.Args[1]) case s3.ErrCodeNoSuchKey:
exitErrorf("object with key %s does not exist in bucket %s", os.Args[2], os.Args[1])
} }
See the complete example on GitHub.
Additional Error Information
In addition to the awserr.Error interface, you might be able to use other interfaces to get more information about an error.
Specific Error Interfaces
Other packages might provide their own error interfaces. For example, the service/s3/s3manager package provides a MultiUploadFailure interface to retrieve the upload ID. This is helpful when you must manually clean up a failed multi-part upload.
output, err := s3manager.Upload(svc, input, opts) if err != nil {
if multierr, ok := err.(MultiUploadFailure); ok { // Process error and its associated uploadID
fmt.Println("Error:", multierr.Code(), multierr.Message(), multierr.UploadID()) } else {
// Process error generically fmt.Println("Error:", err.Error()) }
}
For more information, see the s3Manager.MultiUploadFailure interface in the AWS SDK for Go API Reference.
SDK Request Examples
AWS SDK for Go Code Examples
The AWS SDK for Go examples can help you write your own Go applications that use Amazon Web Services. The examples assume you have already set up and configured the SDK (that is, you have imported all required packages and set your credentials and region). For more information, see Getting Started with the AWS SDK for Go (p. 3) and Configuring the AWS SDK for Go (p. 5).
Find the source code for these examples and others in the AWS documentation code examples repository on GitHub. To propose a new code example for the AWS documentation team to consider producing, create a new request. The team is looking to produce code examples that cover broader scenarios and use cases, versus simple code snippets that cover only individual API calls. For instructions, see the Proposing new code examples section in the Readme on GitHub.
Topics
• AWS SDK for Go Request Examples (p. 31)
• AWS CloudTrail Examples Using the AWS SDK for Go (p. 32)
• Amazon CloudWatch Examples Using the AWS SDK for Go (p. 38)
• AWS CodeBuild Examples Using the AWS SDK for Go (p. 50)
• Amazon DynamoDB Examples Using the AWS SDK for Go (p. 53)
• Amazon EC2 Examples Using the AWS SDK for Go (p. 64)
• Amazon S3 Glacier Examples Using the AWS SDK for Go (p. 86)
• IAM Examples Using the AWS SDK for Go (p. 88)
• AWS Key Management Service Examples Using the AWS SDK for Go (p. 112)
• AWS Lambda Examples Using the AWS SDK for Go (p. 115)
• Amazon Polly Examples Using the AWS SDK for Go (p. 121)
• Amazon S3 Examples Using the AWS SDK for Go (p. 124)
• Amazon SES Examples Using the AWS SDK for Go (p. 159)
• Amazon SNS Examples Using the AWS SDK for Go (p. 165)
• Amazon SQS Examples Using the AWS SDK for Go (p. 169)
• Amazon WorkDocs Examples (p. 184)
AWS SDK for Go Request Examples
The AWS SDK for Go examples can help you write your own applications. The examples assume you have already set up and configured the SDK (that is, you have imported all required packages and set your credentials and region). For more information, see Getting Started with the AWS SDK for Go (p. 3) and Configuring the AWS SDK for Go (p. 5).
Using context.Context with SDK Requests
In Go 1.7, the context.Context type was added to http.Request. This type provides an easy way to implement deadlines and cancellations on requests.
To use this pattern with the SDK, call WithContext on the HTTPRequest field of the SDK’s request.Request type, and provide your Context value. The following example highlights this process with a timeout on Amazon SQSReceiveMessage.