} } }}
2. Sign in to the AWS CloudFormation console, and then choose Create Stack.
3. On the Specify Template panel, choose Upload a template file, choose your MyQueue.json file, and then choose Next.
4. On the Specify Details page, type MyQueue for Stack Name, and then choose Next.
5. On the Options page, choose Next.
6. On the Review page, choose Create.
AWS CloudFormation begins to create the MyQueue stack and displays the CREATE_IN_PROGRESS status. When the process is complete, AWS CloudFormation displays the CREATE_COMPLETE status.
7. (Optional) To display the name, URL, and ARN of the queue, choose the name of the stack and then on the next page expand the Outputs section.
Tutorial: Sending a message to an Amazon SQS queue from Amazon Virtual Private Cloud
In this tutorial, you learn how to send messages to an Amazon SQS queue over a secure, private network.
This network consists of a VPC that contains an Amazon EC2 instance. The instance connects to Amazon SQS through an interface VPC endpoint, allowing you to connect to the Amazon EC2 instance and send messages to the Amazon SQS queue even though the network is disconnected from the public internet.
For more information, see Amazon Virtual Private Cloud endpoints for Amazon SQS (p. 116).
Important
• You can use Amazon Virtual Private Cloud only with HTTPS Amazon SQS endpoints.
• When you configure Amazon SQS to send messages from Amazon VPC, you must enable private DNS and specify endpoints in the format sqs.us-east-2.amazonaws.com.
• Private DNS doesn't support legacy endpoints such as queue.amazonaws.com or us-east-2.queue.amazonaws.com.
Topics
• Step 1: Create an Amazon EC2 key pair (p. 98)
• Step 2: Create AWS resources (p. 98)
• Step 3: Confirm that your EC2 instance isn't publicly accessible (p. 99)
• Step 4: Create an Amazon VPC endpoint for Amazon SQS (p. 99)
• Step 5: Send a message to your Amazon SQS queue (p. 100)
Step 1: Create an Amazon EC2 key pair
Step 1: Create an Amazon EC2 key pair
A key pair lets you connect to an Amazon EC2 instance. It consists of a public key that encrypts your login information and a private key that decrypts it.
1. Sign in to the Amazon EC2 console.
2. On the navigation menu, under Network & Security, choose Key Pairs.
3. Choose Create Key Pair.
4. In the Create Key Pair dialog box, for Key pair name, enter SQS-VPCE-Tutorial-Key-Pair, and then choose Create.
5. Your browser downloads the private key file SQS-VPCE-Tutorial-Key-Pair.pem automatically.
Important
Save this file in a safe place. EC2 does not generate a .pem file for the same key pair a second time.
6. To allow an SSH client to connect to your EC2 instance, set the permissions for your private key file so that only your user can have read permissions for it, for example:
chmod 400 SQS-VPCE-Tutorial-Key-Pair.pem
Step 2: Create AWS resources
To set up the necessary infrastructure, you must use an AWS CloudFormation template, which is a blueprint for creating a stack comprised of AWS resources, such as Amazon EC2 instances and Amazon SQS queues.
The stack for this tutorial includes the following resources:
• A VPC and the associated networking resources, including a subnet, a security group, an internet gateway, and a route table
• An Amazon EC2 instance launched into the VPC subnet
• An Amazon SQS queue
1. Download the AWS CloudFormation template named SQS-VPCE-Tutorial-CloudFormation.yaml from GitHub.
2. Sign in to the AWS CloudFormation console.
3. Choose Create Stack.
4. On the Select Template page, choose Upload a template to Amazon S3, select the SQS-VPCE-SQS-Tutorial-CloudFormation.yaml file, and then choose Next.
5. On the Specify Details page, do the following:
a. For Stack name, enter SQS-VPCE-Tutorial-Stack.
b. For KeyName, choose SQS-VPCE-Tutorial-Key-Pair.
c. Choose Next.
6. On the Options page, choose Next.
7. On the Review page, in the Capabilities section, choose I acknowledge that AWS CloudFormation might create IAM resources with custom names., and then choose Create.
AWS CloudFormation begins to create the stack and displays the CREATE_IN_PROGRESS status. When the process is complete, AWS CloudFormation displays the CREATE_COMPLETE status.
Step 3: Confirm that your EC2 instance isn't publicly accessible
Step 3: Confirm that your EC2 instance isn't publicly accessible
Your AWS CloudFormation template launches an EC2 instance named SQS-VPCE-Tutorial-EC2-Instance into your VPC. This EC2 instance doesn't allow outbound traffic and isn't able to send messages to Amazon SQS. To verify this, you must connect to the instance, try to connect to a public endpoint, and then try to message Amazon SQS.
1. Sign in to the Amazon EC2 console.
2. On the navigation menu, under Instances, choose Instances.
3. Select SQS-VPCE-Tutorial-EC2Instance.
4. Copy the hostname under Public DNS (IPv4), for example, ec2-203-0-113-0.us-west-2.compute.amazonaws.com.
5. From the directory that contains the key pair that you created earlier (p. 98), connect to the instance using the following command, for example:
ssh -i SQS-VPCE-Tutorial-KeyPair.pem [email protected]
6. Try to connect to any public endpoint, for example:
ping amazon.com
The connection attempt fails, as expected.
7. Sign in to the Amazon SQS console.
8. From the list of queues, select the queue created by your AWS CloudFormation template, for example, VPCE-SQS-Tutorial-Stack-CFQueue-1ABCDEFGH2IJK.
9. On the Details table, copy the URL, for example, https://sqs.us-east-2.amazonaws.com/123456789012/.
10. From your EC2 instance, try to publish a message to the queue using the following command, for example:
aws sqs send-message --region us-east-2 --endpoint-url
https://sqs.us-east-2.amazonaws.com/ queue-url https://sqs.us-https://sqs.us-east-2.amazonaws.com/123456789012/ --message-body "Hello from Amazon SQS."
The sending attempt fails, as expected.
Important
Later, when you create a VPC endpoint for Amazon SQS, your sending attempt will succeed.
Step 4: Create an Amazon VPC endpoint for Amazon SQS
To connect your VPC to Amazon SQS, you must define an interface VPC endpoint. After you add the endpoint, you can use the Amazon SQS API from the EC2 instance in your VPC. This allows you to send messages to a queue within the AWS network without crossing the public internet.
NoteThe EC2 instance still doesn't have access to other AWS services and endpoints on the internet.
1. Sign in to the Amazon VPC console.
Step 5: Send a message to your Amazon SQS queue
2. On the navigation menu, choose Endpoints.
3. Choose Create Endpoint.
4. On the Create Endpoint page, for Service Name, choose the service name for Amazon SQS.
Note
The service names vary based on the current AWS Region. For example, if you are in US East (Ohio), the service name is com.amazonaws.us-east-2.sqs.
5. For VPC, choose SQS-VPCE-Tutorial-VPC.
6. For Subnets, choose the subnet whose Subnet ID contains SQS-VPCE-Tutorial-Subnet.
7. For Security group, choose Select security groups, and then choose the security group whose Group Name contains SQS VPCE Tutorial Security Group.
8. Choose Create endpoint.
The interface VPC endpoint is created and its ID is displayed, for example, vpce-0ab1cdef2ghi3j456k.
9. Choose Close.
The Amazon VPC console opens the Endpoints page.
Amazon VPC begins to create the endpoint and displays the pending status. When the process is complete, Amazon VPC displays the available status.
Step 5: Send a message to your Amazon SQS queue
Now that your VPC includes an endpoint for Amazon SQS, you can connect to your EC2 instance and send messages to your queue.
1. Reconnect to your EC2 instance, for example:
ssh -i SQS-VPCE-Tutorial-KeyPair.pem [email protected]
2. Try to publish a message to the queue again using the following command, for example:
aws sqs send-message --region us-east-2 --endpoint-url
https://sqs.us-east-2.amazonaws.com/ queue-url https://sqs.us-https://sqs.us-east-2.amazonaws.com/123456789012/ --message-body "Hello from Amazon SQS."
The sending attempt succeeds and the MD5 digest of the message body and the message ID are displayed, for example:
{
"MD5OfMessageBody": "a1bcd2ef3g45hi678j90klmn12p34qr5", "MessageId": "12345a67-8901-2345-bc67-d890123e45fg"
}
For information about receiving and deleting the message from the queue created by your AWS CloudFormation template (for example, VPCE-SQS-Tutorial-Stack-CFQueue-1ABCDEFGH2IJK), see Receiving and deleting messages (console) (p. 22).
For information about deleting your resources, see the following:
• Deleting a VPC Endpoint in the Amazon VPC User Guide
• Deleting an Amazon SQS queue (p. 23)
Step 5: Send a message to your Amazon SQS queue
• Terminate Your Instance in the Amazon EC2 User Guide for Linux Instances
• Deleting Your VPC in the Amazon VPC User Guide
• Deleting a Stack on the AWS CloudFormation Console in the AWS CloudFormation User Guide
• Deleting Your Key Pair in the Amazon EC2 User Guide for Linux Instances
Quotas related to queues
Amazon SQS quotas
This topic lists quotas within Amazon Simple Queue Service (Amazon SQS).
Topics
• Quotas related to queues (p. 102)
• Quotas related to messages (p. 103)
• Quotas related to policies (p. 106)
Quotas related to queues
The following table lists quotas related to queues.
Quota Description
Delay queue The default (minimum) delay for a queue is 0 seconds. The maximum is 15 minutes.
Listed queues 1,000 queues per ListQueues request.
Long polling wait time The maximum long polling wait time is 20 seconds.
Message groups There is no quota to the number of message groups within a FIFO queue.
Messages per queue (backlog) The number of messages that an Amazon SQS queue can store is unlimited.
For most standard queues (depending on queue traffic and message backlog), there can be a maximum of approximately 120,000 inflight messages (received from a queue by a consumer, but not yet deleted from the queue).
If you reach this quota while using short polling (p. 43), Amazon SQS returns the OverLimit error message. If you use long polling (p. 44), Amazon SQS returns no error messages. To avoid reaching the quota, you should delete messages from the queue after they're processed. You can also increase the number of queues you use to process your messages. To request a quota increase, submit a support request.
Messages per queue (in flight)
For FIFO queues, there can be a maximum of 20,000 inflight messages (received from a queue by a consumer, but not yet deleted from the queue). If you reach this quota, Amazon SQS returns no error messages.
Queue name A queue name can have up to 80 characters. The following characters are accepted: alphanumeric characters, hyphens (-), and underscores (_).
Note
Queue names are case-sensitive (for example, Test-queue and test-queue are different queues).
Quotas related to messages
Quota Description
The name of a FIFO queue must end with the .fifo suffix.
The suffix counts towards the 80-character queue name quota. To determine whether a queue is FIFO (p. 28), you can check whether the queue name ends with the suffix.
We don't recommend adding more than 50 tags to a queue.
The tag Key is required, but the tag Value is optional.
The tag Key and tag Value are case-sensitive.
The tag Key and tag Value can include Unicode alphanumeric characters in UTF-8 and whitespaces. The following special characters are allowed: _ . : / = + - @ The tag Key or Value must not include the reserved prefix aws: (you can't delete tag keys or values with this prefix).
The maximum tag Key length is 128 Unicode characters in UTF-8. The tag Key must not be empty or null.
The maximum tag Value length is 256 Unicode characters in UTF-8. The tag Value may be empty or null.
Queue tag
Tagging actions are limited to 5 TPS per AWS account. If your application requires a higher throughput, submit a request.
Quotas related to messages
The following table lists quotas related to messages.
Quota Description
Batched message ID A batched message ID can have up to 80 characters. The following characters are accepted: alphanumeric characters, hyphens (-), and underscores (_).
Message attributes A message can contain up to 10 metadata attributes.
Message batch A single message batch request can include a maximum of 10 messages. For more information, see Configuring AmazonSQSBufferedAsyncClient (p. 170) in the Amazon SQS batch actions (p. 168) section.
Message content A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed: #x9 |
#xA | #xD | #x20 to #xD7FF | #xE000 to #xFFFD | #x10000 to #x10FFFF
Any characters not included in this list are rejected. For more information, see the W3C specification for characters.
Message group ID Consume messages from the backlog to avoid building up a large backlog of messages with the same message group ID (p. 62).
Quotas related to messages
Quota Description
MessageGroupId is required for FIFO queues. You can't use it for Standard queues.
You must associate a non-empty MessageGroupId with a message. If you don't provide a MessageGroupId, the action fails.
The length of MessageGroupId is 128 characters. Valid values: alphanumeric characters and punctuation (!"#$
%&'()*+,-./:;<=>?@[\]^_`{|}~).
Message retention By default, a message is retained for 4 days. The minimum is 60 seconds (1 minute). The maximum is 1,209,600 seconds (14 days).
Standard queues support a nearly unlimited number of API calls per second, per API action (SendMessage, ReceiveMessage, or DeleteMessage).
Message throughput
FIFO queues
• Without batching, FIFO queues support up to 300 API calls per second, per API method (SendMessage, ReceiveMessage, or DeleteMessage).
• If you use batching (p. 168), FIFO queues support up to 3,000 messages per second, per API method (SendMessageBatch, ReceiveMessage, or
DeleteMessageBatch). The 3000 messages per second represent 300 API calls, each with a batch of 10 messages.
To request a quota increase, submit a support request.
Quotas related to messages
Quota Description
High throughput for FIFO queues (p. 32)
• Without batching (SendMessage, ReceiveMessage, and DeleteMessage), high throughput for FIFO queues support up to 3,000 messages per second, per API method. For maximum throughput, increase the number of message group IDs you use for messages sent without batching.
• You can increase throughput up to 30,000 messages per second by using batching APIs (SendMessageBatch and DeleteMessageBatch). The 30,000 messages per second represents 3,000 API calls, each with a batch of 10 messages. To achieve the maximum throughput when using SendMessageBatch and DeleteMessageBatch, all messages in a batch request must use the same message group ID.
For more information, see Partitions and data distribution for high throughput for SQS FIFO queues (p. 32).
NoteThe above quotas are available in the following AWS Regions:
• US East (Ohio)
• US East (N. Virginia)
• US West (Oregon)
• Europe (Ireland)
In all other AWS Regions, maximum throughput is 1,500 (without batching) or 15,000 (using batching) messages per second, per API action.
Message timer The default (minimum) delay for a message is 0 seconds. The maximum is 15 minutes.
Message size The minimum message size is 1 byte (1 character). The maximum is 262,144 bytes (256 KB).
To send messages larger than 256 KB, you can use the Amazon SQS Extended Client Library for Java. This library allows you to send an Amazon SQS message that contains a reference to a message payload in Amazon S3. The maximum payload size is 2 GB.
Message visibility timeout The default visibility timeout for a message is 30 seconds.
The minimum is 0 seconds. The maximum is 12 hours.
Policy information The maximum quota is 8,192 bytes, 20 statements, 50 principals, or 10 conditions. For more information, see Quotas related to policies (p. 106).
Quotas related to policies
Quotas related to policies
The following table lists quotas related to policies.
Name Maximum
Bytes 8,192
Conditions 10
Principals 50
Statements 20
Automating notifications using EventBridge
Automating and troubleshooting Amazon SQS queues
This section provides information about automating and troubleshooting Amazon SQS queues.
Topics
• Automating notifications from AWS services to Amazon SQS using Amazon EventBridge (p. 107)
• Troubleshooting Amazon Simple Queue Service queues using AWS X-Ray (p. 107)
Automating notifications from AWS services to Amazon SQS using Amazon EventBridge
Amazon EventBridge lets you automate AWS services and respond to system events such as application availability issues or resource changes. Events from AWS services are delivered to EventBridge nearly in real time. You can write simple rules to indicate which events are of interest to you and what automated actions to take when an event matches a rule.
EventBridge lets you set a variety of targets—such as Amazon SQS standard and FIFO queues—which receive events in JSON format. For more information, see the Amazon EventBridge User Guide.
Troubleshooting Amazon Simple Queue Service queues using AWS X-Ray
AWS X-Ray collects data about requests that your application serves and lets you view and filter data to identify potential issues and opportunities for optimization. For any traced request to your application, you can see detailed information about the request, the response, and the calls that your application makes to downstream AWS resources, microservices, databases and HTTP web APIs.
To send AWS X-Ray trace headers through Amazon SQS, you can do one of the following:
• Use the X-Amzn-Trace-Id tracing header.
• Use the AWSTraceHeader message system attribute (p. 41).
To collect data on errors and latency, you must instrument the AmazonSQS client using the AWS X-Ray SDK.
You can use the AWS X-Ray console to view the map of connections between Amazon SQS and other services that your application uses. You can also use the console to view metrics such as average latency and failure rates. For more information, see Amazon SQS and AWS X-Ray in the AWS X-Ray Developer Guide.
Data protection
Security in Amazon SQS
This section provides information about Amazon SQS security, authentication and access control, and the Amazon SQS Access Policy Language.
Topics
• Data protection (p. 108)
• Identity and access management in Amazon SQS (p. 117)
• Logging and monitoring in Amazon SQS (p. 145)
• Compliance validation for Amazon SQS (p. 157)
• Resilience in Amazon SQS (p. 158)
• Infrastructure security in Amazon SQS (p. 159)
• Amazon SQS security best practices (p. 159)
Data protection
The AWS shared responsibility model applies to data protection in Amazon Simple Queue Service.
As described in this model, AWS is responsible for protecting the global infrastructure that runs all of the AWS Cloud. You are responsible for maintaining control over your content that is hosted on this infrastructure. This content includes the security configuration and management tasks for the AWS services that you use. For more information about data privacy, see the Data Privacy FAQ. For information about data protection in Europe, see the AWS Shared Responsibility Model and GDPR blog post on the AWS Security Blog.
For data protection purposes, we recommend that you protect AWS account credentials and set up individual user accounts with AWS Identity and Access Management (IAM). That way each user is given only the permissions necessary to fulfill their job duties. We also recommend that you secure your data in the following ways:
• Use multi-factor authentication (MFA) with each account.
• Use SSL/TLS to communicate with AWS resources. We recommend TLS 1.2 or later.
• Set up API and user activity logging with AWS CloudTrail.
• Use AWS encryption solutions, along with all default security controls within AWS services.
• Use advanced managed security services such as Amazon Macie, which assists in discovering and securing personal data that is stored in Amazon S3.
• If you require FIPS 140-2 validated cryptographic modules when accessing AWS through a command line interface or an API, use a FIPS endpoint. For more information about the available FIPS endpoints, see Federal Information Processing Standard (FIPS) 140-2.
We strongly recommend that you never put confidential or sensitive information, such as your customers' email addresses, into tags or free-form fields such as a Name field. This includes when you work with Amazon SQS or other AWS services using the console, API, AWS CLI, or AWS SDKs. Any data that you enter into tags or free-form fields used for names may be used for billing or diagnostic logs.
If you provide a URL to an external server, we strongly recommend that you do not include credentials information in the URL to validate your request to that server.