"detail": { "status": [ "FAILED"
] }}
This code defines a EventBridge rule that matches any event where the job status is FAILED. For more information about event patterns, see Events and Event Patterns in the Amazon EventBridge User Guide.
7. This rule applies across all of your AWS Batch groups and to every AWS Batch event. Alternatively, you can create a more specific rule to filter out some results.
8. For Select event bus, choose AWS default event bus. You can only create scheduled rules on the default event bus.
9. For Select targets, in Target, choose SNS topic, and select JobFailedAlert.
10. For Retry policy and dead-letter queue:, under Retry policy:
a. For Maximum age of event, enter a value between 1 minute (00:01) and 24 hours (24:00).
b. For Retry attempts, enter a number between 0 and 185.
c. For Dead-letter queue, choose whether to use a standard Amazon SQS queue as a dead-letter queue. EventBridge sends events that match this rule to the dead-letter queue if it can't deliver them to the target. Do one of the following:
• Choose None to not use a dead-letter queue.
• Choose Select an Amazon SQS queue in the current AWS account to use as the dead-letter queue and then select the queue to use from the drop-down list.
• Choose Select an Amazon SQS queue in an other AWS account as a dead-letter queue and then enter the ARN of the queue to use.
11. (Optional) Enter one or more tags for the rule.
12. Choose Create.
Step 3: Test Your Rule
To test your rule, submit a job that exits shortly after it starts with a non-zero exit code. If your event rule is configured correctly, you should receive an email message within a few minutes with the event text.
To test a rule
1. Open the AWS Batch console at https://console.aws.amazon.com/batch/.
2. Submit a new AWS Batch job. For more information, see Submitting a Job (p. 14). For the job's command, substitute this command to exit the container with an exit code of 1.
/bin/sh, -c, 'exit 1'
3. Check your email to confirm that you received an email alert for the failed job notification.
CloudWatch Logs IAM Policy
Using CloudWatch Logs with AWS Batch
You can configure your jobs to send log information to CloudWatch Logs. This enables you to view different logs from your jobs in one convenient location. This topic helps you get started using CloudWatch Logs on your jobs that were launched with an Amazon ECS-optimized Amazon Linux AMI.
For information about sending logs from your jobs to CloudWatch Logs, see Using the awslogs log driver (p. 64). For more information about CloudWatch Logs, see Monitoring Log Files in the Amazon CloudWatch User Guide.
Topics
• CloudWatch Logs IAM Policy (p. 159)
• Installing and configuring the CloudWatch agent (p. 160)
• Viewing CloudWatch Logs (p. 160)
CloudWatch Logs IAM Policy
Before your jobs can send log data to CloudWatch Logs, you must create an IAM policy to allow your container instances to use the CloudWatch Logs APIs, and then you must attach that policy to ecsInstanceRole.
To create the ECS-CloudWatchLogs IAM policy
1. Open the IAM console at https://console.aws.amazon.com/iam/.
2. In the navigation pane, choose Policies.
3. Choose Create policy, JSON.
4. Enter the following policy:
{ "Version": "2012-10-17", "Statement": [
{
"Effect": "Allow", "Action": [
"logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams"
],
"Resource": [
"arn:aws:logs:*:*:*"
] } ] }
5. Choose Review policy.
6. On the Review policy page, enter ECS-CloudWatchLogs for the Name and choose Create policy.
Installing and configuring the CloudWatch agent
To attach the ECS-CloudWatchLogs policy to ecsInstanceRole 1. Open the IAM console at https://console.aws.amazon.com/iam/.
2. In the navigation pane, choose Roles.
3. Choose ecsInstanceRole. If the role does not exist, follow the procedures in Amazon ECS instance role (p. 145) to create the role.
4. Choose Permissions, Attach policies.
5. To narrow the available policies to attach, for Filter, type ECS-CloudWatchLogs.
6. Select the ECS-CloudWatchLogs policy and choose Attach policy.
Installing and configuring the CloudWatch agent
After you have added the ECS-CloudWatchLogs policy to your ecsInstanceRole, you can install the CloudWatch agent on your container instances.
For more information, see Download and configure the CloudWatch agent using the command line in the Amazon CloudWatch User Guide.
Viewing CloudWatch Logs
After you have given your container instance role the proper permissions to send logs to CloudWatch Logs, and you have configured and started the agent, your container instance should be sending its log data to CloudWatch Logs. You can view and search these logs in the AWS Management Console.
NoteNew instance launches may take a few minutes to send data to CloudWatch Logs.
To view your CloudWatch Logs data
1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.
2. In the left navigation pane, choose Logs, Log groups.
3. Choose a log group to view.
Viewing CloudWatch Logs
4. Choose a log stream to view. By default, the streams are identified by the first 200 characters of the job name and the Amazon ECS task ID.
AWS Batch Information in CloudTrail
Logging AWS Batch API Calls with AWS CloudTrail
AWS Batch is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in AWS Batch. CloudTrail captures all API calls for AWS Batch as events. The calls captured include calls from the AWS Batch console and code calls to the AWS Batch API operations.
If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for AWS Batch. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in Event history. Using the information collected by CloudTrail, you can determine the request that was made to AWS Batch, the IP address from which the request was made, who made the request, when it was made, and additional details.
To learn more about CloudTrail, see the AWS CloudTrail User Guide.
AWS Batch Information in CloudTrail
CloudTrail is enabled on your AWS account when you create the account. When activity occurs in AWS Batch, that activity is recorded in a CloudTrail event along with other AWS service events in Event history. You can view, search, and download recent events in your AWS account. For more information, see Viewing Events with CloudTrail Event History.
For an ongoing record of events in your AWS account, including events for AWS Batch, create a trail.
A trail enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs.
For more information, see the following:
• Overview for Creating a Trail
• CloudTrail Supported Services and Integrations
• Configuring Amazon SNS Notifications for CloudTrail
• Receiving CloudTrail Log Files from Multiple Regions and Receiving CloudTrail Log Files from Multiple Accounts
All AWS Batch actions are logged by CloudTrail and are documented in the https://
docs.aws.amazon.com/batch/latest/APIReference/. For example, calls to the SubmitJob, ListJobs and DescribeJobs sections generate entries in the CloudTrail log files.
Every event or log entry contains information about who generated the request. The identity information helps you determine the following:
• Whether the request was made with root or AWS Identity and Access Management (IAM) user credentials.
• Whether the request was made with temporary security credentials for a role or federated user.
• Whether the request was made by another AWS service.
For more information, see the CloudTrail userIdentity Element.
Understanding AWS Batch Log File Entries
Understanding AWS Batch Log File Entries
A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the public API calls, so they don't appear in any specific order.
The following example shows a CloudTrail log entry that demonstrates the CreateComputeEnvironment action.
{ "eventVersion": "1.05", "userIdentity": { "type": "AssumedRole",
"principalId": "AIDACKCEVSQ6C2EXAMPLE:admin",
"arn": "arn:aws:sts::012345678910:assumed-role/Admin/admin", "accountId": "012345678910",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE", "sessionContext": {
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2017-12-20T00:48:46Z"
},
"sessionIssuer": { "type": "Role",
"principalId": "AIDACKCEVSQ6C2EXAMPLE", "arn": "arn:aws:iam::012345678910:role/Admin", "accountId": "012345678910",
"userAgent": "aws-cli/1.11.167 Python/2.7.10 Darwin/16.7.0 botocore/1.7.25", "requestParameters": {
"computeResources": { "subnets": [
"instanceRole": "ecsInstanceRole", "maxvCpus": 128,
"type": "EC2"
},
"state": "ENABLED", "type": "MANAGED",
"serviceRole": "service-role/AWSBatchServiceRole", "computeEnvironmentName": "Test"
Understanding AWS Batch Log File Entries
},
"responseElements": {
"computeEnvironmentName": "Test",
"computeEnvironmentArn": "arn:aws:batch:us-east-1:012345678910:compute-environment/
Test"
},
"requestID": "890b8639-e51f-11e7-b038-EXAMPLE", "eventID": "874f89fa-70fc-4798-bc00-EXAMPLE", "readOnly": false,
"eventType": "AwsApiCall",
"recipientAccountId": "012345678910"
}