AWS X-Ray

(1)

AWS X-Ray

Developer Guide

(2)

AWS X-Ray: Developer Guide

Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be aﬃliated with, connected to, or sponsored by Amazon.

(3)

What is AWS X-Ray?

AWS X-Ray is a service that collects data about requests that your application serves, and provides tools that you can use to view, ﬁlter, and gain insights into that data to identify issues and opportunities for optimization. For any traced request to your application, you can see detailed information not only about the request and response, but also about calls that your application makes to downstream AWS resources, microservices, databases, and web APIs.

AWS X-Ray receives traces from your application, in addition to AWS services your application uses that are already integrated with X-Ray. Instrumenting your application involves sending trace data for incoming and outbound requests and other events within your application, along with metadata about each request. Many instrumentation scenarios require only conﬁguration changes. For example, you can instrument all incoming HTTP requests and downstream calls to AWS services that your Java application makes. There are several SDKs, agents, and tools that can be used to instrument your application for X- Ray tracing. See Instrumenting your application (p. 212) for more information.

AWS services that are integrated with X-Ray (p. 181) can add tracing headers to incoming requests, send trace data to X-Ray, or run the X-Ray daemon. For example, AWS Lambda can send trace data about requests to your Lambda functions, and run the X-Ray daemon on workers to make it simpler to use the X-Ray SDK.

(10)

Instead of sending trace data directly to X-Ray, each client SDK sends JSON segment documents to a daemon process listening for UDP traﬃc. The X-Ray daemon (p. 165) buﬀers segments in a queue and uploads them to X-Ray in batches. The daemon is available for Linux, Windows, and macOS, and is included on AWS Elastic Beanstalk and AWS Lambda platforms.

X-Ray uses trace data from the AWS resources that power your cloud applications to generate a detailed service graph. The service graph shows the client, your front-end service, and backend services that your front-end service calls to process requests and persist data. Use the service graph to identify bottlenecks, latency spikes, and other issues to solve to improve the performance of your applications.

(11)

See the getting started tutorial (p. 4) to start using X-Ray with an instrumented sample application.

(12)

Getting started with AWS X-Ray

To get started with AWS X-Ray, launch a sample app in Elastic Beanstalk that is already

instrumented (p. 228) to generate trace data. In a few minutes, you can launch the sample app, generate traﬃc, send segments to X-Ray, and view a service graph and traces in the AWS Management Console.

This tutorial uses a sample Java application (p. 134) to generate segments and send them to X-Ray.

The application uses the Spring framework to implement a JSON web API and the AWS SDK for Java to persist data to Amazon DynamoDB. A servlet ﬁlter in the application instruments all incoming requests served by the application, and a request handler on the AWS SDK client instruments downstream calls to DynamoDB.

With the X-Ray SDK for Java, you can trace all of your application's primary and downstream AWS resources by making two conﬁguration changes:

• Add the X-Ray SDK for Java's tracing filter to your servlet configuration in a WebConfig class or web.xml file.

• Take the X-Ray SDK for Java's submodules as build dependencies in your Maven or Gradle build conﬁguration.

(13)

Prerequisites

You can also access the raw service map and trace data by using the AWS CLI to call the X-Ray API. The service map and trace data are JSON that you can query to ensure that your application is sending data, or to check speciﬁc ﬁelds as part of your test automation.

Sections

• Prerequisites (p. 5)

• Deploy to Elastic Beanstalk and generate trace data (p. 6)

• View the service map in the X-Ray console (p. 6)

• Conﬁguring Amazon SNS notiﬁcations (p. 9)

• Explore the sample application (p. 11)

• Optional: Least privilege policy (p. 13)

• Clean up (p. 15)

• Next steps (p. 15)

Prerequisites

This tutorial uses Elastic Beanstalk to create and conﬁgure the resources that run the sample application and X-Ray daemon. If you use an IAM user with limited permissions, add the Elastic Beanstalk managed user policy to grant your IAM user permission to use Elastic Beanstalk, and the AWSXrayReadOnlyAccess managed policy for permission to read the service map and traces in the X- Ray console.

Create an Elastic Beanstalk environment for the sample application. If you haven't used Elastic Beanstalk before, this will also create a service role and instance proﬁle for your application. A default VPC

must exist in the region you're going to deploy to or Elastic Beanstalk will fail to deploy the sample application.

To create an Elastic Beanstalk environment

1. Open the Elastic Beanstalk Management Console with this preconﬁgured link: https://

console.aws.amazon.com/elasticbeanstalk/#/newApplication?applicationName=scorekeep 2. Under the Platform section, set the Platform to Java and the Platform branch to Java 8

running on 64bit Amazon Linux.

Note

The current version of the Scorekeep application requires Java 8. You will receive a warning that Java 8 is a deprecated platform branch. While this is not recommended for production applications, you may run this sample application on Java 8 for non-production use, in order to learn about X-Ray.

3. Choose Create application to create an application with an environment running the Java 8 SE platform.

4. When your environment is ready, the console redirects you to the environment Dashboard.

5. Click the URL at the top of the page to open the site.

The instances in your environment need permission to send data to the AWS X-Ray service. Additionally, the sample application uses Amazon S3 and DynamoDB. Modify the default Elastic Beanstalk instance proﬁle to include permissions to use these services.

To add X-Ray, Amazon S3 and DynamoDB permissions to your Elastic Beanstalk environment

1. Open the Elastic Beanstalk instance proﬁle in the IAM console: aws-elasticbeanstalk-ec2-role.

2. Choose Attach Policies.

3. Attach AWSXrayFullAccess, AmazonS3FullAccess, and AmazonDynamoDBFullAccess to the role.

(14)

Deploy to Elastic Beanstalk and generate trace data

Note

Full access policies are not best practice for general use. For instructions on conﬁguring a policy with least privilege to reduce security risks, see Optional: Least privilege policy (p. 13).

Deploy to Elastic Beanstalk and generate trace data

Deploy the sample application to your Elastic Beanstalk environment. The sample application uses Elastic Beanstalk configuration files to configure the environment for use with X-Ray and create the DynamoDB that it uses automatically.

To deploy the source code

1. Download the sample app: eb-java-scorekeep-xray-gettingstarted-v1.3.zip 2. Open the Elastic Beanstalk console.

3. Navigate to the management console for your environment.

4. Choose Upload and Deploy.

5. Upload eb-java-scorekeep-xray-gettingstarted-v1.3.zip, and then choose Deploy.

The sample application includes a front-end web app. Use the web app to generate traﬃc to the API and send trace data to X-Ray.

To generate trace data

1. In the environment Dashboard, click the URL to open the web app.

2. Choose Create to create a user and session.

3. Type a game name, set the Rules to Tic Tac Toe, and then choose Create to create a game.

4. Choose Play to start the game.

5. Choose a tile to make a move and change the game state.

Each of these steps generates HTTP requests to the API, and downstream calls to DynamoDB to read and write user, session, game, move, and state data.

View the service map in the X-Ray console

You can see the service map and traces generated by the sample application in the X-Ray console.

To use the X-Ray console

1. Open the service map page of the X-Ray console.

2. The console shows a representation of the service graph that X-Ray generates from the trace data sent by the application.

The service map shows the web app client, the API running in Elastic Beanstalk, the DynamoDB service, and each DynamoDB table that the application uses. Every request to the application, up to a conﬁgurable maximum number of requests per second, is traced as it hits the API, generates requests to downstream services, and completes.

(15)

View the service map in the X-Ray console

You can choose any node in the service graph to view traces for requests that generated traﬃc to that node. Currently, the Amazon SNS node is yellow. Drill down to ﬁnd out why.

To ﬁnd the cause of the error

1. Choose the node named Amazon SNS. The node details panel is displayed.

2. Choose View traces to access the Trace overview screen.

3. Choose the trace from the Trace list. This trace doesn't have a method or URL because it was recorded during startup instead of in response to an incoming request.

(16)

View the service map in the X-Ray console

4. Choose the error status icon within the Amazon SNS segment at the bottom of the page, to open the Exceptions page for the SNS subsegment.

5. The X-Ray SDK automatically captures exceptions thrown by instrumented AWS SDK clients and records the stack trace.

(17)

Conﬁguring Amazon SNS notiﬁcations

The cause indicates that the email address provided in a call to createSubscription made in the WebConfig class was invalid. Let's ﬁx that.

Conﬁguring Amazon SNS notiﬁcations

Scorekeep uses Amazon SNS to send notiﬁcations when users complete a game. When the application starts up, it tries to create a subscription for an email address deﬁned in an environment variable.

That call is currently failing, causing the Amazon SNS node in your service map to be red. Configure a notification email in an environment variable to enable notifications and make the service map green.

To conﬁgure Amazon SNS notiﬁcations for scorekeep

1. Open the Elastic Beanstalk console.

3. Choose Conﬁguration.

4. Choose Software Conﬁguration.

5. Under Environment Properties, replace the default value with your email address.

(18)

Conﬁguring Amazon SNS notiﬁcations

Note

The default value uses an AWS CloudFormation function to retrieve a parameter stored in a conﬁguration ﬁle (a dummy value in this case).

6. Choose Apply.

When the update completes, Scorekeep restarts and creates a subscription to the SNS topic. Check your email and conﬁrm the subscription to see updates when you complete a game.

(19)

Explore the sample application

The sample application is an HTTP web API in Java that is conﬁgured to use the X-Ray SDK for Java.

When you deploy the application to Elastic Beanstalk, it creates the DynamoDB tables, compiles the API with Gradle, and conﬁgures the nginx proxy server to serve the web app statically at the root path. At the same time, Elastic Beanstalk routes requests to paths starting with /api to the API.

To instrument incoming HTTP requests, the application adds the TracingFilter provided by the SDK.

Example src/main/java/scorekeep/WebConﬁg.java - servlet ﬁlter

import javax.servlet.Filter;

import com.amazonaws.xray.javax.servlet.AWSXRayServletFilter;

...

@Configuration

public class WebConfig { @Bean

public Filter TracingFilter() {

return new AWSXRayServletFilter("Scorekeep");

}...

This ﬁlter sends trace data about all incoming requests that the application serves, including request URL, method, response status, start time, and end time.

The application also makes downstream calls to DynamoDB using the AWS SDK for Java. To instrument these calls, the application simply takes the AWS SDK-related submodules as dependencies, and the X- Ray SDK for Java automatically instruments all AWS SDK clients.

The application uses a Buildfile ﬁle to build the source code on-instance with Gradle and a

Procfile ﬁle to run the executable JAR that Gradle generates. Buildfile and Procfile support is a feature of the Elastic Beanstalk Java SE platform.

Example Buildﬁle

build: gradle build

Example Procﬁle

web: java -Dserver.port=5000 -jar build/libs/scorekeep-api-1.0.0.jar

The build.gradle ﬁle downloads the SDK submodules from Maven during compilation by declaring them as dependencies.

Example build.gradle -- dependencies

...dependencies {

compile("org.springframework.boot:spring-boot-starter-web") testCompile('org.springframework.boot:spring-boot-starter-test') compile('com.amazonaws:aws-java-sdk-dynamodb')

compile("com.amazonaws:aws-xray-recorder-sdk-core") compile("com.amazonaws:aws-xray-recorder-sdk-aws-sdk")

compile("com.amazonaws:aws-xray-recorder-sdk-aws-sdk-instrumentor") ...

}

(20)

Explore the sample application

dependencyManagement { imports {

mavenBom("com.amazonaws:aws-java-sdk-bom:1.11.67")

mavenBom("com.amazonaws:aws-xray-recorder-sdk-bom:2.11.0") }

}

The core, AWS SDK, and AWS SDK Instrumentor submodules are all that's required to automatically instrument any downstream calls made with the AWS SDK.

To run the X-Ray daemon, the application uses another feature of Elastic Beanstalk, conﬁguration ﬁles.

The conﬁguration ﬁle tells Elastic Beanstalk to run the daemon and send its log on demand.

Example .ebextensions/xray.conﬁg

option_settings:

aws:elasticbeanstalk:xray:

XRayEnabled: true files:

"/opt/elasticbeanstalk/tasks/taillogs.d/xray-daemon.conf" : mode: "000644"

owner: root group: root content: |

/var/log/xray/xray.log

The X-Ray SDK for Java provides a class named AWSXRay that provides the global recorder, a

TracingHandler that you can use to instrument your code. You can conﬁgure the global recorder to customize the AWSXRayServletFilter that creates segments for incoming HTTP calls. The sample includes a static block in the WebConfig class that conﬁgures the global recorder with plugins and sampling rules.

Example src/main/java/scorekeep/WebConﬁg.java - recorder

import com.amazonaws.xray.AWSXRay;

import com.amazonaws.xray.AWSXRayRecorderBuilder;

import com.amazonaws.xray.plugins.EC2Plugin;

import com.amazonaws.xray.plugins.ElasticBeanstalkPlugin;

import com.amazonaws.xray.strategy.sampling.LocalizedSamplingStrategy;

@Configuration

public class WebConfig { ...

static {

AWSXRayRecorderBuilder builder = AWSXRayRecorderBuilder.standard().withPlugin(new EC2Plugin()).withPlugin(new ElasticBeanstalkPlugin());

URL ruleFile = WebConfig.class.getResource("/sampling-rules.json");

builder.withSamplingStrategy(new LocalizedSamplingStrategy(ruleFile));

AWSXRay.setGlobalRecorder(builder.build());

}}

This example uses the builder to load sampling rules from a ﬁle named sampling-rules.json.

Sampling rules determine the rate at which the SDK records segments for incoming requests.

Example src/main/java/resources/sampling-rules.json

{

(21)

Optional: Least privilege policy

"version": 1, "rules": [ {

"description": "Resource creation.", "service_name": "*",

"http_method": "POST", "url_path": "/api/*", "fixed_target": 1, "rate": 1.0

}, {

"description": "Session polling.", "service_name": "*",

"http_method": "GET",

"url_path": "/api/session/*", "fixed_target": 0,

"rate": 0.05 },

{

"description": "Game polling.", "service_name": "*",

"http_method": "GET", "url_path": "/api/game/*/*", "fixed_target": 0,

"rate": 0.05 },

{

"description": "State polling.", "service_name": "*",

"http_method": "GET",

"url_path": "/api/state/*/*/*", "fixed_target": 0,

"rate": 0.05 }

], "default": {

"fixed_target": 1, "rate": 0.1 }

}

The sampling rules file defines four custom sampling rules and the default rule. For each incoming request, the SDK evaluates the custom rules in the order in which they are defined. The SDK applies the first rule that matches the request's method, path, and service name. For Scorekeep, the first rule catches all POST requests (resource creation calls) by applying a fixed target of one request per second and a rate of 1.0, or 100 percent of requests after the fixed target is satisfied.

The other three custom rules apply a five percent rate with no fixed target to session, game, and state reads (GET requests). This minimizes the number of traces for periodic calls that the front end makes automatically every few seconds to ensure the content is up to date. For all other requests, the file defines a default rate of one request per second and a rate of 10 percent.

The sample application also shows how to use advanced features such as manual SDK client

instrumentation, creating additional subsegments, and outgoing HTTP calls. For more information, see AWS X-Ray sample application (p. 134).

Optional: Least privilege policy

You have just deployed this tutorial using the AmazonS3FullAccess and AmazonDynamoDBFullAccess security policies. Using a full access policy isn't the best practice in the long term. To improve the security

(22)

Optional: Least privilege policy

of what you deployed, follow these steps to update your permissions. To learn more about security best practices in IAM policies, see Identity and access management for AWS X-Ray.

To update your policies, ﬁrst you identify the ARNs of your Amazon S3 and DynamoDB resources. Then you use the ARNs in two custom IAM policies. Finally, you apply those policies to your instance proﬁle.

To identify your Amazon S3 resource

1. Open the Resources page of the AWS Conﬁg console.

2. Under Resource type, ﬁlter by Amazon S3 Bucket to ﬁnd the ARN of the Amazon S3 bucket that your application uses.

3. Choose the Resource identiﬁer that's attached to elasticbeanstalk.

4. Record its full Amazon resource name.

5. Insert the ARN into the following IAM policy.

Example

{ "Version": "2012-10-17", "Statement": [

{ "Sid": "ScorekeepS3", "Action": [

"s3:GetObject", "s3:PutObject"

],

"Effect": "Allow",

"Resource": "arn:aws:s3:::elasticbeanstalk-region-0987654321"

} ] }

To identify your DynamoDB resource

1. Open the Resources page of the AWS Conﬁg console.

2. Under Resource type, ﬁlter by AWS DynamoDB Table to ﬁnd the ARN of the DynamoDB tables that your application uses.

3. Choose the Resource identiﬁer that's attached to one of the scorekeep tables.

4. Record its full Amazon resource name.

5. Insert the ARN into the following IAM policy.

Example

{ "Version": "2012-10-17", "Statement": [

{ "Sid": "ScorekeepDynamoDB", "Effect": "Allow",

"Action": [

"dynamodb:PutItem", "dynamodb:UpdateItem", "dynamodb:DeleteItem", "dynamodb:GetItem"

],

"Resource": "arn:aws:dynamodb:region:1234567890:table/scorekeep-*"

}

(23)

Clean up

] }

The tables that the application creates follow a consistent naming convention. You can use the format of scorekeep-* to indicate all tables following that convention.

To change your IAM policy

1. Open the Elastic Beanstalk instance proﬁle in the IAM console: aws-elasticbeanstalk-ec2-role.

2. Remove the AmazonS3FullAccess and AmazonDynamoDBFullAccess policies from the role.

3. Choose Attach policies, and then Create policy.

4. Choose the JSON and paste in one of the policies created previously.

5. Choose Review policy.

6. For Name, assign a name.

7. Choose Create policy.

8. Assign the newly created policy to the aws-elasticbeanstalk-ec2-role.

9. Repeat for the second policy created previously.

Clean up

Terminate your Elastic Beanstalk environment to shut down the Amazon EC2 instances, DynamoDB tables, and other resources.

To terminate your Elastic Beanstalk environment

1. Open the Elastic Beanstalk console.

3. Choose Actions.

4. Choose Terminate Environment.

5. Choose Terminate.

Trace data is automatically deleted from X-Ray after 30 days.

Next steps

Learn more about X-Ray in the next chapter, AWS X-Ray concepts (p. 16).

To instrument your own app, learn more about the X-Ray SDK for Java or one of the other X-Ray SDKs:

• X-Ray SDK for Java – AWS X-Ray SDK for Java (p. 228)

• X-Ray SDK for Node.js – AWS X-Ray SDK for Node.js (p. 265)

• X-Ray SDK for .NET – AWS X-Ray SDK for .NET (p. 305)

To run the X-Ray daemon locally or on AWS, see AWS X-Ray daemon (p. 165).

To contribute to the sample application on GitHub, see eb-java-scorekeep.

(24)

Segments

AWS X-Ray concepts

AWS X-Ray receives data from services as segments. X-Ray then groups segments that have a common request into traces. X-Ray processes the traces to generate a service graph that provides a visual representation of your application.

Concepts

• Segments (p. 16)

• Subsegments (p. 17)

• Service graph (p. 19)

• Traces (p. 20)

• Sampling (p. 21)

• Tracing header (p. 21)

• Filter expressions (p. 22)

• Groups (p. 22)

• Annotations and metadata (p. 23)

• Errors, faults, and exceptions (p. 23)

Segments

The compute resources running your application logic send data about their work as segments. A segment provides the resource's name, details about the request, and details about the work done. For example, when an HTTP request reaches your application, it can record the following data about:

• The host – hostname, alias or IP address

• The request – method, client address, path, user agent

• The response – status, content

• The work done – start and end times, subsegments

• Issues that occur – errors, faults and exceptions (p. 23), including automatic capture of exception stacks.

(25)

Subsegments

The X-Ray SDK gathers information from request and response headers, the code in your application, and metadata about the AWS resources on which it runs. You choose the data to collect by modifying your application conﬁguration or code to instrument incoming requests, downstream requests, and AWS SDK clients.

Forwarded Requests

If a load balancer or other intermediary forwards a request to your application, X-Ray takes the client IP from the X-Forwarded-For header in the request instead of from the source IP in the IP packet. The client IP that is recorded for a forwarded request can be forged, so it should not be trusted.

You can use the X-Ray SDK to record additional information such as annotations and metadata. (p. 23) For details about the structure and information that is recorded in segments and subsegments, see AWS X-Ray segment documents (p. 121). Segment documents can be up to 64 kB in size.

Subsegments

A segment can break down the data about the work done into subsegments. Subsegments provide more granular timing information and details about downstream calls that your application made to fulfill the original request. A subsegment can contain additional details about a call to an AWS service, an external HTTP API, or an SQL database. You can even define arbitrary subsegments to instrument specific functions or lines of code in your application.

(26)

Subsegments

For services that don't send their own segments, like Amazon DynamoDB, X-Ray uses subsegments to generate inferred segments and downstream nodes on the service map. This lets you see all of your downstream dependencies, even if they don't support tracing, or are external.

Subsegments represent your application's view of a downstream call as a client. If the downstream service is also instrumented, the segment that it sends replaces the inferred segment generated from the upstream client's subsegment. The node on the service graph always uses information from the service's segment, if it's available, while the edge between the two nodes uses the upstream service's subsegment.

For example, when you call DynamoDB with an instrumented AWS SDK client, the X-Ray SDK records a subsegment for that call. DynamoDB doesn't send a segment, so the inferred segment in the trace, the DynamoDB node on the service graph, and the edge between your service and DynamoDB all contain information from the subsegment.

When you call another instrumented service with an instrumented application, the downstream service sends its own segment to record its view of the same call that the upstream service recorded in a subsegment. In the service graph, both services' nodes contain timing and error information from those services' segments, while the edge between them contains information from the upstream service's subsegment.

(27)

Service graph

Both viewpoints are useful, as the downstream service records precisely when it started and ended work on the request, and the upstream service records the round trip latency, including time that the request spent traveling between the two services.

Service graph

X-Ray uses the data that your application sends to generate a service graph. Each AWS resource that sends data to X-Ray appears as a service in the graph. Edges connect the services that work together to serve requests. Edges connect clients to your application, and your application to the downstream services and resources that it uses.

Service Names

A segment's name should match the domain name or logical name of the service that generates the segment. However, this is not enforced. Any application that has permission to PutTraceSegments can send segments with any name.

A service graph is a JSON document that contains information about the services and resources that make up your application. The X-Ray console uses the service graph to generate a visualization or service map.

(28)

Traces

For a distributed application, X-Ray combines nodes from all services that process requests with the same trace ID into a single service graph. The ﬁrst service that the request hits adds a tracing header (p. 21) that is propagated between the front end and services that it calls.

For example, Scorekeep (p. 134) runs a web API that calls a microservice (an AWS Lambda function) to generate a random name by using a Node.js library. The X-Ray SDK for Java generates the trace ID and includes it in calls to Lambda. Lambda sends tracing data and passes the trace ID to the function.

The X-Ray SDK for Node.js also uses the trace ID to send data. As a result, nodes for the API, the Lambda service, and the Lambda function all appear as separate, but connected, nodes on the service map.

Service graph data is retained for 30 days.

Traces

A trace ID tracks the path of a request through your application. A trace collects all the segments generated by a single request. That request is typically an HTTP GET or POST request that travels through a load balancer, hits your application code, and generates downstream calls to other AWS services or external web APIs. The ﬁrst supported service that the HTTP request interacts with adds a trace ID header to the request, and propagates it downstream to track the latency, disposition, and other request data.

(29)

Sampling

Service graph data is retained for 30 days.

Sampling

To ensure efficient tracing and provide a representative sample of the requests that your application serves, the X-Ray SDK applies a sampling algorithm to determine which requests get traced. By default, the X-Ray SDK records the first request each second, and five percent of any additional requests.

To avoid incurring service charges when you are getting started, the default sampling rate is

conservative. You can conﬁgure X-Ray to modify the default sampling rule and conﬁgure additional rules that apply sampling based on properties of the service or request.

For example, you might want to disable sampling and trace all requests for calls that modify state or handle user accounts or transactions. For high-volume read-only calls, like background polling, health checks, or connection maintenance, you can sample at a low rate and still get enough data to see any issues that arise.

For more information, see Conﬁguring sampling rules in the X-Ray console (p. 76).

Tracing header

All requests are traced, up to a conﬁgurable minimum. After reaching that minimum, a percentage of requests are traced to avoid unnecessary cost. The sampling decision and trace ID are added to HTTP requests in tracing headers named X-Amzn-Trace-Id. The ﬁrst X-Ray-integrated service that the request hits adds a tracing header, which is read by the X-Ray SDK and included in the response.

Example Tracing header with root trace ID and sampling decision

X-Amzn-Trace-Id: Root=1-5759e988-bd862e3fe1be46a994272793;Sampled=1

(30)

Filter expressions

Tracing Header Security

A tracing header can originate from the X-Ray SDK, an AWS service, or the client request. Your application can remove X-Amzn-Trace-Id from incoming requests to avoid issues caused by users adding trace IDs or sampling decisions to their requests.

The tracing header can also contain a parent segment ID if the request originated from an instrumented application. For example, if your application calls a downstream HTTP web API with an instrumented HTTP client, the X-Ray SDK adds the segment ID for the original request to the tracing header of the downstream request. An instrumented application that serves the downstream request can record the parent segment ID to connect the two requests.

Example Tracing header with root trace ID, parent segment ID and sampling decision

X-Amzn-Trace-Id: Root=1-5759e988-bd862e3fe1be46a994272793;Parent=53995c3f42cd8ad8;Sampled=1

Filter expressions

Even with sampling, a complex application generates a lot of data. The AWS X-Ray console provides an easy-to-navigate view of the service graph. It shows health and performance information that helps you identify issues and opportunities for optimization in your application. For advanced tracing, you can drill down to traces for individual requests, or use filter expressions to find traces related to specific paths or users.

Groups

Extending filter expressions, X-Ray also supports the group feature. Using a filter expression, you can define criteria by which to accept traces into the group.

You can call the group by name or by Amazon Resource Name (ARN) to generate its own service graph, trace summaries, and Amazon CloudWatch metrics. Once a group is created, incoming traces are checked against the group’s ﬁlter expression as they are stored in the X-Ray service. Metrics for the number of traces matching each criteria are published to CloudWatch every minute.

Updating a group's ﬁlter expression doesn't change data that's already recorded. The update applies only to subsequent traces. This can result in a merged graph of the new and old expressions. To avoid this, delete the current group and create a fresh one.

Note

Groups are billed by the number of retrieved traces that match the ﬁlter expression. For more information, see AWS X-Ray pricing.

For more information about groups, see Conﬁguring groups in the X-Ray console (p. 90).

(31)

Annotations and metadata

When you instrument your application, the X-Ray SDK records information about incoming and outgoing requests, the AWS resources used, and the application itself. You can add other information to the segment document as annotations and metadata. Annotations and metadata are aggregated at the trace level, and can be added to any segment or subsegment.

Annotations are simple key-value pairs that are indexed for use with ﬁlter expressions (p. 64). Use annotations to record data that you want to use to group traces in the console, or when calling the GetTraceSummaries API.

X-Ray indexes up to 50 annotations per trace.

Metadata are key-value pairs with values of any type, including objects and lists, but that are not indexed. Use metadata to record data you want to store in the trace but don't need to use for searching traces.

You can view annotations and metadata in the segment or subsegment details in the X-Ray console.

Errors, faults, and exceptions

X-Ray tracks errors that occur in your application code, and errors that are returned by downstream services. Errors are categorized as follows.

• Error – Client errors (400 series errors)

• Fault – Server faults (500 series errors)

• Throttle – Throttling errors (429 Too Many Requests)

(32)

Errors, faults, and exceptions

When an exception occurs while your application is serving an instrumented request, the X-Ray SDK records details about the exception, including the stack trace, if available. You can view exceptions under segment details (p. 60) in the X-Ray console.

(33)

Data protection

Security in AWS X-Ray

Cloud security at AWS is the highest priority. As an AWS customer, you beneﬁt from a data center and network architecture that is built to meet the requirements of the most security-sensitive organizations.

Security is a shared responsibility between AWS and you. The shared responsibility model describes this as security of the cloud and security in the cloud:

• Security of the cloud – AWS is responsible for protecting the infrastructure that runs AWS services in the AWS Cloud. AWS also provides you with services that you can use securely. The eﬀectiveness of our security is regularly tested and veriﬁed by third-party auditors as part of the AWS compliance programs. To learn about the compliance programs that apply to X-Ray, see AWS Services in Scope by Compliance Program.

• Security in the cloud – Your responsibility is determined by the AWS service that you use. You are also responsible for other factors including the sensitivity of your data, your organization’s requirements, and applicable laws and regulations.

This documentation will help you understand how to apply the shared responsibility model when using X-Ray. The following topics show you how to conﬁgure X-Ray to meet your security and compliance objectives. You'll also learn how to use other AWS services that can help you to monitor and secure your X-Ray resources.

Topics

• Data protection in AWS X-Ray (p. 25)

• Identity and access management for AWS X-Ray (p. 27)

• Compliance validation for AWS X-Ray (p. 44)

• Resilience in AWS X-Ray (p. 44)

• Infrastructure security in AWS X-Ray (p. 45)

Data protection in AWS X-Ray

AWS X-Ray always encrypts traces and related data at rest. When you need to audit and disable encryption keys for compliance or internal requirements, you can conﬁgure X-Ray to use an AWS Key Management Service (AWS KMS) key to encrypt data.

X-Ray provides an AWS managed key named aws/xray. Use this key when you just want to audit key usage in AWS CloudTrail and don't need to manage the key itself. When you need to manage access to the key or conﬁgure key rotation, you can create a customer managed key.

When you change encryption settings, X-Ray spends some time generating and propagating data keys.

While the new key is being processed, X-Ray may encrypt data with a combination of the new and old settings. Existing data is not re-encrypted when you change encryption settings.

Note

AWS KMS charges when X-Ray uses a KMS key to encrypt or decrypt trace data.

• Default encryption – Free.

(34)

Data protection

• AWS managed key – Pay for key use.

• customer managed key – Pay for key storage and use.

See AWS Key Management Service Pricing for details.

Note

X-Ray insights notiﬁcations sends events to Amazon EventBridge, which does not currently support customer managed keys. For more information, see Data Protection in Amazon EventBridge.

You must have user-level access to a customer managed key to conﬁgure X-Ray to use it, and to then view encrypted traces. See User permissions for encryption (p. 35) for more information.

X-Ray console

To conﬁgure X-Ray to use a KMS key for encryption using the X-Ray console

1. Open the X-Ray console.

2. Choose Encryption.

3. Choose Use a KMS key.

4. Choose a key from the dropdown menu:

• aws/xray – Use the AWS managed key.

• key alias – Use a customer managed key in your account.

• Manually enter a key ARN – Use a customer managed key in a diﬀerent account. Enter the full Amazon Resource Name (ARN) of the key in the ﬁeld that appears.

5. Choose Apply.

CloudWatch console

To conﬁgure X-Ray to use a KMS key for encryption using the CloudWatch console

1. Sign in to the AWS Management Console and open the CloudWatch console at https://

console.aws.amazon.com/cloudwatch/.

2. Choose Settings in the left navigation pane.

3. Choose View settings under Encryption within the X-Ray traces section.

4. Choose Edit in the Encryption conﬁguration section.

5. Choose Use a KMS key.

6. Choose a key from the dropdown menu:

• aws/xray – Use the AWS managed key.

• key alias – Use a customer managed key in your account.

• Manually enter a key ARN – Use a customer managed key in a diﬀerent account. Enter the full Amazon Resource Name (ARN) of the key in the ﬁeld that appears.

7. Choose Update encryption.

Note

X-Ray does not support asymmetric KMS keys.

If X-Ray is unable to access your encryption key, it stops storing data. This can happen if your user loses access to the KMS key, or if you disable a key that's currently in use. When this happens, X-Ray shows a notiﬁcation in the navigation bar.

(35)

Identity and access management

To conﬁgure encryption settings with the X-Ray API, see Conﬁguring sampling, groups, and encryption settings with the AWS X-Ray API (p. 113).

Identity and access management for AWS X-Ray

AWS Identity and Access Management (IAM) is an AWS service that helps an administrator securely control access to AWS resources. IAM administrators control who can be authenticated (signed in) and authorized (have permissions) to use X-Ray resources. IAM is an AWS service that you can use with no additional charge.

Topics

• Audience (p. 27)

• Authenticating with identities (p. 27)

• Managing access using policies (p. 29)

• How AWS X-Ray works with IAM (p. 31)

• AWS X-Ray identity-based policy examples (p. 35)

• Troubleshooting AWS X-Ray identity and access (p. 42)

Audience

How you use AWS Identity and Access Management (IAM) diﬀers, depending on the work that you do in X-Ray.

Service user – If you use the X-Ray service to do your job, then your administrator provides you with the credentials and permissions that you need. As you use more X-Ray features to do your work, you might need additional permissions. Understanding how access is managed can help you request the right permissions from your administrator. If you cannot access a feature in X-Ray, see Troubleshooting AWS X- Ray identity and access (p. 42).

Service administrator – If you're in charge of X-Ray resources at your company, you probably have full access to X-Ray. It's your job to determine which X-Ray features and resources your employees should access. You must then submit requests to your IAM administrator to change the permissions of your service users. Review the information on this page to understand the basic concepts of IAM. To learn more about how your company can use IAM with X-Ray, see How AWS X-Ray works with IAM (p. 31).

IAM administrator – If you're an IAM administrator, you might want to learn details about how you can write policies to manage access to X-Ray. To view example X-Ray identity-based policies that you can use in IAM, see AWS X-Ray identity-based policy examples (p. 35).

Authenticating with identities

Authentication is how you sign in to AWS using your identity credentials. For more information about signing in using the AWS Management Console, see Signing in to the AWS Management Console as an IAM user or root user in the IAM User Guide.

You must be authenticated (signed in to AWS) as the AWS account root user, an IAM user, or by assuming an IAM role. You can also use your company's single sign-on authentication or even sign in using Google or Facebook. In these cases, your administrator previously set up identity federation using IAM roles.

When you access AWS using credentials from another company, you are assuming a role indirectly.

To sign in directly to the AWS Management Console, use your password with your root user email address or your IAM user name. You can access AWS programmatically using your root user or IAM

(36)

Authenticating with identities

users access keys. AWS provides SDK and command line tools to cryptographically sign your request using your credentials. If you don't use AWS tools, you must sign the request yourself. Do this using Signature Version 4, a protocol for authenticating inbound API requests. For more information about authenticating requests, see Signature Version 4 signing process in the AWS General Reference.

Regardless of the authentication method that you use, you might also be required to provide additional security information. For example, AWS recommends that you use multi-factor authentication (MFA) to increase the security of your account. To learn more, see Using multi-factor authentication (MFA) in AWS in the IAM User Guide.

AWS account root user

When you ﬁrst create an AWS account, you begin with a single sign-in identity that has complete access to all AWS services and resources in the account. This identity is called the AWS account root user and is accessed by signing in with the email address and password that you used to create the account. We strongly recommend that you do not use the root user for your everyday tasks, even the administrative ones. Instead, adhere to the best practice of using the root user only to create your ﬁrst IAM user. Then securely lock away the root user credentials and use them to perform only a few account and service management tasks.

IAM users and groups

An IAM user is an identity within your AWS account that has speciﬁc permissions for a single person or application. An IAM user can have long-term credentials such as a user name and password or a set of access keys. To learn how to generate access keys, see Managing access keys for IAM users in the IAM User Guide. When you generate access keys for an IAM user, make sure you view and securely save the key pair. You cannot recover the secret access key in the future. Instead, you must generate a new access key pair.

An IAM group is an identity that speciﬁes a collection of IAM users. You can't sign in as a group. You can use groups to specify permissions for multiple users at a time. Groups make permissions easier to manage for large sets of users. For example, you could have a group named IAMAdmins and give that group permissions to administer IAM resources.

Users are diﬀerent from roles. A user is uniquely associated with one person or application, but a role is intended to be assumable by anyone who needs it. Users have permanent long-term credentials, but roles provide temporary credentials. To learn more, see When to create an IAM user (instead of a role) in the IAM User Guide.

IAM roles

An IAM role is an identity within your AWS account that has speciﬁc permissions. It is similar to an IAM user, but is not associated with a speciﬁc person. You can temporarily assume an IAM role in the AWS Management Console by switching roles. You can assume a role by calling an AWS CLI or AWS API operation or by using a custom URL. For more information about methods for using roles, see Using IAM roles in the IAM User Guide.

IAM roles with temporary credentials are useful in the following situations:

• Temporary IAM user permissions – An IAM user can assume an IAM role to temporarily take on diﬀerent permissions for a speciﬁc task.

• Federated user access – Instead of creating an IAM user, you can use existing identities from AWS Directory Service, your enterprise user directory, or a web identity provider. These are known as federated users. AWS assigns a role to a federated user when access is requested through an identity provider. For more information about federated users, see Federated users and roles in the IAM User Guide.

(37)

Managing access using policies

• Cross-account access – You can use an IAM role to allow someone (a trusted principal) in a diﬀerent account to access resources in your account. Roles are the primary way to grant cross-account access.

However, with some AWS services, you can attach a policy directly to a resource (instead of using a role as a proxy). To learn the diﬀerence between roles and resource-based policies for cross-account access, see How IAM roles diﬀer from resource-based policies in the IAM User Guide.

• Cross-service access – Some AWS services use features in other AWS services. For example, when you make a call in a service, it's common for that service to run applications in Amazon EC2 or store objects in Amazon S3. A service might do this using the calling principal's permissions, using a service role, or using a service-linked role.

• Principal permissions – When you use an IAM user or role to perform actions in AWS, you are considered a principal. Policies grant permissions to a principal. When you use some services, you might perform an action that then triggers another action in a diﬀerent service. In this case, you must have permissions to perform both actions. To see whether an action requires additional dependent actions in a policy, see Actions, Resources, and Condition Keys for AWS X-Ray in the Service Authorization Reference.

• Service role – A service role is an IAM role that a service assumes to perform actions on your behalf.

An IAM administrator can create, modify, and delete a service role from within IAM. For more information, see Creating a role to delegate permissions to an AWS service in the IAM User Guide.

• Service-linked role – A service-linked role is a type of service role that is linked to an AWS service.

The service can assume the role to perform an action on your behalf. Service-linked roles appear in your IAM account and are owned by the service. An IAM administrator can view, but not edit the permissions for service-linked roles.

• Applications running on Amazon EC2 – You can use an IAM role to manage temporary credentials for applications that are running on an EC2 instance and making AWS CLI or AWS API requests.

This is preferable to storing access keys within the EC2 instance. To assign an AWS role to an EC2 instance and make it available to all of its applications, you create an instance proﬁle that is attached to the instance. An instance proﬁle contains the role and enables programs that are running on the EC2 instance to get temporary credentials. For more information, see Using an IAM role to grant permissions to applications running on Amazon EC2 instances in the IAM User Guide.

To learn whether to use IAM roles or IAM users, see When to create an IAM role (instead of a user) in the IAM User Guide.

Managing access using policies

You control access in AWS by creating policies and attaching them to IAM identities or AWS resources. A policy is an object in AWS that, when associated with an identity or resource, deﬁnes their permissions.

You can sign in as the root user or an IAM user, or you can assume an IAM role. When you then make a request, AWS evaluates the related identity-based or resource-based policies. Permissions in the policies determine whether the request is allowed or denied. Most policies are stored in AWS as JSON documents. For more information about the structure and contents of JSON policy documents, see Overview of JSON policies in the IAM User Guide.

Administrators can use AWS JSON policies to specify who has access to what. That is, which principal can perform actions on what resources, and under what conditions.

Every IAM entity (user or role) starts with no permissions. In other words, by default, users can do nothing, not even change their own password. To give a user permission to do something, an administrator must attach a permissions policy to a user. Or the administrator can add the user to a group that has the intended permissions. When an administrator gives permissions to a group, all users in that group are granted those permissions.

IAM policies deﬁne permissions for an action regardless of the method that you use to perform the operation. For example, suppose that you have a policy that allows the iam:GetRole action. A user with that policy can get role information from the AWS Management Console, the AWS CLI, or the AWS API.

(38)

Managing access using policies

Identity-based policies

Identity-based policies are JSON permissions policy documents that you can attach to an identity, such as an IAM user, group of users, or role. These policies control what actions users and roles can perform, on which resources, and under what conditions. To learn how to create an identity-based policy, see Creating IAM policies in the IAM User Guide.

Identity-based policies can be further categorized as inline policies or managed policies. Inline policies are embedded directly into a single user, group, or role. Managed policies are standalone policies that you can attach to multiple users, groups, and roles in your AWS account. Managed policies include AWS managed policies and customer managed policies. To learn how to choose between a managed policy or an inline policy, see Choosing between managed policies and inline policies in the IAM User Guide.

Resource-based policies

Resource-based policies are JSON policy documents that you attach to a resource. Examples of resource- based policies are IAM role trust policies and Amazon S3 bucket policies. In services that support resource- based policies, service administrators can use them to control access to a specific resource. For the resource where the policy is attached, the policy defines what actions a specified principal can perform on that resource and under what conditions. You must specify a principal in a resource-based policy.

Principals can include accounts, users, roles, federated users, or AWS services.

Resource-based policies are inline policies that are located in that service. You can't use AWS managed policies from IAM in a resource-based policy.

Access control lists (ACLs)

Access control lists (ACLs) control which principals (account members, users, or roles) have permissions to access a resource. ACLs are similar to resource-based policies, although they do not use the JSON policy document format.

Amazon S3, AWS WAF, and Amazon VPC are examples of services that support ACLs. To learn more about ACLs, see Access control list (ACL) overview in the Amazon Simple Storage Service Developer Guide.

Other policy types

AWS supports additional, less-common policy types. These policy types can set the maximum permissions granted to you by the more common policy types.

• Permissions boundaries – A permissions boundary is an advanced feature in which you set the maximum permissions that an identity-based policy can grant to an IAM entity (IAM user or role).

You can set a permissions boundary for an entity. The resulting permissions are the intersection of entity's identity-based policies and its permissions boundaries. Resource-based policies that specify the user or role in the Principal ﬁeld are not limited by the permissions boundary. An explicit deny in any of these policies overrides the allow. For more information about permissions boundaries, see Permissions boundaries for IAM entities in the IAM User Guide.

• Service control policies (SCPs) – SCPs are JSON policies that specify the maximum permissions for an organization or organizational unit (OU) in AWS Organizations. AWS Organizations is a service for grouping and centrally managing multiple AWS accounts that your business owns. If you enable all features in an organization, then you can apply service control policies (SCPs) to any or all of your accounts. The SCP limits permissions for entities in member accounts, including each AWS account root user. For more information about Organizations and SCPs, see How SCPs work in the AWS Organizations User Guide.

• Session policies – Session policies are advanced policies that you pass as a parameter when you programmatically create a temporary session for a role or federated user. The resulting session's permissions are the intersection of the user or role's identity-based policies and the session policies.

(39)

How AWS X-Ray works with IAM

Permissions can also come from a resource-based policy. An explicit deny in any of these policies overrides the allow. For more information, see Session policies in the IAM User Guide.

Multiple policy types

When multiple types of policies apply to a request, the resulting permissions are more complicated to understand. To learn how AWS determines whether to allow a request when multiple policy types are involved, see Policy evaluation logic in the IAM User Guide.

How AWS X-Ray works with IAM

Before you use IAM to manage access to X-Ray, you should understand what IAM features are available to use with X-Ray. To get a high-level view of how X-Ray and other AWS services work with IAM, see AWS Services That Work with IAM in the IAM User Guide.

You can use AWS Identity and Access Management (IAM) to grant X-Ray permissions to users and compute resources in your account. IAM controls access to the X-Ray service at the API level to enforce permissions uniformly, regardless of which client (console, AWS SDK, AWS CLI) your users employ.

To use the X-Ray console (p. 49) to view service maps and segments, you only need read permissions.

To enable console access, add the AWSXrayReadOnlyAccess managed policy (p. 39) to your IAM user.

For local development and testing (p. 34), create an IAM user with read and write permissions.

Generate access keys for the user and store them in the standard AWS SDK location. You can use these credentials with the X-Ray daemon, the AWS CLI, and the AWS SDK.

To deploy your instrumented app to AWS (p. 34), create an IAM role with write permissions and assign it to the resources running your application. AWSXRayDaemonWriteAccess includes permission to upload traces, and some read permissions as well to support the use of sampling rules (p. 76).

The read and write policies do not include permission to conﬁgure encryption key settings (p. 25) and sampling rules. Use AWSXrayFullAccess to access these settings, or add conﬁguration APIs (p. 113) in a custom policy. For encryption and decryption with a customer managed key that you create, you also need permission to use the key (p. 35).

Topics

• X-Ray identity-based policies (p. 31)

• X-Ray resource-based policies (p. 34)

• Authorization based on X-Ray tags (p. 34)

• Running your application locally (p. 34)

• Running your application in AWS (p. 34)

• User permissions for encryption (p. 35)

X-Ray identity-based policies

With IAM identity-based policies, you can specify allowed or denied actions and resources as well as the conditions under which actions are allowed or denied. X-Ray supports speciﬁc actions, resources, and condition keys. To learn about all of the elements that you use in a JSON policy, see IAM JSON Policy Elements Reference in the IAM User Guide.

Actions

Administrators can use AWS JSON policies to specify who has access to what. That is, which principal can perform actions on what resources, and under what conditions.

AWS X-Ray

AWS X-Ray

Developer Guide

AWS X-Ray: Developer Guide

Table of Contents

What is AWS X-Ray?

Getting started with AWS X-Ray

Prerequisites

To create an Elastic Beanstalk environment

Note

To add X-Ray, Amazon S3 and DynamoDB permissions to your Elastic Beanstalk environment

Note

Deploy to Elastic Beanstalk and generate trace data

To deploy the source code

To generate trace data

View the service map in the X-Ray console

To use the X-Ray console

To ﬁnd the cause of the error

Conﬁguring Amazon SNS notiﬁcations

To conﬁgure Amazon SNS notiﬁcations for scorekeep

Note

Explore the sample application

Example src/main/java/scorekeep/WebConﬁg.java - servlet ﬁlter

Example Buildﬁle

Example Procﬁle

Example build.gradle -- dependencies

Example .ebextensions/xray.conﬁg

Example src/main/java/scorekeep/WebConﬁg.java - recorder

Example src/main/java/resources/sampling-rules.json

Optional: Least privilege policy

To identify your Amazon S3 resource

Example

To identify your DynamoDB resource

Example

To change your IAM policy

Clean up

To terminate your Elastic Beanstalk environment

Next steps

AWS X-Ray concepts

Segments

Forwarded Requests

Subsegments

Service graph

Service Names

Traces

Sampling

Tracing header

Example Tracing header with root trace ID and sampling decision

Tracing Header Security

Example Tracing header with root trace ID, parent segment ID and sampling decision

Filter expressions

Groups

Note

Annotations and metadata

Errors, faults, and exceptions

Security in AWS X-Ray

Topics

Data protection in AWS X-Ray

Note

Note

To conﬁgure X-Ray to use a KMS key for encryption using the X-Ray console

To conﬁgure X-Ray to use a KMS key for encryption using the CloudWatch console

Note

Identity and access management for AWS X-Ray

Audience

Authenticating with identities

AWS account root user

IAM users and groups

IAM roles

Managing access using policies

Identity-based policies

Resource-based policies

Access control lists (ACLs)

Other policy types

Multiple policy types

How AWS X-Ray works with IAM

X-Ray identity-based policies

Actions