"virtualRouterName" : "serviceB"
}
b. Update the route with the update-route command.
aws appmesh update-route --cli-input-json file://update-route.json 5. Create the serviceA virtual service.
a. Create a file named create-virtual-servicea.json with the following contents:
{ "meshName" : "apps", "spec" : {
"provider" : { "virtualNode" : {
"virtualNodeName" : "serviceA"
} } },
"virtualServiceName" : "servicea.apps.local"
}
b. Create the virtual service.
aws appmesh create-virtual-service --cli-input-json file://create-virtual-servicea.json
Mesh summary
Before you created the service mesh, you had three actual services named servicea.apps.local, serviceb.apps.local, and servicebv2.apps.local. In addition to the actual services, you now have a service mesh that contains the following resources that represent the actual services:
• Two virtual services. The proxy sends all traffic from the servicea.apps.local virtual service to the serviceb.apps.local virtual service through a virtual router.
• Three virtual nodes named serviceA, serviceB, and serviceBv2. The Envoy proxy uses the service discovery information configured for the virtual nodes to look up the IP addresses of the actual services.
• One virtual router with one route that instructs the Envoy proxy to route 75 percent of inbound traffic to the serviceB virtual node and 25 percent of the traffic to the serviceBv2 virtual node.
Step 6: Update services
After creating your mesh, you need to complete the following tasks:
• Authorize the Envoy proxy that you deploy with each Amazon ECS task to read the configuration of one or more virtual nodes. For more information about how to authorize the proxy, see Proxy authorization.
• Update each of your existing Amazon ECS task definitions to use the Envoy proxy.
Credentials
The Envoy container requires AWS Identity and Access Management credentials for signing requests that are sent to the App Mesh service. For Amazon ECS tasks deployed with the Amazon EC2 launch type, the credentials can come from the instance role or from a task IAM role. Amazon ECS tasks deployed with
Step 6: Update services
the Fargate launch type don't have access to the Amazon EC2 metadata server that supplies instance IAM profile credentials. To supply the credentials, you must attach an IAM task role to any tasks deployed with the Fargate launch type.
If a task is deployed with the Amazon EC2 launch type and access is blocked to the Amazon EC2 metadata server, as described in the Important annotation in IAM Role for Tasks, then a task IAM role must also be attached to the task. The role that you assign to the instance or task must have an IAM policy attached to it as described in Proxy authorization.
To update your task definitions using the AWS Management Console
The following steps only show updating the taskB task for the scenario. You also need to update the taskBv2 and taskA tasks by changing the values appropriately.
1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.
2. From the navigation bar, choose the Region that contains your task definition.
3. In the navigation pane, choose Task Definitions.
4. On the Task Definitions page, select the box to the left of the task definition to revise. From the pre-requisites and previous steps, you might have task definitions named taskA, taskB, and taskBv2. Select taskB and choose Create new revision.
5. On the Create new revision of Task Definition page, make the following changes to enable App Mesh integration.
a. For Service Integration, to configure the parameters for App Mesh integration choose Enable App Mesh integration and then do the following:
i. For Application container name, choose the container name to use for the App Mesh application. This container must already be defined within the task definition.
ii. For Envoy image, complete the following task and enter the value that is returned.
• All supported Regions other than me-south-1 and ap-east-1, eu-south-1, af-south-1. You can replace Region-code with any Region other than me-south-1 and ap-east-1, eu-south-1, af-south-1.
840364872350.dkr.ecr.region-code.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• me-south-1 Region:
772975370895.dkr.ecr.me-south-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• ap-east-1 Region:
856666278305.dkr.ecr.ap-east-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• eu-south-1 Region:
422531588944.dkr.ecr.eu-south-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• af-south-1 Region:
924023996002.dkr.ecr.af-south-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• Public repository
Step 6: Update services
public.ecr.aws/appmesh/aws-appmesh-envoy:v1.21.0.0-prod
Important
Only version v1.9.0.0-prod or later is supported for use with App Mesh.
iii. For Mesh name, choose the App Mesh service mesh to use. In this topic, the name of the mesh that was created is apps.
iv. For Virtual node name, choose the App Mesh virtual node to use. For example, for the taskB task, you would choose the serviceB virtual node that you created in a previous step.
v. The value for Virtual node port is pre-populated with the listener port that you specified when you created the virtual node.
vi. Choose Apply, and then choose Confirm. A new Envoy proxy container is created and added to the task definition, and the settings to support the container are also created. The Envoy proxy container then pre-populates the App Mesh Proxy Configuration settings for the next step.
b. For Proxy Configuration, verify all of the pre-populated values.
c. For Network Mode, make sure that awsvpc is selected. To learn more about the awsvpc network mode, see Task Networking with the awsvpc Network Mode.
6. Choose Create.
7. Update your service with the updated task definition. For more information, see Updating a service.
The console creates the task definition's json specification. You can modify some of the settings, but not others. For more information, expand the following section.
Task definition json
Proxy configuration
To configure your Amazon ECS service to use App Mesh, your service's task definition must have the following proxy configuration section. Set the proxy configuration type to APPMESH and the containerName to envoy. Set the following property values accordingly.
IgnoredUID
The Envoy proxy doesn't route traffic from processes that use this user ID. You can choose any user ID that you want for this property value, but this ID must be the same as the user ID for the Envoy container in your task definition. This matching allows Envoy to ignore its own traffic without using the proxy. Our examples use 1337 for historical purposes.
ProxyIngressPort
This is the inbound port for the Envoy proxy container. Set this value to 15000.
ProxyEgressPort
This is the outbound port for the Envoy proxy container. Set this value to 15001.
AppPorts
Specify any inbound ports that your application containers listen on. In this example, the application container listens on port 9080. The port that you specify must match the port configured on the virtual node listener.
EgressIgnoredIPs
Envoy doesn't proxy traffic to these IP addresses. Set this value to
169.254.170.2,169.254.169.254, which ignores the Amazon EC2 metadata server and
Step 6: Update services
the Amazon ECS task metadata endpoint. The metadata endpoint provides IAM roles for tasks credentials. You can add additional addresses.
EgressIgnoredPorts
You can add a comma separated list of ports. Envoy doesn't proxy traffic to these ports. Even if you list no ports, port 22 is ignored.
Note
The maximum number of outbound ports that can be ignored is 15.
"proxyConfiguration": {
{ "name": "EgressIgnoredPorts", "value": "22"
} ] }
Application container Envoy dependency
The application containers in your task definitions must wait for the Envoy proxy to bootstrap and start before they can start. To make sure this happens, you set a dependsOn section in each application container definition to wait for the Envoy container to report as HEALTHY. The following code shows an application container definition example with this dependency. All of the properties in the following example are required. Some of the property values are also required, but some are replaceable.
{
Step 6: Update services
}
Envoy container definition
Your Amazon ECS task definitions must contain an App Mesh Envoy container image.
• All supported Regions other than me-south-1 and ap-east-1, eu-south-1, af-south-1. You can replace Region-code with any Region other than me-south-1 and ap-east-1, eu-south-1, af-south-1.
840364872350.dkr.ecr.region-code.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• me-south-1 Region:
772975370895.dkr.ecr.me-south-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• ap-east-1 Region:
856666278305.dkr.ecr.ap-east-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• eu-south-1 Region:
422531588944.dkr.ecr.eu-south-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• af-south-1 Region:
924023996002.dkr.ecr.af-south-1.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod
• Public repository
public.ecr.aws/appmesh/aws-appmesh-envoy:v1.21.0.0-prod
Important
Only version v1.9.0.0-prod or later is supported for use with App Mesh.
You must use the App Mesh Envoy container image until the Envoy project team merges changes that support App Mesh. For additional details, see the GitHub roadmap issue.
All of the properties in the following example are required. Some of the property values are also required, but some are replaceable.
Note
• The Envoy container definition must be marked as essential.
• We recommend allocating 512 CPU units and at least 64 MiB of memory to the Envoy container. On Fargate the lowest you will be able to set is 1024 MiB of memory.
• The virtual node name for the Amazon ECS service must be set to the value of the APPMESH_RESOURCE_ARN property. This property requires version 1.15.0 or later of the Envoy image. For more information, see Envoy (p. 121).
• The value for the user setting must match the IgnoredUID value from the task definition proxy configuration. In this example, we use 1337.
• The health check shown here waits for the Envoy container to bootstrap properly before reporting to Amazon ECS that the Envoy container is healthy and ready for the application containers to start.
• By default, App Mesh uses the name of the resource you specified in
APPMESH_RESOURCE_ARN when Envoy is referring to itself in metrics and traces. You can
Step 6: Update services
override this behavior by setting the APPMESH_RESOURCE_CLUSTER environment variable with your own name. This property requires version 1.15.0 or later of the Envoy image. For more information, see Envoy (p. 121).
The following code shows an Envoy container definition example.
{
"curl -s http://localhost:9901/server_info | grep state | grep -q LIVE"
],
The following example Amazon ECS task definitions show how to merge the examples from above into a task definition for taskB. Examples are provided for creating tasks for both Amazon ECS launch types with or without using AWS X-Ray. Change the replaceable values, as appropriate, to create task definitions for the tasks named taskBv2 and taskA from the scenario. Substitute your mesh name and virtual node name for the APPMESH_RESOURCE_ARN value and a list of ports that your application listens on for the proxy configuration AppPorts value. By default, App Mesh uses the name of the resource you specified in APPMESH_RESOURCE_ARN when Envoy is referring to itself in metrics and traces. You can override this behavior by setting the APPMESH_RESOURCE_CLUSTER environment variable with your own name. All of the properties in the following examples are required. Some of the property values are also required, but some are replaceable.
If you're running an Amazon ECS task as described in the Credentials section, then you need to add an existing task IAM role, to the examples.
Important
Fargate must use a port value greater than 1024.
Example JSON for Amazon ECS task definition - Fargate launch type
{
"family" : "taskB", "memory" : "1024", "cpu" : "0.5 vCPU", "proxyConfiguration" : { "containerName" : "envoy",
Step 6: Update services
"containerDefinitions" : [ {
"image" : "840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod",
"requiresCompatibilities" : [ "FARGATE" ],
Step 6: Update services
"taskRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskRole",
"executionRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskExecutionRole", "networkMode" : "awsvpc"
}
Example JSON for Amazon ECS task definition with AWS X-Ray - Fargate launch type X-Ray allows you to collect data about requests that an application serves and provides tools that you can use to visualize traffic flow. Using the X-Ray driver for Envoy enables Envoy to report tracing information to X-Ray. You can enable X-Ray tracing using the Envoy configuration. Based on the configuration, Envoy sends tracing data to the X-Ray daemon running as a sidecar container and the daemon forwards the traces to the X-Ray service. Once the traces are published to X-Ray, you can use the X-Ray console to visualize the service call graph and request trace details. The following JSON represents a task definition to enable X-Ray integration.
{
"family" : "taskB", "memory" : "1024", "cpu" : "512",
"proxyConfiguration" : { "containerName" : "envoy",
"containerDefinitions" : [ {
Step 6: Update services
"image" : "840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod",
"requiresCompatibilities" : [ "FARGATE" ],
"taskRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskRole",
"executionRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskExecutionRole", "networkMode" : "awsvpc"
}
Example JSON for Amazon ECS task definition - EC2 launch type
{ "family": "taskB", "memory": "256",
"proxyConfiguration": { "type": "APPMESH", "containerName": "envoy", "properties": [
Step 6: Update services
"value": "169.254.170.2,169.254.169.254"
}, {
"name": "EgressIgnoredPorts", "value": "22"
"image": "840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod",
"essential": true, "environment": [ {
"name": "APPMESH_VIRTUAL_NODE_NAME", "value": "mesh/apps/virtualNode/serviceB"
}
Step 6: Update services
"user": "1337"
}
], "requiresCompatibilities" : [ "EC2" ],
"taskRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskRole",
"executionRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskExecutionRole", "networkMode": "awsvpc"
}
Example JSON for Amazon ECS task definition with AWS X-Ray - EC2 launch type
{
"family": "taskB", "memory": "256", "cpu" : "1024",
"proxyConfiguration": { "type": "APPMESH", "containerName": "envoy", "properties": [
"value": "169.254.170.2,169.254.169.254"
}, {
"name": "EgressIgnoredPorts", "value": "22"
Advanced topics
{
"name": "envoy",
"image": "840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.21.0.0-prod",
"essential": true, "environment": [ {
"name": "APPMESH_VIRTUAL_NODE_NAME", "value": "mesh/apps/virtualNode/serviceB"
}, {
"name": "ENABLE_ENVOY_XRAY_TRACING", "value": "1"
"image": "amazon/aws-xray-daemon", "user": "1337",
"essential": true, "cpu": 32,
"memoryReservation": 256, "portMappings": [
], "requiresCompatibilities" : [ "EC2" ],
"taskRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskRole",
Canary deployments and releases help you switch traffic between an old version of an application and a newly deployed version. It also monitors the health of the newly deployed version. If there are any problems with the new version, the canary deployment can automatically switch traffic back to the old version. Canary deployments give you the ability to switch traffic between application versions with more control.
For more information about how to implement canary deployments for Amazon ECS using App Mesh, see Create a pipeline with canary deployments for Amazon ECS using App Mesh
App Mesh and Kubernetes
Getting started with AWS App Mesh and Kubernetes
When you integrate AWS App Mesh with Kubernetes using the App Mesh controller for Kubernetes, you manage App Mesh resources, such as meshes, virtual services, virtual nodes, virtual routers, and routes through Kubernetes. You also automatically add the App Mesh sidecar container images to Kubernetes pod specifications. This tutorial guides you through the installation of the App Mesh controller for Kubernetes to enable this integration.
The controller is accompanied by the deployment of the following Kubernetes custom resource definitions: meshes, virtual services, virtual nodes, and virtual routers. The controller watches for creation, modification, and deletion of the custom resources and makes changes to the corresponding App Mesh the section called “Meshes” (p. 57), the section called
“Virtual services” (p. 57), the section called “Virtual nodes” (p. 64), the section called “Virtual gateways” (p. 58), the section called “Gateway routes” (p. 62), the section called “Virtual
routers” (p. 69) (including the section called “Routes” (p. 70)) resources through the App Mesh API.
To learn more or contribute to the controller, see the GitHub project.
The controller also installs a webhook that injects the following containers into Kubernetes pods that are labeled with a name that you specify.
• App Mesh Envoy proxy – Envoy uses the configuration defined in the App Mesh control plane to determine where to send your application traffic.
• App Mesh proxy route manager – Updates iptables rules in a pod's network namespace that route inbound and outbound traffic through Envoy. This container runs as a Kubernetes init container inside of the pod.
Prerequisites
• An existing understanding of App Mesh concepts. For more information, see What Is AWS App Mesh? (p. 1).
• An existing understanding of Kubernetes concepts. For more information, see What is Kubernetes in the Kubernetes documentation.
• An existing Kubernetes cluster. If you don't have an existing cluster, see Getting Started with Amazon EKS in the Amazon EKS User Guide. If you're running your own Kubernetes cluster on Amazon EC2, then ensure that Docker is authenticated to the Amazon ECR repository that the Envoy image is in. For more information, see Envoy image, Registry authentication in the Amazon Elastic Container Registry User Guide, and Pull an Image from a Private Registry in the Kubernetes documentation.
• App Mesh supports Linux services that are registered with DNS, AWS Cloud Map, or both. To use this getting started guide, we recommend that you have three existing services that are registered with DNS. The procedures in this topic assume that the existing services are named serviceA, serviceB, and serviceBv2 and that all services are discoverable through a namespace named apps.local.
You can create a service mesh and its resources even if the services don't exist, but you cannot use the mesh until you have deployed actual services.
• The AWS CLI version 1.18.116 or later or 2.0.38 or later installed. To install or upgrade the AWS CLI, see Installing the AWS CLI.
• A kubectl client that is configured to communicate with your Kubernetes cluster. If you're using Amazon Elastic Kubernetes Service, you can use the instructions for installing kubectl and configuring a kubeconfig file.
• Helm version 3.0 or later installed. If you don't have Helm installed, see Using Helm with Amazon EKS in the Amazon EKS User Guide.