Work with open-source tools for Traffic Mirroring
You can use open-source tools to monitor network traffic from Amazon EC2 instances. The following tools work with Traffic Mirroring:
• Zeek — For more information, see the Zeek Network Monitor Security website.
• Suricata — For more information see the Suricata website.
These open-source tools support VXLAN decapsulation, and they can be used at scale to monitor VPC traffic. For information about how Zeek handles VXLAN support and to download the code, see Zeek vxlan on the GitHub website. For information about how Suricata handles VXLAN support and to download the code, see Suricata on the GitHub website.
The following example uses the Suricata open-source tool. You can follow similar steps for Zeek.
Consider the scenario where you want to mirror inbound TCP traffic on an instance and send the traffic to an instance that has the Suricata software installed. You need the following traffic mirror entities for this example:
• An EC2 instance with the Suricata software installed on it
• A traffic mirror target for the EC2 instance (Target A)
• A traffic mirror filter with a traffic mirror rule for the TCP inbound traffic (Filter rule 1)
• A traffic mirror session that has the following:
• A traffic mirror source
• A traffic mirror target for the appliance
• A traffic mirror filter with a traffic mirror rule for the TCP inbound traffic
Step 1: Install the Suricata software on the EC2 instance target
Launch an EC2 instance, and then install the Suricata software on it by using the following commands.
# Become sudo
# Create the default suricata rules directory mkdir /var/lib/suricata/rules
# Add a rule to match all UDP traffic
echo 'alert udp any any -> any any (msg:"UDP traffic detected"; sid:200001; rev:1;)' > / var/lib/suricata/rules/suricata.rules
# Start suricata listening on eth0 in daemon mode
suricata -c /etc/suricata/suricata.yaml -k none -i eth0 -D
# Capture logs can be found in /var/log/suricata/fast.log
Step 2: Create a traffic mirror target
Step 2: Create a traffic mirror target
Create a traffic mirror target (Target A) for the EC2 instance. Depending on your configuration, the target is one of the following types:
• The network interface of the monitoring appliance
• The Network Load Balancer when the appliance is deployed behind one.
For more information, see the section called “Create a traffic mirror target” (p. 19).
Step 3: Create a traffic mirror filter
Create a traffic mirror filter (Filter 1) with the following inbound rule. For more information, see the section called “Create a traffic mirror filter” (p. 23).
Traffic mirror filter rule for inbound TCP traffic
Option Value
Rule action Accept
Protocol TCP
Source port range
Destination port range
Source CIDR block 0.0.0.0/0
Destination CIDR block 0.0.0.0/0
Description TCP Rule
Step 4: Create a traffic mirror session
Create and configure a traffic mirror session with the following options. For more information, see the section called “Create a traffic mirror session” (p. 27).
Traffic mirror session to monitor inbound TCP traffic
Option Value
Mirror source The network interface of the instance that you
want to monitor.
Mirror target Target A
Filter Filter 1
Traffic Mirroring metrics and dimensions
Monitor mirrored traffic using Amazon CloudWatch
You can monitor your mirrored traffic using Amazon CloudWatch, which collects information from your network interface that is part of a traffic mirror session, and creates readable, near real-time metrics. You can use this information to monitor and troubleshoot Traffic Mirroring.
For more information about Amazon CloudWatch, see the Amazon CloudWatch User Guide. For more information, see List the available CloudWatch metrics for your instances in Amazon EC2 User Guide for Linux Instances. For more information, see Amazon CloudWatch Pricing.
Traffic Mirroring metrics and dimensions
The following metrics are available for your mirrored traffic.
Metric Description
NetworkMirrorIn The number of bytes received on all network
interfaces by the instance that are mirrored.
The number reported is the number of bytes received during the period. If you are using basic (five-minute) monitoring, you can divide this number by 300 to find Bytes/second. If you have detailed (one-minute) monitoring, divide it by 60.
Units: Bytes
NetworkMirrorOut The number of bytes sent out on all network
interfaces by the instance that are mirrored.
The number reported is the number of bytes sent during the period. If you are using basic (five-minute) monitoring, you can divide this number by 300 to find Bytes/second. If you have detailed (one-minute) monitoring, divide it by 60.
Units: Bytes
NetworkPacketsMirrorIn The number of packets received on all network interfaces by the instance that are mirrored. This metric is available for basic monitoring only.
Units: Count
NetworkPacketsMirrorOut The number of packets sent out on all network interfaces by the instance that are mirrored. This metric is available for basic monitoring only.
Traffic Mirroring metrics and dimensions
Metric Description
Units: Count
NetworkSkipMirrorIn The number of bytes received, that meet the
traffic mirror filter rules, that did not get mirrored because of production traffic taking priority.
Units: Bytes
NetworkSkipMirrorOut The number of bytes sent out, that meet the
traffic mirror filter rules, that did not get mirrored because of production traffic taking priority.
Units: Bytes
NetworkPacketsSkipMirrorIn The number of packets received, that meet the traffic mirror filter rules, that did not get mirrored because of production traffic taking priority. This metric is available for basic monitoring only.
Units: Count
NetworkPacketsSkipMirrorOut The number of packets sent out, that meet the traffic mirror filter rules, that did not get mirrored because of production traffic taking priority. This metric is available for basic monitoring only.
Units: Count
To filter the metric data, use the following dimensions.
Dimension Description
AutoScalingGroupName This dimension filters the data you request for all instances in a specified capacity group. An Auto Scaling group is a collection of instances you define if you're using Auto Scaling. This dimension is available only for Amazon EC2 metrics when the instances are in such an Auto Scaling group.
Available for instances with Detailed or Basic Monitoring enabled.
ImageId This dimension filters the data you request for
all instances running this Amazon EC2 Amazon Machine Image (AMI). Available for instances with Detailed Monitoring enabled.
InstanceId This dimension filters the data you request for the
identified instance only. This helps you pinpoint an exact instance from which to monitor data.
Available for instances with Detailed or Basic Monitoring enabled.
InstanceType This dimension filters the data you request for
all instances running with this specified instance type. This helps you categorize your data by the type of instance running. For example, you might
View Traffic Mirroring CloudWatch metrics
Dimension Description
compare data from an m1.small instance and an m1.large instance to determine which has the better business value for your application.
Available for instances with Detailed Monitoring enabled.
View Traffic Mirroring CloudWatch metrics
You can view the metrics for Traffic Mirroring as follows.
To view metrics using the CloudWatch console
Metrics are grouped first by the service namespace, and then by the various dimension combinations within each namespace.
1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.
2. In the navigation pane, choose Metrics.
3. Under All metrics, choose the EC2 metric namespace.
4. To view the metrics, select the metric dimension.
To view metrics using the AWS CLI
At a command prompt, use the following command to list the metrics that are available for Traffic Mirroring:
aws cloudwatch list-metrics --namespace "AWS/EC2"
The Traffic Mirroring metrics are included with the metrics for Amazon EC2.
Considerations
Traffic Mirroring quotas and considerations
Considerations
• Encapsulated mirror traffic is routed by using the VPC route table. Make sure that your route table is configured to send the mirrored traffic to the correct traffic mirror target.
• You can only create a traffic mirror session if you are the owner of the source network interface or its subnet.
• We truncate the packet to the MTU value when both of the following are true:
• The traffic mirror target is a standalone instance.
• The mirrored traffic packet size is greater than the traffic mirror target MTU value.
For example, if an 8996 byte packet is mirrored, and the traffic mirror target MTU value is 9001 bytes, the mirror encapsulation results in the mirrored packet being greater than the MTU value. In this case, the mirror packet is truncated. To prevent mirror packets from being truncated, set the traffic mirror source interface MTU value to 54 bytes less than the traffic mirror target MTU value for IPv4 and 74 bytes less than the traffic mirror target MTU value when you use IPv6. Therefore, the maximum MTU value supported by Traffic Mirroring with no packet truncation is 8947 bytes. For more information about configuring the network MTU value, see Network Maximum Transmission Unit (MTU) for Your EC2 Instance in the Amazon EC2 User Guide for Linux Instances.
• Mirrored traffic counts toward instance bandwidth. For example, if you mirror a network interface that has 1 Gbps of inbound traffic and 1 Gbps of outbound traffic, the instance must handle 4 Gbps of traffic (1 Gbps inbound, 1 Gbps mirrored inbound, 1 Gbps outbound, and 1 Gbps mirrored outbound).
• Production traffic has a higher priority than mirrored traffic when there is traffic congestion. As a result, mirrored traffic is dropped when there is congestion.
• Mirrored outbound traffic from a source instance is not subject to security group evaluation.
• Flow logs do not capture mirrored traffic.
• When you delete a network interface that is a traffic mirror source, the traffic mirror sessions that are associated with the source are automatically deleted.
• Packets that are dropped at the traffic mirror source by security group rules or by network ACL rules are not mirrored.
• An elastic network interface cannot be a traffic mirror target and a traffic mirror session source at the same time.
• We recommend using a Network Load Balancer as a target for high availability.
• If you do not have UDP listeners on the Network Load Balancer, you can still use the Network Load Balancer as a target. However, Traffic Mirroring cannot occur because there are no UDP listeners.
• If you remove the UDP listeners from a Network Load Balancer that is a traffic mirror target, Traffic Mirroring fails without an error indication.
• When the Network Load Balancer removes the node in an Availability Zone from the DNS table, Traffic Mirroring continues to send the mirrored packets to that node.
• When you have an existing Network Load Balancer which is a traffic mirror target and you add additional subnets to it, there is no effect. For example, mirrored traffic in the Availability Zone of the new subnet is not routed in the same Availability Zone unless you enable cross-zone load balancing.
Limitations
• We recommend that you use cross-zone load balancing with your Network Load Balancer to ensure that the packets continue to be mirrored when all targets in an Availability Zone are not healthy.
Limitations
Traffic Mirroring is not available on the following EC2 instance types:
• These current generation instances: C6gn, C6i, Im4gn, Is4gen, M6a, M6i, R6i, T2
• Any bare metal instances
• Any previous generation instances
The following traffic types cannot be mirrored:
• ARP
• DHCP
• Instance metadata service
• NTP
• Windows activation
Quotas
The following are the quotas for Traffic Mirroring for your AWS account.
Sessions
• Maximum number of sessions per account: 10,000
• Maximum number of sessions per source network interface: 3
Targets
• Maximum number of targets per account: 10,000
Filters
• Maximum number of filters per account: 10,000
• Maximum number of rules per filter: 10 †
Sources
• Maximum number of sources per Network Load Balancer: No limit
• Maximum number of sources per target (smaller sizes): 10 †
• Maximum number of sources per target (largest size): 100 *
† This quota cannot be increased.
* Applies only to the largest instance size for an instance type. For example, for M5 instances, the maximum is 100 for m5.24xlarge and 10 for all other M5 instance sizes.
Checksum offloading
Checksum offloading
The Elastic Network Adapter (ENA) provides checksum offloading capabilities. If a packet is truncated, this might result in the packet checksum not being calculated for the mirrored packet. The following checksums are not calculated when the mirrored packet is truncated:
• If the mirror packet is truncated, the mirror packet L4 checksum is not calculated.
• If any part of the L3 header is truncated, the L3 checksum is not calculated.
Use the following commands to disable ENA checksum offloading on Amazon Linux 2 AMI:
[ec2-user@ip-11-0-0-166 ~]$ sudo ethtool --offload eth0 tx off [ec2-user@ip-11-0-0-166 ~]$ sudo ethtool --show-offload eth0 Features for eth0:
rx-checksumming: on tx-checksumming: off tx-checksum-ipv4: off
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
Document history for Traffic Mirroring
The following table describes the releases for Traffic Mirroring.
Feature Description Release Date
Support for additional
instances For more information, see Quotas and
considerations (p. 36). February 10, 2021
Support for Amazon
CloudWatch Monitor your mirrored traffic using Amazon CloudWatch. For more information, see Monitor mirrored traffic (p. 33).
Nov 25, 2019
Initial release This release introduces Traffic Mirroring. June 25, 2019