Amazon Lookout for Metrics

(1)

Amazon Lookout for Metrics

Developer Guide

(2)

Amazon Lookout for Metrics: Developer Guide

Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be aﬃliated with, connected to, or sponsored by Amazon.

(3)

What is Amazon Lookout for Metrics?

Amazon Lookout for Metrics is a service that ﬁnds anomalies in your data, determines their root causes, and enables you to quickly take action. Built from the same technology used by Amazon.com, Amazon Lookout for Metrics reﬂects 20 years of expertise in anomaly detection and machine learning.

In Lookout for Metrics, you create detectors that monitor data to find anomalies. You configure the detector with a datasource and choose the values that it monitors (the dataset's measures). The detector can monitor all values of a measure overall, or use other data to sort measures into groups. For example, you can choose to monitor the availability of an application worldwide, or use a location field in your data as a dimension to monitor availability separately in each AWS Region or Availability Zone. Each combination of measure and dimension value is called a metric.

To gain an understanding of your data and better ﬁnd anomalies, a detector uses your data to learn. When you create a detector, you choose an interval from between 5 minutes and 1 day, which determines how often the detector imports data and ﬁnds anomalies. Depending on the interval that you choose, a detector spends between a few hours and a few days learning about your data. To speed up the learning process, you can provide historical data. You can also use historical data to run a detector in backtest mode to see how well it works on your data.

When Lookout for Metrics finds an anomaly, it gives it a severity score to indicate how unexpected it is, based on the detector's understanding of your data. When multiple metrics are affected by an anomaly, the detector collects them in a group. To send anomaly alerts when high severity anomalies occur, you can add alerts to the detector. Alerts can run a Lambda function or send a notification to an Amazon SNS topic. Anomaly alerts summarize all affected metrics for an anomaly, which reduces the amount of similar alerts that it sends.

Lookout for Metrics can monitor data from many diﬀerent domains, stored in many diﬀerent formats.

In addition to monitoring operational metrics for emergent issues, you can ﬁnd anomalies in sales data, marketing data, and customer engagement metrics.

Lookout for Metrics integrates with other services to provide additional datasources and alert channels.

Lookout for Metrics supports the following services as datasources.

• Amazon S3 (p. 56) – Read metrics data from text objects in a bucket. Supports using historical data for training and testing a detector.

• Amazon CloudWatch (p. 41) – Analyze CloudWatch metrics sent by an AWS service, AWS resource, or an application.

• Amazon Relational Database Service (Amazon RDS) (p. 44) – Analyze items in a relational database.

The following database engines are supported:

• Amazon Aurora

• MySQL

• PostgreSQL

• MariaDB

• Microsoft SQL Server

• Amazon Redshift (p. 50) – Analyze items in a Amazon Redshift data warehouse.

• Amazon AppFlow (p. 39) – Analyze data ﬂow from a web service with an Amazon AppFlow data ﬂow. The following services are supported:

(7)

Are You a First-Time User of Lookout for Metrics?

• Salesforce

• Marketo

• Dynatrace

• Singular

• Zendesk

• Servicenow

• Infor Nexus

• Trendmicro

• Veeva

• Google Analytics

• Amplitude

To get started with Lookout for Metrics and learn more about the service, continue to Getting started with Lookout for Metrics (p. 5).

Are You a First-Time User of Lookout for Metrics?

If you are a ﬁrst-time user of Lookout for Metrics, we recommend that you start with the following pages:

1.Getting started (p. 5) –Get started with Lookout for Metrics.

2.Key terms and concepts (p. 3) – Learn about the key terms and concepts of Lookout for Metrics.

3.API reference – Familiarize yourself with the Lookout for Metrics API actions and data types.

(8)

Detector

Lookout for Metrics terms and concepts

This section describes key concepts and terminology you need to understand to use Amazon Lookout for Metrics eﬀectively.

Detector

A detector is a Lookout for Metrics resource that monitors a dataset and identiﬁes anomalies. Detectors use machine learning to ﬁnd patterns in data, and to distinguish between expected variations in data and legitimate anomalies. To improve its performance, a detector learns more about your data over time.

When you create a detector, you configure an interval to tell the detector how often to update the dataset and look for anomalies. An interval can range from 5 minutes to 1 day. At the end of each interval, the detector looks for anomalies in the data from the interval. If all data is not available immediately at the end of each interval, you can configure an offset that specifies an amount of time for the detector to wait before it starts processing data.

In addition to a dataset, a detector can have alerts. For more information, see Working with detectors (p. 18).

Datasource

A datasource is a service or resource that provides time-series data for analysis. A datasource has dated records that each have metrics and (optionally) dimensions. If you make an application that writes records to Amazon S3 as they occur, you can use the Amazon S3 bucket as a datasource.

Lookout for Metrics integrates with other AWS services to provide additional datasources. A detector can access records in a relational database in Amazon Relational Database Service or Amazon Redshift, metrics in Amazon CloudWatch, or data in an Amazon AppFlow ﬂow. With these services, you can collect data from even more services and resources.

Dataset

A dataset is a detector's copy of your data. At the end of each interval, the detector copies metrics and dimensions from a datasource into its dataset for analysis. Each row or document in a dataset is called a record. Each labeled value or key-value pair in a record is a field. Each record must have a timestamp field and at least one field that the detector monitors, called a measure.

A dataset can have continuous data, historical data or both. A detector looks for anomalies in continuous data, and uses historical data for learning. If its dataset has only historical data, a detector runs in backtest mode. In backtest mode, a detector uses most of the data for learning and attempts to ﬁnd anomalies in a smaller subset of recent data.

Datasets have a datasource, a service role, and a mapping that tells the detector which values to monitor.

For more information, see Managing a dataset in Amazon S3 (p. 24).

(9)

Metrics

After giving your detector a datasource, you choose fields to be the dataset's measures. Measures are the primary fields that the detector monitors. You can also configure up to five additional fields as dimensions. Dimensions are secondary fields that create subgroups of measures based on their value.

Each combination of measure and dimension is called a metric.

For example, you can choose a ﬁeld named availability for a measure. If you don't choose a

dimension, the detector monitors availability across all records. If you choose a ﬁeld named country for a dimension, then the detector monitors availability in each country as a separate metric: availability in Canada, availability in Italy, and so on.

Important

Pricing in Lookout for Metrics is based on the number of metrics that detectors monitor. Each dimension increases the number of metrics by the number of unique values in the dimension.

For example, the number of countries that appear in a country dimension. Choose dimensions that have a limited number of known values.

For more information, see Pricing (p. 11).

For more information, see Setting up a detector (p. 19).

Alert

To send a notifications or initiate a processing workflow when the detector finds an anomaly, you can create an alert. An alert has a target that can be an Amazon Simple Notification Service (Amazon SNS) topic or an AWS Lambda function. When the detector finds an anomaly with a severity score over a configurable threshold, the alert sends a record of the anomaly to the target. A severity score is a number between 0 and 100 that indicates how far the metric value is outside of the expected range.

You can process the anomaly record in the programming language of your choice with a Lambda function. With an Amazon SNS topic, you can send the anomaly record to multiple subscribers. A subscriber can use any protocol that Amazon SNS supports, including email addresses, mobile devices, and webhooks. With a webhook, you can send the anomaly record to third-party services, such as Slack.

For more information, see Working with alerts (p. 32).

Feedback

When the detector ﬁnds an anomaly, it creates a report with details about all of the metrics that had unexpected values during the interval in the Lookout for Metrics console. When you view this report , you can provide feedback on the relevance of each metric. The detector uses your feedback to improve its accuracy.

For more information, see Working with anomalies (p. 28).

(10)

Getting started with Lookout for Metrics

This section shows you how to get started with Amazon Lookout for Metrics by creating a detector and ﬁnding anomalies.

Topics

• Create a datasource in Amazon S3 (p. 6)

• Set up a detector (p. 8)

• Pricing (p. 11)

(11)

Create a datasource

Create a datasource in Amazon S3

To get started with Lookout for Metrics, create a dataset that you can look for anomalies in. You can use data that you already have, or generate data with a sample application.

A Lookout for Metrics detector can read data in an Amazon S3 bucket in CSV or JSON lines format.

Each record must have a numerical field to monitor (a measure) and a field that represents a date or timestamp. Records can also have categorical fields (dimensions), additional measures, and fields that are not monitored.

For example, the following example is a CSV file that has a measure field named Calls, a timestamp field named Date, and text fields that could be used as dimensions.

Example CSV ﬁle

Date,EventSource,EventName,ReadOnly,AccessKeyId,Username,Calls

2021-03-12 22:00:07,kms.amazonaws.com,GenerateDataKey,true,none,none,1 2021-03-12 22:00:12,kms.amazonaws.com,Decrypt,true,ASIAXMPLV4CQYXGMLRQ,greg,1 2021-03-12 22:00:36,kms.amazonaws.com,Decrypt,true,ASIAXMPLVMP6KEG3BRA,michael,1

In the example, there is a header row with names for each field, fields are separated by commas, and values are not enclosed in quotes. The header is optional, but it simplilfies configuration. Values can be enclosed in single or double quotes. In addition to commas, tabs, spaces, and pipes (|) can be used as delimeters.

Sections

• Collect existing data (p. 6)

• Generate data with a sample application (p. 6)

• Download sample data (p. 7)

Collect existing data

To use existing data to run a backtest, collect it in one or more ﬁles and store it in a single folder in an Amazon S3 bucket. The data must contain timestamped entries for at least 285 intervals, where an interval is 5 minutes, 10 minutes, 1 hour, or 1 day long. Each ﬁle can have entries for one interval or multiple intervals.

s3://lookoutmetrics-dataset-1234567891012/backtest/20210104-20210110.csv 20210111-20210117.csv 20210118-20210125.csv

To use a detector in continuous mode, you need to store it at a diﬀerent path for each interval as the intervals occur. For an example of how to do this, run the sample application (p. 6).

For more information about supported data formats and folder structures, see Managing a dataset in Amazon S3 (p. 24).

Generate data with a sample application

The GitHub repository for this guide includes a sample application that you can use to generate a dataset with data from your AWS account. The sample application runs locally or in AWS Lambda. It uses the AWS SDK to read events from AWS CloudTrail, reformats them with Pandas, and stores them in Amazon S3.

(12)

Download sample data

Example lambda_function.py – Processing data

...def process_events(end_time, service):

start_time = end_time - timedelta(minutes=60) records = []

# call CloudTrail

responses = get_events(service, start_time, end_time)

records = [{k: event.get(k) for k in fields} for response in responses for event in response.get('Events')]

# format timeseries dataframe df = pd.DataFrame(records) ...

To set up the application, follow the instructions in the README ﬁle. To generate a dataset for a backtest, run the backﬁll script.

(data-processor) data-processor$ ./5-backfill.sh

In Lambda, the application runs once every hour to generate a timeseries for the previous hour. The backﬁll script runs the application locally to generate two weeks worth of data and store it in the application's S3 bucket.

Download sample data

Sample data is available in the service's GitHub repository at github.com/aws-samples/amazon-lookout- for-metrics-samples. The samples repository provides an archive of sample data and instructions for using it with a detector.

(13)

Set up a detector

After you've prepared data in an Amazon Simple Storage Service (Amazon S3) bucket (p. 6), create a detector and add a dataset. At the end of each interval, the detector imports data from the bucket (your datasource) into the dataset and analyzes it.

Steps

• Create a detector (p. 8)

• Add a dataset (p. 8)

• Activate the detector (p. 9)

• (Optional) Add an alert (p. 9)

• Review anomalies (p. 9)

• Clean up (p. 10)

Create a detector

Use the Lookout for Metrics console to create the detector that monitors your data.

To create a detector

1. Open the Lookout for Metrics console.

2. Choose Create detector.

3. For Name, enter sample-detector.

4. For Description, enter Getting started detector.

5. For Interval, choose the interval that you used to organize your data.

6. Choose a Create.

Add a dataset

Add a dataset by specifying the location of your data in Amazon S3 and the names of ﬁelds in your data that are timestamps, measures, and dimensions.

To add a dataset

1. Open the Lookout for Metrics console Detectors page.

2. Choose sample-detector.

3. Choose Add dataset.

4. For Name, enter S3-dataset.

5. For Timezone, choose the timezone that is reﬂected in your data's timestamps.

6. For Datasource, choose Amazon S3.

7. Choose a detector mode. For Backtest, you only provide historical data. For Continous, you provide continuous data and can optionally provide historical data to speed up learning.

8. Follow the instructions to specify the data path(s) and then choose Next.

9. Choose the ﬁeld in your data that speciﬁes a timestamp for each record. The detector checks the timestamp on each record, and only processes records that match the interval that it is analyzing.

10. Choose a numerical field in your data to be a Measure. The detector aggregates values of the measure field within each interval and monitors the aggregate value. For example, availability 11. (Optional) Choose additional fields to be measures and dimensions. A dimension is a field that

creates subgroups of measure values that are monitored separately. For example, city.

(14)

Activate the detector

To import data from the bucket and look for anomalies, activate the detector.

To activate the detector

2. Choose the detector.

3. Choose Activate detector.

When you activate the detector, it uses data from several intervals to learn, before attempting to ﬁnd anomalies. If no historical data is available, the training process takes approximately one day for a ﬁve- minute interval. Training time varies depending on the detector's interval (p. 70).

(Optional) Add an alert

In this section, you will add an alert to the detector. You will need to create an Amazon SNS topic. You will use its ARN in this step. For more information about creating an Amazon SNS topic, see Tutorial:

Creating an Amazon SNS topic.

To add an alert

2. Choose a detector.

3. Choose Add alert.

4. Conﬁgure the following options.

• Name – my-alert

• Description – Send anomaly alerts to an SNS topic.

• Threshold – 50

• Channel – Amazon SNS

• Resource – Your SNS topic

• Role – Create a role 5. Choose Add alert.

Review anomalies

In this section, you will review the anomalies found by the detector.

To review anomalies

2. Choose sample-detector.

3. Choose View anomalies.

4. Choose an anomaly.

Each anomaly can have multiple metrics associated with it. Review each metric to see if it is relevant or not, and use the feedback options to help the detector learn about your data.

(15)

Clean up

If you are done using the detector, delete it.

To delete the detector

2. Choose the detector, sample-detector.

3. Choose Delete.

The detector's dataset and alerts are deleted automatically.

(16)

Pricing

Pricing for Amazon Lookout for Metrics is based on the number of metrics that your detector monitors for anomalies. This number varies based on the number of measures and dimensions that you choose when you conﬁgure a dataset. It can also vary from interval to interval, based on the number of diﬀerent values that appear for each dimension.

For example, if you choose a ﬁeld named availability as a measure and choose no dimensions, Lookout for Metrics monitors values of availability for each interval (1 metric). If you choose availability as a measure and state as a dimension, where state has 50 possible values, Lookout for Metrics monitors values of availability for each value of state (50 metrics).

For another example, if you choose a ﬁeld named orders with a dimension of state, the number of metrics can vary from interval to interval. You might have orders from 40 states in 1 interval, and 45 in another. For these intervals, the number of metrics would be 40 and 45, respectively.

For speciﬁc prices and pricing examples, see Amazon Lookout for Metrics pricing . Sections

• Charges for metrics (p. 11)

• Charges for other services (p. 11)

Charges for metrics

When a detector monitors data, it generates charges for each metric that appears in the dataset during an interval. To determine the number of metrics, the number of measures that you assign when you conﬁgure your dataset is multiplied by the number of unique combinations of dimension name and dimension value that appear in live data.

For example, if you choose 2 ﬁelds for measures, and 3 dimensions that each have 5 possible values, the maximum number of metrics that can appear in an interval is 250. If not all dimension values appear in every interval, the number of metrics can vary between 6 and 250 for each interval.

Important

The cost of using a detector increases linearly with the number of diﬀerent values that appear in each interval for the dimensions that you conﬁgure for your dataset. For example, if you use a dimension named continent that has 7 possible values, the cost of that dimension is limited by the number of measures times 7. If you use a dimension named id that stores a GUID, the cost of the dimension increases with the number of GUIDs that appear in the data.

Charges for other services

When Lookout for Metrics copies data into a dataset from a datasource, data transfer and API usage charges can apply. Several services have a free tier that applies if your overall use of that service is low.

Some free tiers are limited to the ﬁrst year after you create your account. Others apply permanently or under speciﬁc conditions. When using Lookout for Metrics, you might incur charges for the following services:

• Amazon Simple Storage Service (Amazon S3) – Charges for storage, API requests, and data transfer.

Eligible for free tier under certain conditions.

• Amazon CloudWatch – Charges for CloudWatch metric storage and retrieval (transfer out). Eligible for free tier under certain conditions.

• Amazon Relational Database Service – Charges for database instances and data transfer. Eligible for free tier under certain conditions.

(17)

Charges for other services

• Amazon Redshift – Charges for data warehouse nodes and data transfer. Eligible for free tier under certain conditions.

• Amazon AppFlow – Charges per ﬂow run (transfer in) and for each GB of data processed.

This list highlights the main billing components of each service. Additional charges apply for use of optional features. For more information, see the pricing page for each service.

(18)

Amazon Lookout for Metrics permissions

You can use AWS Identity and Access Management (IAM) to manage access to the Lookout for Metrics API and resources like detectors and datasets. For users and applications in your account that use Lookout for Metrics, you manage permissions in a permissions policy that you can apply to IAM users, groups, or roles.

Lookout for Metrics uses IAM service roles (p. 16) to access other services on your behalf. You create or choose a service role when you create a dataset that reads data from Amazon S3 or another service. You also pass service roles to Lookout for Metrics when you conﬁgure an alert that targets a Lambda function or an Amazon SNS topic. The Lookout for Metrics console can create these roles for you if you have the required IAM permissions.

For more information about IAM, see What is IAM? in the IAM User Guide.

Topics

• Identity-based IAM policies for Lookout for Metrics (p. 14)

• Service roles for Amazon Lookout for Metrics (p. 16)

(19)

User policies

Identity-based IAM policies for Lookout for Metrics

To grant users in your account access to Lookout for Metrics, you use identity-based policies in AWS Identity and Access Management (IAM). Identity-based policies can apply directly to IAM users, or to IAM groups and roles that are associated with a user. You can also grant users in another account permission to assume a role in your account and access your Lookout for Metrics resources.

The following IAM policy allows a user to access all Lookout for Metrics API actions, and to pass service roles (p. 16) to Lookout for Metrics.

Example User policy

{ "Version": "2012-10-17", "Statement": [

{

"Effect": "Allow", "Action": [

"lookoutmetrics:*"

],

"Resource": "*"

}, {

"iam:PassRole"

],

"Resource": "*", "Condition": { "StringEquals": {

"iam:PassedToService": "lookoutmetrics.amazonaws.com"

} } } ]}

The preceding policy does not allow a user to create IAM roles. For a user with these permissions to create a dataset or alert, an administrator must create the service role that grants Lookout for Metrics permission to access datasources and alert channels. For more information, see Service roles for Amazon Lookout for Metrics (p. 16).

In addition to Lookout for Metrics, a user needs permission to view resources in services that they use as a detector's datasource or as alert channels. When you work with a detector in the Lookout for Metrics console, the console uses your permissions to simplify the conﬁguration process.

You can grant full access to each service or limit the scope of permissions by resource name. The following example shows a policy that provides read-only access to a subset of resources in Lookout for Metrics. The Resources key for applicable actions limits access to resources whose names start with intern-.

{

"Sid": "readonly-0", "Effect": "Allow", "Action": [

"lookoutmetrics:List*", "lookoutmetrics:Get*",

(20)

User policies

"lookoutmetrics:Describe*"

],

"Resource": [

"arn:aws:lookoutmetrics:us-east-2:123456789012:MetricSet/intern-*/intern-

*",

"arn:aws:lookoutmetrics:us-east-2:123456789012:Alert:intern-*",

"arn:aws:lookoutmetrics:us-east-2:123456789012:AnomalyDetector:intern-*"

] }, {

"Sid": "readonly-1", "Effect": "Allow", "Action": [

"lookoutmetrics:ListAnomalyDetectors", "lookoutmetrics:GetSampleData"

],

"Resource": "*"

}, {

"Sid": "readonly-2", "Effect": "Allow", "Action": "iam:PassRole", "Resource": "*",

"Condition": { "StringEquals": {

"iam:PassedToService": "lookoutmetrics.amazonaws.com"

} } } ] }

If you tag your resources (p. 35), you can also use condition keys to limit access to a resource based on the presence or value of a tag. For permissions purposes, detectors, datasets and alerts are independent resources. If you grant or deny permission to a detector, the permission does not automatically apply to the detector's dataset or alerts.

The resources and conditions supported for use in policies vary among API actions. For more

information, see Actions, resources, and condition keys for Amazon Lookout for Metrics in the Service Authorization Reference.

(21)

Service roles

Service roles for Amazon Lookout for Metrics

You can use service roles to grant Amazon Lookout for Metrics permission to access data and send alerts to other services. A service role is an AWS Identity and Access Management (IAM) role that an AWS service can use to access resources from other services in your account. For example, to invoke an AWS Lambda function when an alert is sent, Lookout for Metrics assumes a role that allows it to invoke the function. Lookout for Metrics uses a separate service role for each supported data source and notiﬁcation target, including Amazon Simple Storage Service (Amazon S3), Lambda, and Amazon Simple Notiﬁcation Service (Amazon SNS)

When you conﬁgure a dataset or alert in the Lookout for Metrics console, the console can create the required role for you if you have the necessary permissions. If you don't have permissions to create roles, or if you use the Lookout for Metrics API to manage resources, you can create the roles manually. Service roles must have the following trust policy.

{

"Effect": "Allow", "Principal": {

"Service": "lookoutmetrics.amazonaws.com"

},

"Action": "sts:AssumeRole"

} ]}

The trust policy allows Lookout for Metrics to assume the role.

Sections

• Sample IAM policies (p. 16)

• Sample AWS CloudFormation templates (p. 17)

Sample IAM policies

The GitHub repository for this guide provides sample IAM policies that you can use as reference for developing service roles. You can use a single role that grants permission for both importing data and sending alerts by combining the applicable policies.

Example alert-lambda.json – Send anomaly alerts to a Lambda function

{

"Version": "2012-10-17", "Statement": [

{

"lambda:InvokeFunction"

],

"Resource": [

"arn:aws:lambda:${Region}:${Account}:function:${FunctionName}"

] } ] }

(22)

Sample AWS CloudFormation templates

Values contained in brackets are placeholders for account-speciﬁc information such as resource names and account IDs. For more information, see Sample policies in the GitHub repo.

Sample AWS CloudFormation templates

The GitHub repository for this guide provides sample AWS CloudFormation templates that you can use to automate the creation of service roles. The templates use parameters and naming patterns to apply least-privilege permissions where possible.

The following sample template creates a service role that gives Lookout for Metrics permission to access S3 buckets, SNS topics, and Lambda functions that have names preﬁxed with lookoutmetrics-.

Example servicerole-s3.yml – Amazon S3 and Lambda permissions

Parameters:

bucketName:

Description: Bucket name Type: String

Default: lookoutmetrics-*

queueName:

Description: Queue name Type: String

functionName:

Description: Function name Type: String

Resources:

serviceRole:

Type: AWS::IAM::Role Properties:

Policies:

- PolicyName: read-s3 PolicyDocument:

Version: 2012-10-17 Statement:

- Effect: Allow Action:

- s3:ListBucket - s3:GetBucketAcl

Resource: !Sub arn:${AWS::Partition}:s3:::${bucketName}

- Effect: Allow Action:

- s3:GetObject - s3:GetBucketAcl

Resource: !Sub arn:aws:s3:::${bucketName}/*

- PolicyName: invoke-function PolicyDocument:

Version: 2012-10-17 Statement:

- Effect: Allow

Action: lambda:InvokeFunction

Resource: !Sub arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:

${functionName}

...

For more information, see Sample templates in the GitHub repo.

(23)

Working with detectors

In Amazon Lookout for Metrics, a detector is a resource that monitors a dataset and identiﬁes anomalies (data that falls outside of the expected range). Detectors use machine learning (ML) to ﬁnd patterns in business data, and to distinguish between expected variations in data and legitimate anomalies. A detector can monitor a dataset that contains metrics data that you manage in Amazon Simple Storage Service (Amazon S3), live data from another service such as Amazon CloudWatch, or events from a database. When new data points fall outside of the expected range, the detector records the anomaly and sends an alert.

A dataset (p. 24) is a collection of timestamped data points that can each have multiple metrics. You choose one of the metrics to be the measure, which is the primary metric that the detector monitors for anomalies. You can also conﬁgure up to ﬁve additional metrics as dimensions. Dimensions are secondary metrics that the detector uses to segment anomalies and identify contributing factors.

For example, you can choose a ﬁeld named availability for a measure. If you don't choose a

dimension, the detector monitors availability across all records. If you choose a ﬁeld named country for a dimension, then the detector monitors availability in each country as a separate metric: availability in Canada, availability in Italy, and so on.

Detectors primarily work against live data. A detector analyzes new data periodically to ﬁnd anomalies in measure values. The interval at which it analyzes data can be between 5 minutes and 1 day. To allow time for the datasource to collect all data before analysis starts, you also conﬁgure a delay on the dataset. At the end of an interval, the detector waits for the duration of the delay before analyzing data.

When you create a detector, you can also provide historical data. If you provide historical data, the detector uses it to learn patterns and relationships between ﬁelds in your data. If not, the detector spends several intervals learning on live data.

Every time it runs, the detector analyzes all of the data generated during the interval, identifies anomalous data points, and assigns a severity score to each. If the severity of an anomaly exceeds a threshold, the detector sends an alert. You can configure alerts (p. 32) to send a notification to an Amazon Simple Notification Service (Amazon SNS) topic, or to invoke an AWS Lambda function. If you get too many or too few results, you can change the threshold that triggers the alert.

Topics

• Setting up a detector (p. 19)

• Managing detectors (p. 22)

• Managing a dataset in Amazon S3 (p. 24)

• Working with anomalies (p. 28)

• Working with alerts (p. 32)

• Tagging Lookout for Metrics resources (p. 35)

(24)

Setting up

Setting up a detector

An anomaly detector is an Amazon Lookout for Metrics resource that monitors a dataset to ﬁnd anomalies.

By default, detectors use a key managed by Lookout for Metrics. To use a key that you manage in your own account, create a symmetric key in AWS KMS and grant Lookout for Metrics permission to use it.

Topics

• Creating a detector (p. 19)

• Creating a dataset (p. 20)

Creating a detector

To create a detector, you provide a name, description, and interval. To start the detector and begin ﬁnding anomalies, add a dataset and notiﬁcations and activate the detector.

To create a detector

2. Choose Create detector.

3. Provide the following information:.

• Name – A name for the detector.

• Description – A description of the detector.

• Interval – The amount of time between attempts to detect anomalies.

4. (Optional) Lookout for Metrics encrypts all imported data with an AWS Key Management Service (AWS KMS) key. To choose an encryption key, select Customize encryption settings. You can't change the encryption key later.

5. Choose Create.

Detector Statuses

After creating a detector, Lookout for Metrics initializes the detector, uses data for training, and activates the detector so that it can begin identifying anomalies. When you create a detector, Lookout for Metrics displays the following statuses:

1.Initializing - Lookout for Metrics begins initializing your anomaly detector based on your datasource, measure, and dimension conﬁgurations. It also ingests all historical data that you provided.

2.Learning - After initialization, the detector begins learning by ingesting continuous data. Learning time varies depending on the detector's interval setting and whether you provided historical data.

3.Active - After the detector has ingested a suﬃcient amount of data, the status changes to Active”and the detector begins identifying anomalies.

Lookout for Metrics usually requires at least 300 cumulative data points before a detector's status changes to Active. For example, if you provide 200 data points of historical data, the detector need to ingest 100 data points of continuous data before it is active. For the daily time interval, the detector's status changes to Active after 14 days of learning.

If no historical data is available, the learning stage takes the following amount of time:

(25)

Creating a dataset

• 5-minute interval: 25 hours

• 10-minute interval: 50 hours

• 1-hour interval: 12.5 days

• Daily interval: 14 days

If you are backtesting, also refer to the backtesting data requirements (p. 71).

Creating a dataset

A detector imports data from a datasource, transforms it, and stores it in a dataset. The datasource can be an Amazon Simple Storage Service (Amazon S3) bucket that you store data in, a database, or another AWS service that Lookout for Metrics supports.

Note

To communicate with other services, Lookout for Metrics needs permissions from an AWS Identity and Access Management (IAM) service role. When you use the console to conﬁgure a dataset, you can create a new service role or choose an existing one. For more information, see Service roles for Amazon Lookout for Metrics (p. 16).

To create a dataset

3. Choose Create dataset.

4. Provide the following information:

• Name – A name for the dataset.

• Description – A description of the dataset.

• Timezone – The timezone where the data is generated. When a detector analyzes data, it veriﬁes that each data point falls within the current interval.

5. Choose a datasource. The datasource can be an Amazon S3 bucket, an AWS service that Lookout for Metrics supports, or a database.

6. Conﬁgure the datasource and then choose Next. Options vary depending on the datasource that you choose. Common settings include the following:

• Interval – The amount of time between analysis attempts. Use the same setting as the detector's interval.

• Oﬀset – The minimum number of seconds that the detector waits at the end of each interval before accessing data. Using an oﬀset is supported only for Amazon S3 and Amazon Redshift datasources.

• Permissions – A service role (p. 16) that gives Lookout for Metrics permission to access either the datasource or a secret that contains credentials for the datasource.

7. Provide the metrics that the detector analyzes:

• Measures – The primary metrics that the detector analyzes to ﬁnd anomalies.

• Dimensions – The secondary metrics, which segment the data by, for example, location or demographic.

• Timestamp – The metric that speciﬁes when the data point was created.

8. Choose Next.

9. Choose Save and activate.

(26)

Creating a dataset

To get started with Amazon S3 as a datasource, see Managing a dataset in Amazon S3 (p. 24). For other datasources, see Using Amazon Lookout for Metrics with other services (p. 37).

(27)

Managing

Managing detectors

You manage detectors (p. 18) in the Lookout for Metrics console.

When you create a detector, you conﬁgure just a name, description, and interval. Optionally, you can choose an encryption key to use with the dataset's data, instead of the service's default key. After the detector is created, you can add a dataset (p. 24) and activate the detector to start ﬁnding anomalies.

Topics

• Viewing detector logs (p. 22)

• Deleting detectors (p. 22)

• Deactivating detectors (p. 22)

• Reactivating detectors (p. 23)

Viewing detector logs

When activation is complete, the detector analyzes data after each interval. If there are any anomalies in the data for an interval, the detector assigns each a severity score. If the score exceeds an alert target's threshold, the detector sends an alert to that target. You can view the results of each analysis in the detector log.

To view the detector log

3. Scroll down to the bottom of the page.

Deleting detectors

To stop analyzing data and delete it from Lookout for Metrics, delete the detector. This also deletes the detector's dataset and alerts.

To delete a detector

3. Choose Delete.

You can't recover a deleted detector or its subresources. Data stored in Amazon Lookout for Metrics is completely deleted within 90 days of the detector being deleted.

Deactivating detectors

You can deactivate detectors if its status is LEARNING or ACTIVE. To reconﬁgure a detector, you must ﬁrst deactivate the detector.

To deactivate a detector

(28)

Reactivating detectors

3. Choose Deactivate detector.

Deactivating a detector does not delete the detector or its subresources. If you are using the Lookout for Metrics SDK, use the DeactivateAnomalyDetector operation.

Reactivating detectors

Failed and inactive detectors can be edited and reactivated. You can reconﬁgure the detector and dataset before reactivating the detector. If a detector is active, you can deactivate, reconﬁgure, and then reactivate the detector.

A detector can only be reactivated if its status is FAILED or INACTIVE. Detectors that are INITIALIZING, LEARNING, or ACTIVE cannot be reactivated.

When reconﬁguring the detector, the Detector name (AnomalyDetectorName) and Tags ﬁelds are not editable. You can change the KMS key (KmsKeyArn) if the detector is inactive. If the detector failed, the KMS key cannot be changed.

When reconﬁguring the dataset, the Detector Arn (AnomalyDetectorArn), Dataset name (MetricSetName), and Tags ﬁelds are not editable.

To reactivate a detector

2. Choose the failed or inactive detector.

3. [Optional] To reconﬁgure the detector (p. 19), choose Edit under Create a detector.

4. [Optional] To reconﬁgure the dataset (p. 20), choose Edit under Add a dataset.

5. Choose Activate detector.

If you are using the Lookout for Metrics SDK, use the UpdateAnomalyDetector operation to edit the detector settings and the UpdateMetricSet operation to edit the dataset. Use ActivateAnomalyDetector to reactivate the detector.

(29)

Dataset

Managing a dataset in Amazon S3

You can use Amazon Simple Storage Service (Amazon S3) to store data for an Amazon Lookout for Metrics detector. With Amazon S3, you have complete control over your data's format and content. You can preprocess your data before handing it oﬀ to Lookout for Metrics, and aggregate data from multiple sources.

NoteFor information about using other AWS services as a datasource, see Using Amazon Lookout for Metrics with other services (p. 37).

You can provide two types of data to a detector: continuous data and historical data. A detector monitors continuous data to identify anomalies. You write continuous data to Amazon S3 as it is generated, to a path that represents the current interval. At the end of each interval, the detector reads data from the interval and analyzes it. The following example shows one possible path structure for continuous data with a 5-minute interval.

s3://my-lookoutmetrics-dataset-123456789012/

continuous/20201225/1520/data.jsonl continuous/20201225/1525/data.jsonl continuous/20201225/1530/data.jsonl

In this example, data for each 5-minute interval is stored in a single ﬁle named data.jsonl at a path that represents the interval. continuous/20201225/1520/ is a path for data generated in the 5- minute period starting at 3:20 PM on December 25th, 2020. Every 5 minutes, a new path is used.

Historical data is a collection of data stored at a single path in Amazon S3 that represents many previous intervals. You can provide historical data to a detector to train it prior to processing continuous data.

Historical data should have metrics from hundreds or thousands of intervals in one or more ﬁles. The following example shows historical data in separate ﬁles for each month at the path historical/.

historical/data-202009.jsonl historical/data-202010.jsonl historical/data-202011.jsonl

If you provide historical data (p. 26), but not continuous data, the detector operates in backtest mode.

Prior to creating an application or pipeline that generates continuous data, you can run a backtest to see how the detector works with your historical data.

You can store your data in CSV or JSON lines format. Both formats support one record per line of text.

With CSV format (p. 25), each field in a line is plain text separated by a comma or other supported delimiter. A CSV file can have a header row with field names, or define field names in the dataset. With JSON lines format (p. 25), each line is a JSON object with key-value pairs that define the name and value of each field.

Topics

• Structuring continuous and historical data (p. 25)

• CSV data (p. 25)

• JSON lines (p. 26)

• Providing historical data (p. 26)

• Path template keys (p. 26)

(30)

Structuring continuous and historical data

When you conﬁgure an Amazon S3 bucket as a datasource, you provide a path template that tells the detector where to ﬁnd the continuous data. Consider the following example path structure.

continuous/20201225/1520/

historical/

For historical data for this example, the path is s3://my-lookoutmetrics-

dataset-123456789012/historical. Lookout for Metrics looks for data ﬁles directly under historical and ignores subpaths.

For continuous data, the detector needs to know where to look for data for the current interval. The path template for the example structure is s3://my-lookoutmetrics-dataset-123456789012/

continuous/{{yyyyMMdd}}/{{HHmm}}. The letters in double brackets represent parts of the path that change depending on the date and time. Construct a path template with the following keys.

• yyyy – The 4-digit year

• MM – The 2- digit month

• HH – The 2-digit hour (in 24-hour format)

• mm – The 2-digit minute

For a complete list of supported keys, see Path template keys (p. 26).

Within a path for a single interval, data can be stored in one tor more text ﬁles. The detector uses only data with timestamps that fall within the interval for analysis. The detector uses the dataset's timezone to determine if data belongs to the current interval and ignores data that falls outside of the expected range.

CSV data

The following is an example of a correctly formatted CSV input ﬁle. Notice that this sample includes headers for each parameter (target_value is a metric and item_id is a dimension). Headers are optional, but recommended.

item_id,timestamp,target_value item_001,2020-05-07,1591.702780 item_002,2020-05-07,2342.481244 item_003,2020-05-07,1794.275162 item_004,2020-05-07,2716.692446 ...

The following is an example of a correctly formatted CSV ﬁle with headers. Here, there are two measures (target_1 and target_2) and two dimensions (item_id and store_id).

item_id,store_id,timestamp,target_1,target_2

item_001,store_001,2020-04-01 00:00:00,2117.0433697865165,27.521563807224712 item_002,store_002,2020-04-01 00:00:00,2221.312595828157,28.87706374576604 item_002,store_001,2020-04-01 00:00:00,4224.364287792719,54.91673574130534 item_003,store_002,2020-04-01 00:00:00,1420.3210031715096,18.464173041229625 item_001,store_002,2020-04-01 00:00:00,3222.8693491500876,41.89730153895114

(31)

JSON lines

...

JSON lines

The following is an example of a correctly formatted JSON lines input ﬁle, with target_value as a metric and item_id as a dimension.

{ "item_id":"item_001" , "timestamp" : "2020-05-07", "target_value" : 1591.702780 } { "item_id":"item_002" , "timestamp" : "2020-05-07", "target_value" : 2342.481244 } { "item_id":"item_003" , "timestamp" : "2020-05-07", "target_value" : 1794.275162 } { "item_id":"item_004" , "timestamp" : "2020-05-07", "target_value" : 2716.692446 } ...

The following is another example of a correctly formatted JSON lines ﬁle. Here, target1 and target2 are metrics and item_id and store_id are dimensions.

{"item_id": "item_001", "store_id": "store_001", "timestamp": "2020-04-01 00:00:00", "target1": 2117.0433697865165, "target2": 27.521563807224712}

...

Providing historical data

Your detector imports continuous data from your Amazon S3 bucket, stores it in its dataset, and uses it for learning. Learning is the process of analyzing data over multiple intervals to identify patterns and to distinguish between legitimate anomalies and uncommon but expected variations. A detector can also learn by using historical data.

To train a detector before it starts processing continuous data, you can provide historical data that represents up to 2,500 previous intervals. Historical data must fall within a timeframe that varies depending on the dataset's interval.

• 5-minute interval – 3 months

• 10-minute interval – 6 months

• 1-hour interval – 3 years

• 1-day interval – 5 years

If you don't specify a path for historical data when you create the dataset, the detector looks for data from previous intervals in the continuous data path. If available, it uses this data for learning to reduce the amount of time that it takes to start ﬁnding anomalies.

Path template keys

The following table lists the supported keys for continuous data path templates. A path template is an Amazon S3 URI that has placeholder keys in double curly brackets, which represent the folder names in the bucket that change for each interval.

(32)

Path template keys

Letter Date or time

component Presentation Examples

y Year Year 1996; 96

Y Week year Year 2009; 09

M Month in year Month July;Jul;07

d Day in month Number 10

a AM/PM marker Text PM

H Hour in day (0-23) Number 0

k Hour in day (1-24) Number 24

k (with AM/PM marker) Hour in AM/PM (0-11) Number 11

m Minute in hour Number 30

s Second in minute Number 55

Example path structure – daily interval

For this example, the path template is s3://my-lookoutmetrics-dataset-123456789012/

continuous/{{yyyy}}/{{MM}}/{{dd}}. The continuous folder has a subfolder structure that indicates the year, month, and day of each one day interval.

historical/

2020Q2.csv 2020Q3.csv continuous/

2020/12/01/

20201201-01.csv 2020/12/02/

20201202-01.csv 20201202-02.csv

(33)

Anomalies

Working with anomalies

An anomaly is a data point that the detector doesn't expect based on its understanding of your data.

An anomaly isn't necessarily good or bad, it's just unexpected. A detector learns over time to more accurately identify anomalies based on patterns that it ﬁnds in your data.

Lookout for Metrics organizes anomalies that aﬀect the same measure (p. 4) during the same interval groups and displays them as a single event. To get details about anomalies, view them in the Amazon Lookout for Metrics console.

To view anomalies

3. Choose Anomalies.

4. Use the options on the page to sort and ﬁlter the list of anomalies.

• Search bar – To ﬁlter the list of anomalies by title, enter the name of a measure.

• Severity threshold – To ﬁlter out lower-severity anomalies but maintain the display order, use the severity threshold slider.

• Sort columns – To sort by title, severity score, or timestamp, choose the name of the relevant column.

5. For details, choose an anomaly from the list.

The detector assigns a severity score between 0 and 100. The severity score indicates how far the data point is outside of the expected range based on the data that the detector has analyzed. The higher the severity score, the more unexpected the anomaly is.

Topics

• Causal relationships (p. 28)

• Aﬀected measures (p. 29)

• Providing feedback (p. 30)

• Contributing metrics (p. 31)

Causal relationships

The Causal relationships section identifies the measures in your datasets that are potentially causing anomalies at specific time points, and also identifies other measures that may be impacted by the anomaly.

During anomaly detection periods, Lookout for Metrics identifies measures that are likely causing particular anomalies, determines the direction of influence, and quantifies the level of contribution.

Causal relationships is divided into two sections: Potential causes and Eﬀects. Potential causes are measures that are potentially causing the inspected anomaly. The Eﬀects section shows the inspected anomaly and any measures that may be impacted by the anomaly.

(34)

Aﬀected measures

Potential causes

All measures listed under Potential causes are contributing to the anomaly. Measures with anomalies are anomalous measures, and you can use the links to learn more about those anomalies. Measures without anomalies are measures that do not contain anomalies, but still contribute to the anomaly you are inspecting.

You can gauge the impact each measure had in causing the anomaly by comparing the contribution percentages of the listed measures.

The unexplained portion of the anomaly is represented as “Unknown”. This is the contribution of factors that are not captured in the measures.

Eﬀects

The Eﬀects section shows the anomaly you are reviewing and any measures that may be impacted by the anomaly.

The anomaly you are reviewing is listed under This anomaly, and any measures that may be impacted by this anomaly are listed under Other impacted measures. You can use the links to learn more about the anomalies within those measures.

Aﬀected measures

The title of an anomaly indicates the measure that was aﬀected. If your dataset has dimensions, the values of each dimension are also part of the metric. In the following example, the number of views recorded was anomalous. The dataset has two dimensions, marketplace and platform, with values that indicate where the metric was recorded and the kind of device that was in use. The number of views was found to be aﬀected only in the UK marketplace (Uk) on mobile platforms (Mobile_web). The 100% for each dimension value indicates that the anomaly was only observed for one value within each dimension.

(35)

Providing feedback

Below the impact summary, there is a graph of the aﬀected metric over time. In this example, the number of mobile views in the UK dropped sharply between 7 PM and 8 PM. This drop was unexpected considering the marketplace, platform, and time of day.

Providing feedback

This drop could indicate an issue in the marketplace and platform, or an external factor that isn't represented in the data. To indicate whether the anomaly is relevant, use the feedback buttons above the graph. When the detector ﬁnds similar anomalies later, it will consider the feedback as it determines the severity score.

(36)

Contributing metrics

If the detector ﬁnds anomalies in multiple metrics for the same measure, it groups them together into a single event. In the following example, metrics for revenue on the PC platform are aﬀected on PC in both Italy and France, but to a greater degree in Italy.

In this case, revenue on PC in Italy is one metric, and revenue on PC in France is a second. If mobile revenue was also aﬀected in both marketplaces, there would be four metrics, and the platform dimension would show the impact of PC vs mobile revenue.

(37)

Alerts

Working with alerts

Amazon Lookout for Metrics detectors find anomalies in data. When an anomaly is severe, the detector can send details about it to another AWS service or resource. You can configure a detector to run an AWS Lambda function to process anomaly alerts, or send details to an Amazon Simple Notification Service (Amazon SNS) topic. Amazon SNS can then send the information to email subscribers or an HTTP endpoint, among numerous other supported destinations.

A severity threshold determines when the detector sends anomaly alerts. If you get anomaly alerts for anomalies that are not interesting, you can increase the threshold. If you don't get enough alerts, you can lower it.

To send anomaly alerts, a detector uses a service role (p. 16). When you use the console to create alerts, you can create a service role or choose one you already have. If you don't have permission to create roles, ask an administrator to create a service role for Lookout for Metrics.

To create an alert

3. Choose Add alert.

4. Conﬁgure the following options.

• Alert name – The name of the alert. Alert names must be unique across all detectors in a Region.

• Description – A description of the alert.

• Severity threshold – The severity score that the anomaly must reach for the detector to send an anomaly alert.

• Channel – The destination service.

• SNS topic or Lambda function – The resource in the target service that receives the anomaly alert.

• Role – A service role that allows the detector to send alerts to the resource.

5. Choose Create.

When the detector finds an anomaly that meets the severity threshold, it sends an anomaly alert. An anomaly alert is a JSON document with details about the metrics that were affected by the anomaly. The following example shows an anomaly alert where revenue is affected on the pc_web platform in two marketplaces, it (Italy) and fr (France).

Example alert (line endings and indentation added)

{ "alertName": "sns-alert",

"alertEventId": "arn:aws:lookoutmetrics:us-east-2:123456789012:Alert:sns- alert:event/61975451-xmpl-4639-a133-8a060535fc08",

"anomalyDetectorArn": "arn:aws:lookoutmetrics:us- east-2:123456789012:AnomalyDetector:my-detector",

"alertArn": "arn:aws:lookoutmetrics:us-east-2:123456789012:Alert:sns-alert", "alertDescription": "SNS alert for email notifications.",

"impactedMetric": {

"metricName": "revenue", "dimensionContribution": [ {

"dimensionName": "marketplace", "dimensionValueContributions": [ {

(38)

Alerts

"dimensionValue": "it", "valueContribution": 73 },

{

"dimensionValue": "fr", "valueContribution": 27 }

] }, {

"dimensionName": "platform", "dimensionValueContributions": [ {

"dimensionValue": "pc_web", "valueContribution": 100 }

] } ],

"relevantTimeSeries": [ {

"timeSeriesId":

"d488c9xmpl2e46c53e73ee853cc6258e499897c4b59d595805d28fb0d290d717bc57f10ca41df45de4063f5a8491e07d23c2c26ad1c6535c69ba96264b79b60f", "dimensions": [

{

"dimensionName": "marketplace", "dimensionValue": "it"

}, {

"dimensionName": "platform", "dimensionValue": "pc_web"

} ],

"metricValue": 148.5 },

{

"timeSeriesId":

"950feacxmpl22866d1f128991cf722b03f88f62f91770000af4b09bddb13d54691ce26346b03383d0c5c67bfc11a6c61853b71eee1c0ca47fb144f9caf5af39a", "dimensions": [

{

"dimensionName": "marketplace", "dimensionValue": "fr"

}, {

"dimensionName": "platform", "dimensionValue": "pc_web"

} ],

"metricValue": 230.1 }

] },

"timestamp": "2021-02-14T04:00Z[Atlantic/St_Helena]", "anomalyScore": 86.78

}

In the preceding example, the detector identiﬁed unusually high revenue values (148.5 and 230.1) in Italy and France for customers using the website on a PC. The value for Italy was farther from the expected value, so its contribution to the anomaly (73%) is greater than the value for France. Overall, the anomaly for the two metrics had an severity score of 86.78.

The severity score is a measurement of how unexpected the observed metric values are based on the detector's understanding of your data. It takes into consideration when the anomaly occurred, such as the time of day, and how many metrics were aﬀected.

(39)

Alerts

As the detector learns more about your data, anomalies for similar events might have higher or lower severity scores. You can guide its learning by providing feedback on aﬀected metrics in an anomaly. For more information, see Working with anomalies (p. 28).

(40)

Tagging Lookout for Metrics resources

Organize your Amazon Lookout for Metrics resources by owner, project or department with tags. Tags are key-value pairs that are supported across AWS services. You can use tags to ﬁlter resources, create least-privilege permissions policies, and add detail to billing reports. Lookout for Metrics also supports tag-based authorization. With tag-based authorization, you can create permissions policies (p. 14) that limit a user's access to resources with speciﬁc tags. For more information about tag-based authorization, see Controlling access to AWS resources using resource tags in the IAM User Guide.

You can add tags to detectors (p. 18), datasets (p. 24), and alerts (p. 32) when you create them, or you can add tags to existing resources. You can use the Lookout for Metrics console or manage tags with the Lookout for Metrics API (p. 35). Start by tagging your detectors to organize them into logical groups.

For more information about tags, see Tagging AWS resources in the Amazon Web Services General Reference.

To add tags to a detector in the Lookout for Metrics console 1. Open the Lookout for Metrics console Detectors page.

3. Choose Tags.

4. Choose Manage tags.

5. Enter a key and value. For example, Department and Marketing.

6. To add additional tags, choose Add tag.

7. Choose Save.

Tags apply to each detector, dataset, and alert individually. They are not shared or inherited.

Sections

• Using tags (Lookout for Metrics API) (p. 35)

• Tag key and value requirements (p. 36)

Using tags (Lookout for Metrics API)

When you create resources with the CreateAnomalyDetector, CreateMetricSet and CreateAlert operations, you can include tags with the --tags option. The following example shows how to apply tags when creating an anomaly detector with the AWS Command Line Interface (AWS CLI).

$ aws lookoutmetrics create-anomaly-detector --anomaly-detector-name my-detector \ --anomaly-detector-config AnomalyDetectorFrequency=PT10M \

--anomaly-detector-description "10-minute S3 detector" \ --tags Department=Marketing,CostCenter=1234ABCD

{ "AnomalyDetectorArn": "arn:aws:lookoutmetrics:us- east-2:123456789012:AnomalyDetector:my-detector"

}

To add tags to an existing resource, use the TagResource operation.

$ aws lookoutmetrics tag-resource --resource-arn arn:aws:lookoutmetrics:us- east-2:123456789012:AnomalyDetector:my-detector \

--tags Department=Marketing,CostCenter=1234ABCD

Amazon Lookout for Metrics

Amazon Lookout for Metrics

Developer Guide

Amazon Lookout for Metrics: Developer Guide

Table of Contents

What is Amazon Lookout for Metrics?

Are You a First-Time User of Lookout for Metrics?

Lookout for Metrics terms and concepts

Detector

Datasource

Dataset

Metrics

Alert

Feedback

Getting started with Lookout for Metrics

Create a datasource in Amazon S3

Collect existing data

Generate data with a sample application

Download sample data

Set up a detector

Create a detector

Add a dataset

Activate the detector

(Optional) Add an alert

Review anomalies

Clean up

Pricing

Charges for metrics

Charges for other services

Amazon Lookout for Metrics permissions

Identity-based IAM policies for Lookout for Metrics

Service roles for Amazon Lookout for Metrics

Sample IAM policies

Sample AWS CloudFormation templates

Working with detectors

Setting up a detector

Creating a detector

Detector Statuses

Creating a dataset

Managing detectors

Viewing detector logs

Deleting detectors

Deactivating detectors

Reactivating detectors

Managing a dataset in Amazon S3

Structuring continuous and historical data

CSV data

JSON lines

Providing historical data

Path template keys

Working with anomalies

Causal relationships

Potential causes

Eﬀects

Aﬀected measures

Providing feedback

Contributing metrics

Working with alerts

Tagging Lookout for Metrics resources

Using tags (Lookout for Metrics API)