Amazon Forecast
Developer Guide
Amazon Forecast: Developer Guide
Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.
Table of Contents
What Is Amazon Forecast? ... 1
Using Amazon Forecast ... 2
Features of Amazon Forecast ... 2
Pricing for Amazon Forecast ... 2
Are You a First-Time User of Amazon Forecast? ... 3
How Amazon Forecast Works ... 4
Setting Up ... 5
Sign Up for AWS ... 5
Set Up AWS CLI ... 5
Set Up Permissions ... 6
Create an IAM Role for Amazon Forecast (IAM Console) ... 6
Create an IAM for Amazon Forecast (AWS CLI) ... 8
Cross-service confused deputy prevention ... 10
Getting Started ... 11
Prepare Input Data ... 11
Getting Started (Console) ... 12
Getting Started (AWS CLI) ... 21
Getting Started (Python Notebooks) ... 30
Advanced Tutorials ... 31
Clean Up Resources ... 31
Tutorials ... 33
Automating with AWS CloudFormation ... 33
Prerequisites ... 35
Deploying an AWS CloudFormation Template for Forecast automation ... 35
Clean Up ... 38
Importing Datasets ... 39
Datasets ... 39
Dataset Domains and Dataset Types ... 40
Dataset Schema ... 41
Dataset Groups ... 42
Resolving Conflicts in Data Collection Frequency ... 42
Time Boundaries ... 43
Data Aggregation Guidelines ... 43
Related Time Series ... 44
Historical and Forward-looking Related Time Series ... 44
Related Time Series Dataset Validation ... 45
Example: Forward-looking Related Time Series File ... 45
Example: Forecasting Granularity ... 46
Legacy Predictors and Related Time Series ... 47
Item Metadata ... 47
Example: Item Metadata File and Schema ... 48
Legacy Predictors and Item Metadata ... 49
See Also ... 49
Predefined Dataset Domains and Dataset Types ... 49
RETAIL Domain ... 51
CUSTOM Domain ... 52
INVENTORY_PLANNING Domain ... 53
EC2 CAPACITY Domain ... 54
WORK_FORCE Domain ... 55
WEB_TRAFFIC Domain ... 56
METRICS Domain ... 57
Updating Data ... 57
Handling Missing Values ... 58
Choosing Filling Logic ... 58
Target Time Series and Related Time Series Filling Logic ... 59
Missing Value Syntax ... 60
Dataset Guidelines ... 61
Training Predictors ... 63
Creating a Predictor ... 63
Upgrading to AutoPredictor ... 65
Using additional datasets ... 65
Working with legacy predictors ... 66
Predictor Metrics ... 67
Interpreting Accuracy Metrics ... 67
Weighted Quantile Loss (wQL) ... 68
Weighted Absolute Percentage Error (WAPE) ... 69
Root Mean Square Error (RMSE) ... 70
Mean Absolute Percentage Error (MAPE) ... 70
Mean Absolute Scaled Error (MASE) ... 71
Exporting Accuracy Metrics ... 71
Choosing Forecast Types ... 73
Working With Legacy Predictors ... 75
Retraining Predictors ... 78
Weather Index ... 79
Enabling the Weather Index ... 79
Adding Geolocation Information to Datasets ... 80
Specifying Time Zones ... 93
Conditions and Restrictions ... 98
Holidays Featurization ... 99
Enabling the Holidays Featurization ... 99
Country Codes ... 100
Additional Holiday Calendars ... 102
Predictor Explainability ... 103
Interpreting Impact Scores ... 103
Creating Predictor Explainability ... 104
Exporting Predictor Explainability ... 105
Restrictions and best practices ... 106
Forecast Algorithms ... 107
Built-in Forecast Algorithms ... 107
Comparing Forecast Algorithms ... 108
ARIMA ... 109
CNN-QR ... 110
DeepAR+ ... 114
ETS ... 120
NPTS ... 121
Prophet ... 124
Generating Forecasts ... 125
Forecast Explainability ... 126
Interpreting Impact Scores ... 126
Creating Forecast Explainability ... 127
Specifying time series ... 128
Specifying time points ... 129
Visualizing Forecast Explainability ... 131
Exporting Forecast Explainability ... 131
Restrictions and best practices ... 132
Managing Resources ... 134
Stopping Resources ... 134
Deleting Resources ... 136
Understanding Resource Trees ... 136
Deleting Individual Resources ... 137
Deleting Resource Trees ... 138
Tagging Resources ... 139
Managing Tags ... 140
Using Tags in IAM Policies ... 140
Adding Tags to Resources ... 141
Additional Information ... 142
Receiving Notifications ... 142
Monitoring Forecast Resource Jobs ... 143
Creating an EventBridge Rule for Job Status Notifications ... 144
Creating a CloudWatch Events Rule for Job Status Notifications ... 144
Guidelines and Quotas ... 145
Supported AWS Regions ... 145
Compliance ... 145
Service Quotas ... 145
Reserved Field Names ... 149
Security ... 169
Data Protection ... 169
Encryption at Rest ... 170
Encryption in Transit ... 170
Key Management ... 170
Identity and Access Management ... 170
Audience ... 170
Authenticating With Identities ... 171
Managing Access Using Policies ... 173
How Amazon Forecast Works with IAM ... 174
Identity-Based Policy Examples ... 177
Troubleshooting ... 181
Logging and Monitoring ... 183
Logging Forecast API Calls with AWS CloudTrail ... 183
CloudWatch Metrics for Amazon Forecast ... 185
Compliance Validation ... 186
Resilience ... 186
Infrastructure Security ... 187
VPC endpoints (AWS PrivateLink) ... 187
Considerations for Forecast VPC endpoints ... 187
Creating an interface VPC endpoint for Forecast ... 188
Creating a VPC endpoint policy for Forecast ... 188
API Reference ... 190
Actions ... 190
Amazon Forecast Service ... 191
Amazon Forecast Query Service ... 333
Data Types ... 336
Amazon Forecast Service ... 337
Amazon Forecast Query Service ... 403
Common Errors ... 405
Common Parameters ... 407
Document History ... 409
AWS glossary ... 411
What Is Amazon Forecast?
Amazon Forecast is a fully managed service that uses statistical and machine learning algorithms to deliver highly accurate time-series forecasts. Based on the same technology used for time-series forecasting at Amazon.com, Forecast provides state-of-the-art algorithms to predict future time-series data based on historical data, and requires no machine learning experience.
Time-series forecasting is useful in multiple fields, including retail, finance, logistics, and healthcare. You can also use Forecast to predict domain-specific metrics for your inventory, workforce, web traffic, server capacity, and finances.
For more information about the technical aspects of Amazon Forecast, see Time Series Forecasting Principles with Amazon Forecast.
Using Amazon Forecast
Topics
• Using Amazon Forecast (p. 2)
• Features of Amazon Forecast (p. 2)
• Pricing for Amazon Forecast (p. 2)
• Are You a First-Time User of Amazon Forecast? (p. 3)
Using Amazon Forecast
You can use the APIs (p. 190), AWS Command Line Interface (p. 21) (AWS CLI), Python Software Development Kit (p. 30) (SDK), and Amazon Forecast console (p. 12) to import time series datasets, train predictors, and generate forecasts.
Here are some common use cases for Amazon Forecast:
• Retail demand planning – Predict product demand, allowing you to more accurately vary inventory and pricing at different store locations.
• Supply chain planning – Predict the quantity of raw goods, services, or other inputs required by manufacturing.
• Resource planning – Predict requirements for staffing, advertising, energy consumption, and server capacity.
• Operational planning – Predict levels of web traffic, AWS usage, and IoT sensor usage.
Features of Amazon Forecast
Amazon Forecast automates much of the time-series forecasting process, enabling you to focus on preparing your datasets and interpretting your predictions.
Forecast provides the following features:
• Automated machine learning – Forecast automates complex machine learning tasks by finding the optimal combination of machine learning algorithms for your datasets.
• State-of-the-art algorithms – Apply a combination of machine learning algorithms that are based on the same technology used at Amazon.com. Forecast offers a wide range of training algorithms, from commonly used statistical methods to complex neural networks.
• Missing value support – Forecast provides several filling methods to automatically handle missing values in your datasets.
• Additional built-in datasets – Forecast can automatically incorporate built-in datasets to
improve your model. These datasets are already feature engineered and do not require additional configuration.
Pricing for Amazon Forecast
With Amazon Forecast, you pay only for what you use. There are no minimum fees and no upfront commitments. The costs of Amazon Forecast depend on the number generated forecasts, data storage, and training hours.
The AWS Free Tier allows you a monthly limit of up to 10,000 time series forecasts, up to 10GB of storage, and up to 10 hours of training time. The Amazon Forecast free tier is valid for the first two months of usage.
Are You a First-Time User of Amazon Forecast?
For a complete list of charges and prices, see Amazon Forecast pricing.
Are You a First-Time User of Amazon Forecast?
If you are a first-time user of Amazon Forecast, we recommend that you start with the following pages:
1.How Amazon Forecast Works (p. 4) – Learn about the key concepts and the process of importing datasets, creating predictors, and generating forecasts.
2.Getting Started (p. 11) – Follow one of the tutorials to create your first Amazon Forecast forecasting predictor.
3.API Reference (p. 190) – Familiarize yourself with the Amazon Forecast API actions and data types.
How Amazon Forecast Works
When creating forecasting projects in Amazon Forecast, you work with the following resources:
• Importing Datasets (p. 39) – Datasets are collections of your input data. Dataset groups are
collections of datasets that contain complimentary information. Forecast algorithms use your dataset groups to train custom forecasting models, called predictors.
• Training Predictors (p. 63) – Predictors are custom models trained on your data. You can train a predictor by choosing a prebuilt algorithm,or by choosing the AutoML option to have Amazon Forecast pick the best algorithm for you.
• Generating Forecasts (p. 125) – You can generate forecasts for your time-series data, query them using the QueryForecast API, or visualize them in the console.
Sign Up for AWS
Setting Up
Before using Amazon Forecast to evaluate or forecast time-series data, create an AWS account, configure access permissions, and set up the AWS Command Line Interface (AWS CLI).
Topics
• Sign Up for AWS (p. 5)
• Set Up the AWS CLI (p. 5)
• Set Up Permissions for Amazon Forecast (p. 6)
Sign Up for AWS
When you sign up for Amazon Web Services (AWS), your AWS account is automatically signed up for all services in AWS, including Amazon Forecast. You are charged only for the services that you use.
If you already have an AWS account, skip to the next task. If you don't have an AWS account, use the following procedure to create one.
To sign up for AWS
1. Open https://aws.amazon.com/, and then choose Create an AWS Account.
2. Follow the on-screen instructions to complete the account creation. Note your 12-digit AWS account number. Part of the sign-up procedure involves receiving a phone call and entering a PIN using the phone keypad.
3. Create an AWS Identity and Access Management (IAM) admin user. See Creating Your First IAM User and Group in the AWS Identity and Access Management User Guide for instructions.
Set Up the AWS CLI
The AWS Command Line Interface (AWS CLI) is a unified developer tool for managing AWS services, including Amazon Forecast. We recommend that you install and use it.
1. To install the AWS CLI, follow the instructions in Installing the AWS Command Line Interface in the AWS Command Line Interface User Guide.
2. To configure the AWS CLI and set up a profile to call it, follow the instructions in Configuring the AWS CLI in the AWS Command Line Interface User Guide.
3. To confirm that the AWS CLI profile is configured correctly, run the following command in a command window:
aws configure --profile default
If your profile has been configured correctly, you should see output similar to the following:
AWS Access Key ID [****************52FQ]:
AWS Secret Access Key [****************xgyZ]:
Default region name [us-west-2]:
Default output format [json]:
4. To verify that the AWS CLI is configured for use with Amazon Forecast, run the following commands.
Set Up Permissions
aws forecast help
aws forecastquery help
If the AWS CLI is configured correctly, you will see a list of the supported CLI commands for Amazon Forecast or Amazon Forecast Query.
Set Up Permissions for Amazon Forecast
Amazon Forecast uses Amazon Simple Storage Service (Amazon S3) to store the target time-series data that are used to train predictors that can generate forecasts. To access Amazon S3 on your behalf, Amazon Forecast needs your permission.
To grant Amazon Forecast permission to use Amazon S3 on your behalf, you must have an AWS Identity and Access Management (IAM) role and IAM policy in your account. The IAM policy specifies the required permissions, and must be attached to the IAM role.
To create the IAM role and policy and to attach the policy to the role, you can use the IAM console or the AWS Command Line Interface (AWS CLI).
NoteForecast does not communicate with AWS VPCs and is unable to support the S3 VPCE gateway.
Using S3 buckets that only allow VPC access will result in an AccessDenied error.
Topics
• Create an IAM Role for Amazon Forecast (IAM Console) (p. 6)
• Create an IAM for Amazon Forecast (AWS CLI) (p. 8)
• Cross-service confused deputy prevention (p. 10)
Create an IAM Role for Amazon Forecast (IAM Console)
You can use the AWS IAM console to do the following:
• Create an IAM role with Amazon Forecast as a trusted entity
• Create an IAM policy with permissions that allows Amazon Forecast to show, read, and write data in an Amazon S3 bucket
• Attach the IAM policy to the IAM role
To create an IAM role and policy that allows Amazon Forecast to access Amazon S3 (IAM console)
1. Sign in to the IAM console (https://console.aws.amazon.com/iam).
2. Choose Policies and do the following to create the required policy:
a. On the Create policy page, in the policy editor, choose the JSON tab.
b. Copy the following policy and replace the text in the editor by pasting the this policy over it. Be sure to replace bucket-name with the name of your S3 bucket, then choose Review policy.
Create an IAM Role for Amazon Forecast (IAM Console)
{ "Version":"2012-10-17", "Statement":[
{
"Effect":"Allow", "Action":[
"s3:Get*", "s3:List*", "s3:PutObject"
],
"Resource":[
"arn:aws:s3:::bucket-name", "arn:aws:s3:::bucket-name/*"
] } ]}
c. In Review policy, for Name, enter a name for the policy. For example, AWSS3BucketAccess.
Optionally, provide a description for this policy, then choose Create policy.
3. In the navigation pane, choose Roles. Then do the following to create the IAM role:
a. Choose Create role.
b. For Select type of trusted entity, choose AWS service.
c. For Choose the service that will use this role, if you don't see Amazon Forecast listed, choose EC2. Otherwise, choose Amazon Forecast.
d. Choose Next: Permissions.
e. For Attach permissions policies, choose the check box next to the policy that you just created.
To display the policy in the list, type part of your policy name in the Filter policies query filter.
Then, choose Next: Tags.
f. You don't need to add tags, so choose Next: Review.
g. In the Review section, for Role name, enter a name for the role (for example, ForecastRole).
Update the description for the role in Role description, then choose Create role.
h. Choose the new role to open the role's details page.
i. In the Summary, copy the Role ARN value and save it. You need it to import a dataset into Amazon Forecast.
j. If you didn't choose Amazon Forecast as the service that will use this role, choose Trust relationships, and then choose Edit trust relationship to update the trust policy as follows.
{ "Version": "2012-10-17", "Statement": [
{
"Sid": "",
"Effect": "Allow", "Principal": {
"Service": "forecast.amazonaws.com"
},
"Action": "sts:AssumeRole", "Condition": {
"StringEquals": {
"aws:SourceAccount": "account-id"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:forecast:region:account-id:*"
} } }
Create an IAM for Amazon Forecast (AWS CLI)
] }
k. [OPTIONAL] When using a KMS key to enable encryption, attach the KMS key and ARN:
{ "Version": "2012-10-17", "Statement": [
{
"Sid": "ForecastKMS", "Effect": "Allow", "Action": "kms:*",
"Resource": "arn:aws:kms:region:account-id:key/KMS-key-id"
} ] }
Create an IAM for Amazon Forecast (AWS CLI)
You can use the AWS CLI to do the following:
• Create an IAM role with Amazon Forecast as a trusted entity
• Create an IAM policy with permissions that allows Amazon Forecast to show, read, and write data in an Amazon S3 bucket
• Attach the IAM policy to the IAM role
To create an IAM role and policy that allows Amazon Forecast to access Amazon S3 (AWS CLI) 1. Create an IAM role with Amazon Forecast as a trusted entity that can assume the role for you:
aws iam create-role \ --role-name ForecastRole \ --assume-role-policy-document '{
"Version": "2012-10-17", "Statement": [
{
"Sid": "",
"Effect": "Allow", "Principal": {
"Service": "forecast.amazonaws.com"
},
"Action": "sts:AssumeRole", "Condition": {
"StringEquals": {
"aws:SourceAccount": "account-id"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:forecast:region:account-id:*"
} } } ]}'
This command assumes that the default AWS configuration profile is targeted for an AWS Region supported by Amazon Forecast. If you have configured another profile (for example, aws-
forecast) to target an AWS Region that is not supported by Amazon Forecast, you must explicitly specify that configuration by including the profile parameter in the command, for example, --
Create an IAM for Amazon Forecast (AWS CLI)
profile aws-forecast. For more information about setting up an AWS CLI configuration profile, see the AWS CLI configure command.
If the command successfully creates the role, it returns it as output, which should look similar to the following:
{
"Role": {
"Path": "/",
"RoleName": "ForecastRole", "RoleId": your-role-ID,
"Arn": "arn:aws:iam::your-acct-ID:role/ForecastRole", "CreateDate": "creation-date",
"AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [
{
"Sid": "",
"Effect": "Allow", "Principal": {
"Service": "forecast.amazonaws.com"
},
"Action": "sts:AssumeRole", "Condition": {
"StringEquals": {
"aws:SourceAccount": "your-acct-ID"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:forecast:region:your-acct-ID:*"
} } } ] } } }
Record the role's ARN. You need it when you import a dataset to train an Amazon Forecast predictor.
2. Create an IAM policy with permissions to list, read, and write data in Amazon S3, and attach it to the IAM role that you created in Step 1:
aws iam put-role-policy \ --role-name ForecastRole \
--policy-name ForecastBucketAccessPolicy \ --policy-document '{
"Version":"2012-10-17", "Statement":[
{
"Effect":"Allow", "Action":[
"s3:Get*", "s3:List*", "s3:PutObject"
],
"Resource":[
"arn:aws:s3:::bucket-name", "arn:aws:s3:::bucket-name/*"
] } ] }'
Cross-service confused deputy prevention
3. [OPTIONAL] When using a KMS key to enable encryption, attach the KMS key and ARN:
aws iam put-role-policy \ --role-name ForecastRole \
--policy-name ForecastBucketAccessPolicy \ --policy-document '{
"Version":"2012-10-17", "Statement":[
{
"Effect":"Allow", "Action":[
"s3:Get*", "s3:List*", "s3:PutObject"
],
"Resource":[
"arn:aws:s3:::bucket-name", "arn:aws:s3:::bucket-name/*"
] }
]}'aws iam put-role-policy \ --role-name ForecastRole \
--policy-name ForecastKMSAccessPolicy \ --policy-document ‘{
"Version":"2012-10-17", "Statement":[
{
"Effect":"Allow", "Action":[
"kms:*"
],
"Resource":[
"arn:aws:kms:region:account-id:key/KMS-key-id"
] } ]}’
Cross-service confused deputy prevention
The confused deputy problem is a security issue where an entity that doesn’t have permission to perform an action can coerce a more-privileged entity to perform the action. In AWS, cross-service impersonation can result in the confused deputy problem. Cross-service impersonation can occur when one service (the calling service) calls another service (the called service). The calling service can be manipulated to use its permissions to act on another customer’s resources in a way it should not otherwise have permission to access. To prevent this, AWS provides tools that help you protect your data for all services with service principals that have been given access to resources in your account.
We recommend using the aws:SourceArn and aws:SourceAccount global condition context keys in resource policies to limit the permissions that Identity and Access Management (IAM) gives Amazon Forecast access to your resources. If you use both global condition context keys, the aws:SourceAccount value and the account in the aws:SourceArn value must use the same account id when used in the same policy statement.
Prepare Input Data
Getting Started
To get started using Amazon Forecast, you do the following.
• Create a Forecast dataset and import training data.
• Create a Forecast predictor, which you use generate forecasts based on your time-series data. Forecast applies the optimal combination of algorithms to each time series in your datasets.
• Generate a forecast.
In this exercise, you use a modified version of a publicly available electricity usage dataset to train a predictor. For more information, see ElectricityLoadDiagrams20112014 Data Set. The following are sample rows from the dataset:
2014-01-01 01:00:00, 2.53807106598985, client_0 2014-01-01 01:00:00, 23.648648648648624, client_1 2014-01-01 02:00:00, 9.648648648612345, client_0
For this exercise, you use the dataset to train a predictor, and then predict the hourly electricity usage by client.
You can use either the Forecast console or the AWS Command Line Interface (AWS CLI) for this exercise.
Pay attention to the default regions of the Amazon Forecast console, the AWS CLI, and the Amazon Forecast SDKs, as Amazon Forecast resources are not shared across regions.
Important
Before you begin, make sure that you have an AWS account and have installed the AWS CLI.
For more information, see Setting Up (p. 5). We also recommend that you review How Amazon Forecast Works (p. 4).
Topics
• Prepare Input Data (p. 11)
• Getting Started (Console) (p. 12)
• Getting Started (AWS CLI) (p. 21)
• Getting Started (Python Notebooks) (p. 30)
• Clean Up Resources (p. 31)
Prepare Input Data
Regardless of whether you use the Amazon Forecast console or the AWS Command Line Interface (AWS CLI) to set up a forecasting project, you need to set up your input data. To prepare your data, you do the following:
• Download training data to your computer and upload it to an Amazon Simple Storage Service (Amazon S3) bucket in your AWS account. To import your data to an Amazon Forecast dataset, you must store it in an Amazon S3 bucket.
• Create an AWS Identity and Access Management (IAM) role. You give Amazon Forecast permission to access your S3 bucket with the IAM role. For more information about IAM roles, see IAM Roles in the IAM User Guide.
Getting Started (Console)
To prepare training data
1. Download the zip file, electricityusagedata.zip.
For this exercise, you use the individual household electric power consumption dataset. (Dua, D. and Karra Taniskidou, E. (2017). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.) We aggregate the usage data hourly.
2. Unzip the content and save it locally as electricityusagedata.csv.
3. Upload the data file to an S3 bucket.
For step-by-step instructions, see Uploading Files and Folders by Using Drag and Drop in the Amazon Simple Storage Service User Guide.
4. Create an IAM role.
If you want to use the AWS CLI for the Getting Started exercise, you must create an IAM role. If you use the console, you can have it create the role for you. For step-by-step instructions, see Set Up Permissions for Amazon Forecast (p. 6).
Now, use the Amazon Forecast console or the AWS CLI to train a predictor, generate a forecast, and see the forecast.
• Getting Started (Console) (p. 12)
• Getting Started (AWS CLI) (p. 21)
Getting Started (Console)
In this exercise, you use the Amazon Forecast console to import time-series data of electricity usage, create an predictor based on the input dataset, and make predictions of future electricity usage based on the forecast horizon.
For this exercise, we use the individual household electric power consumption dataset. (Dua, D. and Karra Taniskidou, E. (2017). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA:
University of California, School of Information and Computer Science.) We aggregate the usage data hourly.
Prerequisites
• An AWS account. If you don't already have an AWS account, create one as described in Sign Up for AWS (p. 5).
• Training data in your Amazon Simple Storage Service (Amazon S3) bucket. For more information, see Prepare Input Data (p. 11).
• An AWS Identity and Access Management (IAM) role that allows Amazon Forecast to read and write to your S3 buckets. For more information, see Create an IAM Role for Amazon Forecast (IAM Console) (p. 6).
Step 1: Import Training Data
To import time-series data into Amazon Forecast, create a dataset group, choose a domain for your dataset group, specify the details of your data, and point Amazon Forecast to the S3 location of your data. You use a time series of historical electricity usage (p. 11) as an example for the target time series data.
Getting Started (Console)
NoteThis exercise assumes that you haven't created any dataset groups. If you previously created a dataset group, what you see will vary slightly from the following screenshots and instructions.
To import time-series data for forecasting
1. Open the Amazon Forecast console at https://console.aws.amazon.com/forecast/.
2. On the Amazon Forecast home page, choose Create dataset group.
3. On the Create dataset group page, for Dataset group details, provide the following information:
• Dataset group name – Enter a name for your dataset group.
• Forecasting domain – From the drop-down menu, choose Custom. For more information about how to choose a forecasting domain, see How Amazon Forecast Works (p. 4) and dataset domains and types (p. 49).
Leave the Tags section unchanged. Your screen should look similar to the following:
4. Choose Next.
5. On the Create target time series dataset page, for Dataset details, provide the following information:
• Dataset name – Enter a name for your dataset.
Getting Started (Console)
• Frequency of your data – Keep the default value of 1, and choose hour from the drop-down menu. This setting must be consistent with the input time series data. The time interval in the sample electricity-usage data is an hour.
• Data schema – Choose Schema builder and drag the column components to match the time- series data order from top to bottom: first timestamp, then target_value, then item_id. For the electricity usage input data, the columns correspond to: a timestamp, the electricity usage at the specified time (target_value), and the ID of the customer charged for the electricity usage (string). For Timestamp Format, Leave the default (yyyy-MM-dd HH:mm:ss). The format must be consistent with the input time series data.
The Dataset details panel should look similar to the following:
6. For Dataset import job details, provide the following information:
• Dataset import job name – Enter a name for your dataset.
• Select time zone – Leave the default selected (Do not use time zone).
• Data location – Use the following format to enter the location of your .csv file on Amazon S3:
s3://<name of your S3 bucket>/<folder path>/<filename.csv>
• IAM role – Keep the default Enter a custom IAM role ARN.
Alternatively, you can have Amazon Forecast create the required IAM role for you by choosing Create a new role from the drop-down menu and following the on-screen instructions.
Getting Started (Console)
• Custom IAM role ARN – Enter the Amazon Resource Name (ARN) of the IAM role that you created in Create an IAM Role for Amazon Forecast (IAM Console) (p. 6).
The Dataset import details panel should look similar to the following:
7. Choose Start.
8. The dataset group's Dashboard page is displayed. Your screen should look similar to the following:
Next to Target time series data, you will see the status of the import job. Wait for Amazon Forecast to finish importing your time series data. The process can take several minutes or longer. When
Getting Started (Console)
your dataset has been imported, the status transitions to Active and the banner at the top of the dashboard notifies you that you have successfully imported your data.
Now that your target time series dataset has been imported, you can create a predictor.
Step 2: Create a predictor
Next you create a predictor, which you use generate forecasts based on your time series data. Forecast applies the optimal combination of algorithms to each time series in your datasets
To create a predictor with the Forecast console, you specify a predictor name and a forecast frequency, and define a forecast horizon. For more information about the additional fields you can configure, see Training Predictors (p. 63).
To create a predictor
1. After your target time series dataset has finished importing, your dataset group's Dashboard should look similar to the following:
Under Train a predictor, choose Start. The Train predictor page is displayed.
NoteThe Status of the Target time series data must be Active, which signifies that the import successfully finished, before you can train the predictor.
2. On the Train predictor page, for Predictor settings, provide the following information:
• Predictor name – Enter a name for your predictor.
• Forecast frequency – Keep the default value of 1. From the drop-down menu, choose hour.
This setting must be consistent with the input time series data. The time interval in the sample electricity-usage data is an hour.
• Forecast horizon – Choose how far into the future to make predictions. This number multiplied by the data entry frequency (hourly) that you specified in Step 1: Import the Training Data determines how far into the future to make predictions. For this exercise, set the number to 36, to provide predictions for 36 hours.
Getting Started (Console)
• Forecast dimensions and Forecast quantiles – Leave the default values for these fields.
The remaining Input data configuration and Tags sections are optional, so leave the default values.
The Predictor settings sections should look similar to the following:
3. Choose Create. Your dataset group's Dashboard page is displayed. Your screen should look similar to the following:
Getting Started (Console)
4. To find the status of your predictor, choose View predictors.
5. On the Predictors page find the status of your predictor in the Training status column. Your screen should look similar to the following:
Wait for Amazon Forecast to finish training the predictor. The process can take several minutes or longer. When your predictor has been trained, the status transitions to Active and a banner displays notifying you that you can start generating forecasts.
Step 3: Create a Forecast
After your predictor is Active, you can create a forecast. A forecast is a group of predictions, one for every item in the target dataset. To retrieve the complete forecast, you create an export job.
To get and view your forecast
1. On your dataset group's Dashboard, under Forecast generation, choose Start. The Create a forecast page is displayed.
NoteThe Status of Predictor training must be Active before you can generate a forecast.
2. On the Create a forecast page, for Forecast details, provide the following information:
• Forecast name – Enter a name for your forecast.
• Predictor – From the drop-down menu, choose the predictor that you created in Step 2: Train a Predictor.
Getting Started (Console)
The Forecast quantiles and Tags fields are optional, so leave the default value. Your screen should look similar to the following:
3. Choose Create a forecast. The Forecasts page is displayed. Your screen should look similar to the following:
The Status column lists the status of your forecast.. Wait for Amazon Forecast to finish creating the forecast. The process can take several minutes or longer. When your forecast has been created, the status transitions to Active.
Now that your forecast has been created, you can export the forecast.
Step 4: Exporting a Forecast
After the forecast has been created, you can export the complete forecast.
To export the complete forecast
1. In the navigation pane, under your dataset group, choose Forecasts.
2. Choose the radio button next to the forecast that you created in Step 3: Create a Forecast.
3. Choose Create forecast export. The Create forecast export page is displayed.
4. On the Create forecast export page, for Export details, provide the following information.
• Export name – Enter a name for your forecast export job.
Getting Started (Console)
• Generated forecast – From the drop-down menu, choose the forecast that you created in Step 3: Create a Forecast.
• IAM role – Keep the default Enter a custom IAM role ARN.
Alternatively, you can have Amazon Forecast create the required IAM role for you by choosing Create a new role from the drop-down menu and following the on-screen instructions.
• Custom IAM role ARN – Enter the Amazon Resource Name (ARN) of the IAM role that you created in Create an IAM Role for Amazon Forecast (IAM Console) (p. 6).
• S3 forecast export location – Use the following format to enter the location of your Amazon Simple Storage Service (Amazon S3) bucket or folder in the bucket:
s3://<name of your S3 bucket>/<folder path>/
Your screen should look similar to the following:
5. Choose Create forecast export. The my_forecast page is displayed.
Getting Started (AWS CLI)
Your screen should look similar to the following:
You should see the status progress. Wait for Amazon Forecast to finish exporting the forecast.
The process can take several minutes or longer. When your forecast has been exported, the status transitions to Active and you can find the forecast files in your S3 bucket.
Getting Started (AWS CLI)
In this exercise, you use the AWS Command Line Interface (AWS CLI) to explore Amazon Forecast. You create an Amazon Forecast dataset, train an predictor, and use the resulting predictor to generate a forecast. Before you begin, make sure that you have an AWS account and that you've set up the AWS CLI.
For more information, see Setting Up (p. 5).
Note
The CLI commands in this exercise were tested on Linux. For information about using the CLI commands on Windows, see Specifying Parameter Values for the AWS Command Line Interface in the AWS Command Line Interface User Guide.
Step 1: Import Training Data
Begin by creating a dataset and importing the electricity usage data into it.
To create an Amazon Forecast dataset
1. Decide which domain and dataset type is appropriate.
The training data that you will import into the dataset influences your choice of dataset domain and type. So, let's review a few sample rows of the electricity usage data.
2014-01-01 01:00:00, 2.53807106598985, client_0 2014-01-01 01:00:00, 23.648648648648624, client_1 2014-01-01 02:00:00, 9.648648648612345, client_0
The data format is CSV (comma-separated values), and it's collected hourly (as shown by the timestamps). It includes these columns:
• Column 1 – Timestamps that show when electricity usage was recorded.
• Column 2 – Hourly electricity usage values (note how the timestamp values increase by hour).
• Column 3 – Client ID values that identify the customers using the electricity.
For this data, choose the following predefined dataset domain and dataset type:
• Custom domain – None of the dataset domains, such as METRICS, RETAIL, or WEB_TRAFFIC, applies to this data, so choose the Custom domain.
Getting Started (AWS CLI)
• Target time series type – The data is a time series because it tracks electricity usage over time. It also includes the target that we want to forecast (Column 2, electricity usage). Therefore, choose the target time series dataset type.
To understand why you choose this type, see Predefined Dataset Domains and Dataset Types (p. 49).
2. Decide on a dataset schema.
The target time series type for the CUSTOM Domain (p. 52) requires these fields; timestamp, target_value, and item_id. The target_value field is the target. Amazon Forecast generates the forecast for this field.
To map the required fields to columns in your data, you create a schema. Each attribute in the schema maps to a field in the data.
Important
The order of attributes in the schema must match the order of fields in the training data.
{
"Attributes":[
{
"AttributeName": "timestamp", "AttributeType": "timestamp"
}, {
"AttributeName": "target_value", "AttributeType": "float"
}, {
"AttributeName": "item_id", "AttributeType": "string"
} ]}
You now have the information necessary to create a dataset and import data into it.
3. Create the dataset.
aws forecast create-dataset \
--dataset-name electricity_demand_ds \ --domain CUSTOM \
--dataset-type TARGET_TIME_SERIES \ --data-frequency H \
--schema '{
"Attributes": [ {
"AttributeName": "timestamp", "AttributeType": "timestamp"
}, {
"AttributeName": "target_value", "AttributeType": "float"
}, {
"AttributeName": "item_id", "AttributeType": "string"
} ]}'
Getting Started (AWS CLI)
In the request, the data-frequency value H represents a data collection frequency of hourly. The following is an example response.
{
"DatasetArn": "arn:aws:forecast:us-west-2:acct-id:dataset/electricity_demand_ds"
}
For more information about this operation, see CreateDataset (p. 198).
4. (Optional) Get the description of the dataset.
aws forecast describe-dataset \
--dataset-arn arn:aws:forecast:us-west-2:acct-id:dataset/electricity_demand_ds
The following is an example response.
{ "DatasetName": "electricity_demand_ds",
"DatasetArn": "arn:aws:forecast:us-west-2:acct-id:dataset/electricity_demand_ds", "CreationTime": 1564533087.907,
"LastModificationTime": 1564533087.907, "Domain": "CUSTOM",
"DatasetType": "TARGET_TIME_SERIES", "DataFrequency": "H",
"Schema": { ... }, "EncryptionConfig": {}, "Status": "ACTIVE"
}
NoteThe order of the key-value pairs in the response is arbitrary.
5. Create a dataset group and add the dataset to it. The value of the domain parameter must match the domain of the dataset.
aws forecast create-dataset-group \
--dataset-group-name electricity_ds_group \
--dataset-arns arn:aws:forecast:us-west-2:acct-id:ds/electricity_demand_ds \ --domain CUSTOM
The following is an example response.
{
"DatasetGroupArn": "arn:aws:forecast:us-west-2:acct-id:dataset-group/
electricity_ds_group"
}
For more information about this operation, see CreateDatasetGroup (p. 202).
6. (Optional) Get the description of the dataset group.
aws forecast describe-dataset-group \
--dataset-group-arn arn:aws:forecast:us-west-2:acct-id:dataset-group/
electricity_ds_group
The following is an example response.
Getting Started (AWS CLI)
{ "DatasetGroupName": "electricity_ds_group",
"DatasetGroupArn": "arn:aws:forecast:us-west-2:acct-id:dataset-group/
electricity_ds_group", "DatasetArns": [
"arn:aws:forecast:us-west-2:acct-id:dataset-group/electricity_ds_group"
],
"Domain": "CUSTOM",
"CreationTime": 1564533719.852,
"LastModificationTime": 1564533719.852, "Status": "ACTIVE"
}
7. Import the electricity usage training data from your Amazon S3 bucket to the dataset. The IAM role that you provide must have permission to read data from your S3 bucket. For information on how to create an IAM role, see Create an IAM for Amazon Forecast (AWS CLI) (p. 8).
aws forecast create-dataset-import-job \
--dataset-arn arn:aws:forecast:us-west-2:acct-id:dataset/electricity_demand_ds \ --dataset-import-job-name electricity_ds_import_job \
--data-source '{
"S3Config": {
"Path": "s3://bucket/electricityusagedata.csv", "RoleArn": "arn:aws:iam::acct-id:role/Role"
} }'
The following is the shorthand syntax for the data-source parameter.
--data-source S3Config="{Path='s3://bucket/
electricityusagedata.csv',RoleArn='arn:aws:iam::acct-id:role/Role'}"
The following is an example response.
{ "DatasetImportJobArn": "arn:aws:forecast:us-west-2:acct-id:dataset-import-job/
electricity_demand_ds/electricity_ds_import_job"
}
For more information about this operation, see CreateDatasetImportJob (p. 205).
8. Check the import status.
aws forecast describe-dataset-import-job \
--dataset-import-job-arn arn:aws:forecast:us-west-2:acct-id:dataset-import-job/
electricity_demand_ds/electricity_ds_import_job
The following is an example response.
{ "DatasetImportJobName": "electricity_ds_import_job",
"DatasetImportJobArn": "arn:aws:forecast:us-west-2:acct-id:dataset-import-job/
electricity_demand_ds/electricity_ds_import_job",
"DatasetArn": "arn:aws:forecast:us-west-2:acct-id:dataset/electricity_demand_ds", "DataSource": {
"S3Config": {
"Path": "s3://bucket/electricityusagedata.csv", "RoleArn": "arn:aws:iam::acct-id:role/ForecastRole"
Getting Started (AWS CLI)
} },
"DataSize": 0.14639010466635227,
"TimeStampFormat": "yyyy-MM-dd HH:mm:ss", "CreationTime": 1564537011.114,
"LastModificationTime": 1564537028.223, "Status": "CREATE_IN_PROGRESS"
}
When all of the data has been imported, the status changes to ACTIVE and the response includes statistics for the data, as shown in the following example.
{ "DatasetArn": "arn:aws:forecast:us-west-2:acct-id:dataset/electricity_demand_ds", "Status": "ACTIVE",
"FieldStatistics": { "date": {
"Min": "2014-01-01T01:00:00Z", "Max": "2015-01-01T00:00:00Z", "Count": 3241200,
"CountDistinct": 8760, "CountNull": 0
},
"target": { "Min": "0.0", "Max": "168200.0", "Avg": 606.5167610461679, "Stddev": 3518.405223972031, "Count": 3241200,
"CountDistinct": 1196961, "CountNull": 0,
"CountNan": 0 },
"item": {
"Count": 3241200, "CountDistinct": 370, "CountNull": 0 }
}, ...
}
Important
You must wait until the status is ACTIVE before creating a predictor with the dataset group.
For more information about this operation, see DescribeDatasetImportJob (p. 267).
Step 2: Create a Predictor
To create a predictor, you use the CreateAutoPredictor (p. 193) operation and provide the following information.
• Predictor name – Give the predictor a name so you can distinguish it from your other predictors
• Dataset group – You created the dataset group in the preceding step.
• Forecast frequency – The granularity of your forecasts (hourly, daily, weekly, etc).
• Forecast horizon – The number of time steps being forecasted.
Getting Started (AWS CLI)
After the predictor is created, you review the accuracy metrics generated by Amazon Forecast. The metrics help you decide whether to use the predictor for generating a forecast. For more information about predictors, see Training Predictors (p. 63).
To create a predictor and review the accuracy metrics 1. Create the predictor.
aws forecast create-auto-predictor \ --predictor-name electricitypredictor \
--input-data-config DatasetGroupArn="arn:aws:forecast:us-west-2:acct-id:dsgroup/
electricity_ds_group" \ --forecast-horizon 36 \ --forecast-frequency D
The following is an example response.
{ "PredictorArn": "arn:aws:forecast:us-west-2:acct-id:predictor/electricitypredictor"
}
2. Get the predictor's status.
aws forecast describe-auto-predictor \
--predictor-arn arn:aws:forecast:us-west-2:acct-id:predictor/electricitypredictor
The following is an example response.
{
"PredictorArn": "arn:aws:forecast:<region>:<acct-num>:predictor/
electricitypredictor",
"PredictorName": "electricitypredictor", "ForecastHorizon": 36,
"ForecastTypes": [ "0.1",
"0.5", "0.9"
],
"ForecastFrequency": "D", "DatasetImportJobArns": [
"arn:aws:forecast:<region>:<acct-num>:dataset-import-job/
getting_started_dataset/gs_import"
],
"DataConfig": {
"DatasetGroupArn": "arn:aws:forecast:<region>:<acct-num>:dataset-group/
getting_started",
"AttributeConfigs": [ {
"AttributeName": "target_value", "Transformations": {
"aggregation": "sum", "backfill": "zero", "frontfill": "none", "middlefill": "zero"
} } ] },
"EstimatedTimeRemainingInMinutes": 97, "Status": "CREATE_IN_PROGRESS",
"CreationTime": "2022-02-23T09:26:24.643000-08:00",
Getting Started (AWS CLI)
"LastModificationTime": "2022-02-23T09:49:26.899000-08:00", "ExplainabilityInfo": {
"Status": "NOT_AVAILABLE"
} }
Important
Model training takes time. Don't proceed until training has completed and the status of the predictor is ACTIVE.
3. Get the accuracy metrics for the predictor.
aws forecast get-accuracy-metrics \
--predictor-arn arn:aws:forecast:us-west-2:acct-id:predictor/electricitypredictor
The following is an example response.
{ "PredictorEvaluationResults": [ {
"TestWindows": [ {
"EvaluationType": "SUMMARY", "Metrics": {
"RMSE": 448.19602551622864, "WeightedQuantileLosses": [ {
"Quantile": 0.9,
"LossValue": 0.11574311406253326 },
{
"Quantile": 0.5,
"LossValue": 0.1706269067283527 },
{
"Quantile": 0.1,
"LossValue": 0.11724164222477837 }
] } }, {
"EvaluationType": "COMPUTED", "Metrics": {
"RMSE": 448.19602551622864, "WeightedQuantileLosses": [ {
"Quantile": 0.9,
"LossValue": 0.11574311406253326 },
{
"Quantile": 0.5,
"LossValue": 0.1706269067283527 },
{
"Quantile": 0.1,
"LossValue": 0.11724164222477837 }
] },
"TestWindowEnd": 1420070400.0, "TestWindowStart": 1420002000.0 }
Getting Started (AWS CLI)
] } ] }
The metrics show the error loss for each quantile. For example, there was an 11.7% error for the first quantile. The metrics also show the root-mean-square error (RMSE).
The summary metrics show the average of the computed metrics over all test windows. Because there was only one test window, the summary and computed metrics are equal.
For more information about this operation, see GetAccuracyMetrics (p. 296).
Step 3: Create a Forecast
Amazon Forecast creates a forecast for the target_value field (as determined by the dataset domain and type) for each unique item_id in the dataset. In this exercise, the target_value field provides electricity usage and the item_id provides client IDs. You get a forecast for the hourly electricity usage by customer.
After the forecast has been created, you can query for a single item or export the complete forecast.
To create, retrieve, and export a forecast 1. Create the forecast.
aws forecast create-forecast \ --forecast-name electricityforecast \
--predictor-arn arn:aws:forecast:us-west-2:acct-id:predictor/electricitypredictor The operation uses the predictor to create a forecast. In the response, you get the Amazon Resource Name (ARN) of the forecast. You use this ARN to retrieve and export the forecast. The following is an example response.
{ "ForecastArn": "arn:aws:forecast:us-west-2:acct-id:forecast/electricityforecast"
}
For more information about this operation, see CreateForecast (p. 218).
2. Retrieve the first two hours of the forecast for client_1.
NoteThe service name, forecastquery, is different then the service name used elsewhere.
aws forecastquery query-forecast \
--forecast-arn arn:aws:forecast:us-west-2:acct-id:forecast/electricityforecast \ --start-date 2015-01-01T00:00:00 \
--end-date 2015-01-01T02:00:00 \ --filters '{"item_id":"client_1"}'
The operation includes the following parameters.
• start-date and end-date – Specifies an optional date range to retrieve the forecast for. If you don't specify these parameters, the operation returns the entire forecast for client_1.
• filters – Specifies the item_id filter to retrieve the electricity forecast for client_1.
The following is the shorthand syntax for the filters parameter.
Getting Started (AWS CLI)
--filters item_id="client_1"
The following is an example response.
{
"Forecast": {
"Predictions": { "mean": [ {
"Timestamp": "2015-01-01T01:00:00", "Value": 20.952411651611328
}, {
"Timestamp": "2015-01-01T02:00:00", "Value": 19.11078453063965
} ], "p90": [ {
"Timestamp": "2015-01-01T01:00:00", "Value": 24.524038314819336
}, {
"Timestamp": "2015-01-01T02:00:00", "Value": 22.319091796875
} ], "p50": [ {
"Timestamp": "2015-01-01T01:00:00", "Value": 20.7841739654541
}, {
"Timestamp": "2015-01-01T02:00:00", "Value": 19.237524032592773
} ], "p10": [ {
"Timestamp": "2015-01-01T01:00:00", "Value": 18.507278442382812
}, {
"Timestamp": "2015-01-01T02:00:00", "Value": 16.15062141418457
} ] } } }
Because this is an hourly forecast, the response shows hourly forecast values. In the response, note the following:
• mean – For the specific date and time, the mean is the predicted mean electricity usage value for the customer.
• p90, p50, and p10 – Specify the confidence level that the actual value will be below the listed value at the specified date and time. For example, at 2015-01-01T01:00:00, Amazon Forecast is 90% confident that the electric usage will be below 24.5. Amazon Forecast is 50% confident that usage will be below 20.8, and 10% confident that usage will be below 18.5.
Getting Started (Python Notebooks)
For more information about this operation, see QueryForecast (p. 334).
3. Export the complete forecast to your Amazon S3 bucket. The IAM role that you provide must have permission to write data to your S3 bucket. For information on how to create an IAM role, see Create an IAM for Amazon Forecast (AWS CLI) (p. 8).
Create a forecast export job.
aws forecast create-forecast-export-job \
--forecast-export-job-name electricityforecast_exportjob \
--forecast-arn arn:aws:forecast:us-west-2:acct-id:forecast/electricityforecast \ --destination S3Config="{Path='s3://bucket',RoleArn='arn:aws:iam::acct-id:role/Role'}"
The following is an example response.
{ "ForecastExportJobArn": "arn:aws:forecast::us-west-2:acct-id:forecast- export/64bbc087"
}
For more information about this operation, see CreateForecastExportJob (p. 222).
4. Get the status of the export job.
aws forecast describe-forecast-export-job \
--forecast-export-job-arn arn:aws:forecast:us-west-2:acct-id:forecast/
electricityforecast
The following is an example response.
{ "ForecastExportJobArn": "arn:aws:forecast::us-west-2:acct-id:forecast- export/64bbc087",
"ForecastExportJobName": "electricityforecast_exportjob", "Status": "CREATE_IN_PROGRESS"
}
When the status is ACTIVE, you can find the forecast files in the specified S3 bucket.
Getting Started (Python Notebooks)
NoteFor a complete list of tutorials using Python notebooks, see the Amazon Forecast Github Samples page.
To get started using Amazon Forecast APIs with Python notebooks, see the Getting Started Tutorial. The tutorial guides you through the core steps of Forecast from start to finish.
For basic tutorials for specific processes, refer to the following Python notebooks:
1.Preparing data - Prepare a dataset, create a dataset group, define the schema, and import the dataset group.
2.Building your predictor - Train a predictor on the data you imported into your Forecast dataset.
3.Evaluating predictors - Obtain predictions, visualize predictions, and compare results.
4.Retraining predictors - Retrain an existing predictor with updated data.
Advanced Tutorials
5.Upgrade to AutoPredictor - Upgrade legacy predictors to AutoPredictor.
6.Clean Up - Delete the dataset groups, predictors, forecasts created during the tutorials.
To repeat the Getting Started tutorial with AutoML, see Getting Started with AutoML.
Advanced Tutorials
For more advanced tutorials, refer to the following Python notebooks:
• Item-level Explainability - Understand how dataset attributes impact forecasts for specific time series and time points.
• Comparing multiple models - Create predictors using Prophet, ETS, and DeepAR+, and compare their performances by visualizating the results.
• Cold start forecasting - Use item metadata and the DeepAR+ algorithm to forecast for cold-start scenarios (when there is little to no historical data).
• Incorporating related time-series datasets - Use related time-series datasets to improve the accuracy of your model.
• Incorporating item metadata - Use item metadata to improve the accuracy of your model.
• Using the Weather Index - Use the Weather Index to incorporate historical and projected weather information when training your predictors.
• Performing What-if analysis - Explore different pricing scenarios and evaluate how it impacts demand.
• Evaluate item-level accuracy - Export backtest metrics and forecasts, and evaluate the item-level performance of your predictor.
Clean Up Resources
To avoid incurring unnecessary charges, delete the resources you created after you're done with the getting started exercise. To delete the resources, use either the Amazon Forecast console or the Delete APIs from the SDKs or the AWS Command Line Interface (AWS CLI). For example, use the DeleteDataset (p. 235) API to delete a dataset.
To delete a resource, its status must be ACTIVE, CREATE_FAILED, or UPDATE_FAILED. Check the status using the Describe APIs, for example, DescribeDataset (p. 260).
Some resources must be deleted before others, as shown in the following table. This process can take some time.
To delete the training data you uploaded, electricityusagedata.csv, see How Do I Delete Objects from an S3 Bucket?.
Resource to Delete Delete This First Notes
ForecastExportJob
Forecast You can't delete a forecast while
it is being exported. After a forecast is deleted, you can no longer query the forecast.
Predictor All associated forecasts.
DatasetImportJob Can not be deleted.
Clean Up Resources
Resource to Delete Delete This First Notes
Dataset All DatasetImportJobs that
target the dataset are also deleted.
You can't delete a Dataset that is used by a predictor.
DatasetSchema All datasets that reference the
schema.
DatasetGroup All associated predictors All associated forecasts.
All datasets in the dataset group.
You can't delete a
DatasetGroup that contains a Dataset used by a predictor.
Automating with AWS CloudFormation
Tutorials
The following tutorial shows you how to perform common tasks in Amazon Forecast and provide ready- made solutions for common use cases. For a complete list of tutorials using Python notebooks, see the Amazon Forecast GitHub Samples page.
Tutorials
• Automating Forecast with CloudFormation (p. 33) - Use an AWS CloudFormation stack to automatically deploy datasets to an S3 bucket and trigger a Forecast pipeline.
Automating with AWS CloudFormation
In this tutorial, you use an AWS CloudFormation automation stack to launch an Amazon Forecast pipeline and generate forecasts using a demonstration dataset.
The AWS Forecast AWS CloudFormation stack:
• Deploys the Improving Forecast Accuracy with Machine Learning Solution AWS CloudFormation template.
• Deploys the NYC Taxi Datasets to the Forecast Data Amazon S3 bucket.
• Automatically starts the demo NYC taxi forecast pipeline in Forecast.
The AWS CloudFormation template is preloaded with target time-series, related time-series, and item metadata demonstration datasets. Relevant fields in the console are pre-filled with their respective S3 locations.
After completing this tutorial using the demonstration datasets, you can use the same automation stack to generate forecasts with your own datasets.
The following diagram shows the components used in this tutorial.
Automating with AWS CloudFormation
Prerequisites
Prerequisites
Before starting the tutorial, make sure you have logged into your AWS account and installed the AWS CloudFormation template:
1. Log in to your AWS account. If you do not already have one, create an AWS account.
2. Install the AWS CloudFormation template. Choose the Region closest to you:
• Tokyo: ap-northeast-1
• Seoul: ap-northeast-2
• Mumbai: ap-south-1
• Singapore: ap-southeast-1
• Sydney: ap-southeast-2
• Frankfurt: eu-cental-1
• Ireland: eu-west-1
• N. Virginia: us-east-1
• Ohio: us-east-2
• Oregon: us-west-2
This deploys a demonstration stack using the NYC Taxi Dataset.
Deploying an AWS CloudFormation Template for Forecast automation
To deploy the CloudFormation template using the NYC Taxi Dataset Step 1: Accept the defaults and choose Next.
Deploying an AWS CloudFormation Template for Forecast automation
Step 2: Provide an email address for notifications and choose Next.
Deploying an AWS CloudFormation Template for Forecast automation
Step 3: Accept defaults and choose Next.
Step 4: For Capabilities, select both check boxes to allow AWS CloudFormation to create AWS Identity and Access Management (IAM) resources and nested stacks. Choose Create stack.
Clean Up
You have deployed an AWS CloudFormation template in Forecast.
Clean Up
After deploying this AWS CloudFormation template, you can clean up newly created resources, deploy the AWS CloudFormation stack using your own datasets, and explore other deployment options.
• Cleaning up: Deleting the demo stack retains the "Improving Forecast Accuracy with Machine
Learning" stack. Deleting the "Improving Forecast Accuracy with Machine Learning" stack retains all S3, Athena, QuickSight, and Forecast data.
• Using your own datasets: To deploy this AWS CloudFormation template with your own time-series data, enter the S3 locations of your datasets in the Datasets Configuration section in Step 2.
• Other deployment options: For more deployment options, see Automated Deployment. If data is already available, you can deploy the stack without the demo data.
Datasets
Importing Datasets
Datasets contain the data used to train a predictor (p. 63). You create one or more Amazon Forecast datasets and import your training data into them. A dataset group is a collection of complementary datasets that detail a set of changing parameters over a series of time. After creating a dataset group, you use it to train a predictor.
Each dataset group can have up to three datasets, one of each dataset (p. 40) type: target time series, related time series, and item metadata.
To create and manage Forecast datasets and dataset groups, you can use the Forecast console, AWS Command Line Interface (AWS CLI), or AWS SDK.
For example Forecast datasets, see the Amazon Forecast Sample GitHub repository.
Topics
• Datasets (p. 39)
• Dataset Groups (p. 42)
• Resolving Conflicts in Data Collection Frequency (p. 42)
• Using Related Time Series Datasets (p. 44)
• Using Item Metadata Datasets (p. 47)
• Predefined Dataset Domains and Dataset Types (p. 49)
• Updating Data (p. 57)
• Handling Missing Values (p. 58)
• Dataset Guidelines for Forecast (p. 61)
Datasets
To create and manage Forecast datasets, you can use the Forecast APIs, including the
CreateDataset (p. 198) and DescribeDataset (p. 260) operations. For a complete list of Forecast APIs, see API Reference (p. 190).
When creating a dataset, you provide information, such as the following:
• The frequency/interval at which you recorded your data. For example, you might aggregate and record retail item sales every week. In the Getting Started (p. 11) exercise, you use the average electricity used per hour.
• The prediction format (the domain) and dataset type (within the domain). A dataset domain specifies which type of forecast you'd like to perform, while a dataset type helps you organize your training data into Forecast-friendly categories.
• The dataset schema. A schema maps the column headers of your dataset. For instance, when monitoring demand, you might have collected hourly data on the sales of an item at multiple stores.
In this case, your schema would define the order, from left to right, in which timestamp, location, and hourly sales appear in your training data file. Schemas also define each column's data type, such as string or integer.
• Geolocation and time zone information. The geolocation attribute is defined within the schema with the attribute type geolocation. Time zone information is defined with the
CreateDatasetImportJob (p. 205) operation. Both geolocation and time zone data must be included to enable the Weather Index. (p. 79)