The AWS Data Pipeline API contains several data types that various actions use. This section describes each data type in detail.
Note
The order of each element in a data type structure is not guaranteed. Applications should not assume a particular order.
The following data types are supported:
• Field (p. 70)
• InstanceIdentity (p. 71)
• Operator (p. 72)
• ParameterAttribute (p. 74)
• ParameterObject (p. 75)
• ParameterValue (p. 76)
• PipelineDescription (p. 77)
• PipelineIdName (p. 79)
• PipelineObject (p. 80)
• Query (p. 81)
• Selector (p. 82)
• Tag (p. 83)
• TaskObject (p. 84)
• ValidationError (p. 86)
• ValidationWarning (p. 87)
Field
Field
A key-value pair that describes a property of a pipeline object. The value is specified as either a string value (StringValue) or a reference to another object (RefValue) but not as both.
Contents
key
The field identifier.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes refValue
The field value, expressed as the identifier of another object.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: No stringValue
The field value, expressed as a String.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 10240.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
InstanceIdentity
InstanceIdentity
Identity information for the EC2 instance that is hosting the task runner. You can get this value by calling a metadata URI from the EC2 instance. For more information, see Instance Metadata in the Amazon Elastic Compute Cloud User Guide. Passing in this value proves that your task runner is running on an EC2 instance, and ensures the proper AWS Data Pipeline service charges are applied to your pipeline.
Contents
document
A description of an EC2 instance that is generated when the instance is launched and exposed to the instance via the instance metadata service in the form of a JSON representation of an object.
Type: String Required: No signature
A signature which can be used to verify the accuracy and authenticity of the information provided in the instance identity document.
Type: String Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
Operator
Operator
Contains a logical operation for comparing the value of a field with a specified value.
Contents
type
The logical operation to be performed: equal (EQ), equal reference (REF_EQ), less than or equal (LE), greater than or equal (GE), or between (BETWEEN). Equal reference (REF_EQ) can be used only with reference fields. The other comparison types can be used only with String fields. The comparison types you can use apply only to certain object fields, as detailed below.
The comparison operators EQ and REF_EQ act on the following fields:
• name
The comparison operators GE, LE, and BETWEEN act on the following fields:
• @scheduledStartTime
• @scheduledEndTime
• @actualStartTime
• @actualEndTime
Note that fields beginning with the at sign (@) are read-only and set by the web service. When you name fields, you should choose names containing only alpha-numeric values, as symbols may be reserved by AWS Data Pipeline. User-defined fields that you add to a pipeline should prefix their name with the string "my".
Type: String
Valid Values: EQ | REF_EQ | LE | GE | BETWEEN Required: No
values
The value that the actual field value will be compared with.
Type: Array of strings Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
See Also
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
ParameterAttribute
ParameterAttribute
The attributes allowed or specified with a parameter object.
Contents
key
The field identifier.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes stringValue
The field value, expressed as a String.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 10240.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
ParameterObject
ParameterObject
Contains information about a parameter object.
Contents
attributes
The attributes of the parameter object.
Type: Array of ParameterAttribute (p. 74) objects Required: Yes
id
The ID of the parameter object.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
ParameterValue
ParameterValue
A value or list of parameter values.
Contents
id
The ID of the parameter value.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes stringValue
The field value, expressed as a String.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 10240.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
PipelineDescription
PipelineDescription
Contains pipeline metadata.
Contents
description
Description of the pipeline.
Type: String Required: No fields
A list of read-only fields that contain metadata about the pipeline: @userId, @accountId, and
@pipelineState.
Type: Array of Field (p. 70) objects Required: Yes
name
The name of the pipeline.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: Yes pipelineId
The pipeline identifier that was assigned by AWS Data Pipeline. This is a string of the form df-297EG78HU43EEXAMPLE.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: Yes tags
A list of tags to associated with a pipeline. Tags let you control access to pipelines. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide.
Type: Array of Tag (p. 83) objects
Array Members: Minimum number of 0 items. Maximum number of 10 items.
Required: No
See Also
See Also
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
PipelineIdName
PipelineIdName
Contains the name and identifier of a pipeline.
Contents
id
The ID of the pipeline that was assigned by AWS Data Pipeline. This is a string of the form df-297EG78HU43EEXAMPLE.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: No name
The name of the pipeline.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
PipelineObject
PipelineObject
Contains information about a pipeline object. This can be a logical, physical, or physical attempt pipeline object. The complete set of components of a pipeline defines the pipeline.
Contents
fields
Key-value pairs that define the properties of the object.
Type: Array of Field (p. 70) objects Required: Yes
id
The ID of the object.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: Yes name
The name of the object.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
Query
Query
Defines the query to run against an object.
Contents
selectors
List of selectors that define the query. An object must satisfy all of the selectors to match the query.
Type: Array of Selector (p. 82) objects Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
Selector
Selector
A comparison that is used to determine whether a query should return this object.
Contents
fieldName
The name of the field that the operator will be applied to. The field name is the "key" portion of the field definition in the pipeline definition syntax that is used by the AWS Data Pipeline API. If the field is not set on the object, the condition fails.
Type: String Required: No operator
Contains a logical operation for comparing the value of a field with a specified value.
Type: Operator (p. 72) object Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
Tag
Tag
Tags are key/value pairs defined by a user and associated with a pipeline to control access. AWS Data Pipeline allows you to associate ten tags per pipeline. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide.
Contents
key
The key name of a tag defined by a user. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 128.
Required: Yes value
The optional value portion of a tag defined by a user. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 256.
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
TaskObject
TaskObject
Contains information about a pipeline task that is assigned to a task runner.
Contents
attemptId
The ID of the pipeline task attempt object. AWS Data Pipeline uses this value to track how many times a task is attempted.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: No objects
Connection information for the location where the task runner will publish the output of the task.
Type: String to PipelineObject (p. 80) object map
Key Length Constraints: Minimum length of 1. Maximum length of 1024.
Key Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: No pipelineId
The ID of the pipeline that provided the task.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: No taskId
An internal identifier for the task. This ID is passed to the SetTaskStatus (p. 59) and ReportTaskProgress (p. 50) actions.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
See Also
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
ValidationError
ValidationError
Defines a validation error. Validation errors prevent pipeline activation. The set of validation errors that can be returned are defined by AWS Data Pipeline.
Contents
errors
A description of the validation error.
Type: Array of strings
Length Constraints: Minimum length of 0. Maximum length of 10000.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: No id
The identifier of the object that contains the validation error.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
ValidationWarning
ValidationWarning
Defines a validation warning. Validation warnings do not prevent pipeline activation. The set of validation warnings that can be returned are defined by AWS Data Pipeline.
Contents
id
The identifier of the object that contains the validation warning.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 1024.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\n\t]*
Required: No warnings
A description of the validation warning.
Type: Array of strings
Length Constraints: Minimum length of 0. Maximum length of 10000.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3