The Amazon Kinesis API contains several data types that various actions use. This section describes each data type in detail.
Note
The order of each element in a data type structure is not guaranteed. Applications should not assume a particular order.
The following data types are supported:
• ChildShard (p. 106)
• Consumer (p. 107)
• ConsumerDescription (p. 109)
• EnhancedMetrics (p. 111)
• HashKeyRange (p. 112)
• PutRecordsRequestEntry (p. 113)
• PutRecordsResultEntry (p. 114)
• Record (p. 115)
• SequenceNumberRange (p. 117)
• Shard (p. 118)
• ShardFilter (p. 120)
• StartingPosition (p. 122)
• StreamDescription (p. 124)
• StreamDescriptionSummary (p. 127)
• StreamModeDetails (p. 130)
• SubscribeToShardEvent (p. 131)
• SubscribeToShardEventStream (p. 132)
• Tag (p. 134)
ChildShard
ChildShard
Output parameter of the GetRecords API. The existing child shard of the current shard.
Contents
HashKeyRange
The range of possible hash key values for the shard, which is a set of ordered contiguous positive integers.
Type: HashKeyRange (p. 112) object Required: Yes
ParentShards
The current shard that is the parent of the existing child shard.
Type: Array of strings
Pattern: shardId-[0-9]{12}
Required: Yes ShardId
The shard ID of the existing child shard of the current shard.
Type: String
Pattern: shardId-[0-9]{12}
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
Consumer
Consumer
An object that represents the details of the consumer you registered. This type of object is returned by RegisterStreamConsumer (p. 79).
Contents
ConsumerARN
When you register a consumer, Kinesis Data Streams generates an ARN for it. You need this ARN to be able to call SubscribeToShard (p. 95).
If you delete a consumer and then create a new one with the same name, it won't have the same ARN. That's because consumer ARNs contain the creation timestamp. This is important to keep in mind if you have IAM policies that reference consumer ARNs.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: ^(arn):aws.*:kinesis:.*:\d{12}:.*stream\/[a-zA-Z0-9_.-]+\/consumer\/
[a-zA-Z0-9_.-]+:[0-9]+
Required: Yes
ConsumerCreationTimestamp
Type: Timestamp Required: Yes ConsumerName
The name of the consumer is something you choose when you register the consumer.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 128.
Pattern: [a-zA-Z0-9_.-]+
Required: Yes ConsumerStatus
A consumer can't read data while in the CREATING or DELETING states.
Type: String
Valid Values: CREATING | DELETING | ACTIVE Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
See Also
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
ConsumerDescription
ConsumerDescription
An object that represents the details of a registered consumer. This type of object is returned by DescribeStreamConsumer (p. 24).
Contents
ConsumerARN
When you register a consumer, Kinesis Data Streams generates an ARN for it. You need this ARN to be able to call SubscribeToShard (p. 95).
If you delete a consumer and then create a new one with the same name, it won't have the same ARN. That's because consumer ARNs contain the creation timestamp. This is important to keep in mind if you have IAM policies that reference consumer ARNs.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: ^(arn):aws.*:kinesis:.*:\d{12}:.*stream\/[a-zA-Z0-9_.-]+\/consumer\/
[a-zA-Z0-9_.-]+:[0-9]+
Required: Yes
ConsumerCreationTimestamp
Type: Timestamp Required: Yes ConsumerName
The name of the consumer is something you choose when you register the consumer.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 128.
Pattern: [a-zA-Z0-9_.-]+
Required: Yes ConsumerStatus
A consumer can't read data while in the CREATING or DELETING states.
Type: String
Valid Values: CREATING | DELETING | ACTIVE Required: Yes
StreamARN
The ARN of the stream with which you registered the consumer.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: arn:aws.*:kinesis:.*:\d{12}:stream/.+
See Also
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
EnhancedMetrics
EnhancedMetrics
Represents enhanced metrics types.
Contents
ShardLevelMetrics
List of shard-level metrics.
The following are the valid shard-level metrics. The value "ALL" enhances every metric.
• IncomingBytes
• IncomingRecords
• OutgoingBytes
• OutgoingRecords
• WriteProvisionedThroughputExceeded
• ReadProvisionedThroughputExceeded
• IteratorAgeMilliseconds
• ALL
For more information, see Monitoring the Amazon Kinesis Data Streams Service with Amazon CloudWatch in the Amazon Kinesis Data Streams Developer Guide.
Type: Array of strings
Array Members: Minimum number of 1 item. Maximum number of 7 items.
Valid Values: IncomingBytes | IncomingRecords | OutgoingBytes | OutgoingRecords
| WriteProvisionedThroughputExceeded | ReadProvisionedThroughputExceeded | IteratorAgeMilliseconds | ALL
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
HashKeyRange
HashKeyRange
The range of possible hash key values for the shard, which is a set of ordered contiguous positive integers.
Contents
EndingHashKey
The ending hash key of the hash key range.
Type: String
Pattern: 0|([1-9]\d{0,38}) Required: Yes
StartingHashKey
The starting hash key of the hash key range.
Type: String
Pattern: 0|([1-9]\d{0,38}) Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
PutRecordsRequestEntry
PutRecordsRequestEntry
Represents the output for PutRecords.
Contents
Data
The data blob to put into the record, which is base64-encoded when the blob is serialized. When the data blob (the payload before base64-encoding) is added to the partition key size, the total size must not exceed the maximum record size (1 MiB).
Type: Base64-encoded binary data object
Length Constraints: Minimum length of 0. Maximum length of 1048576.
Required: Yes ExplicitHashKey
The hash value used to determine explicitly the shard that the data record is assigned to by overriding the partition key hash.
Type: String
Pattern: 0|([1-9]\d{0,38}) Required: No
PartitionKey
Determines which shard in the stream the data record is assigned to. Partition keys are Unicode strings with a maximum length limit of 256 characters for each key. Amazon Kinesis Data Streams uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
PutRecordsResultEntry
PutRecordsResultEntry
Represents the result of an individual record from a PutRecords request. A record that is successfully added to a stream includes SequenceNumber and ShardId in the result. A record that fails to be added to the stream includes ErrorCode and ErrorMessage in the result.
Contents
ErrorCode
The error code for an individual record result. ErrorCodes can be either ProvisionedThroughputExceededException or InternalFailure.
Type: String Required: No ErrorMessage
The error message for an individual record result. An ErrorCode value of
ProvisionedThroughputExceededException has an error message that includes the account ID, stream name, and shard ID. An ErrorCode value of InternalFailure has the error message
"Internal Service Failure".
Type: String Required: No SequenceNumber
The sequence number for an individual record result.
Type: String
Pattern: 0|([1-9]\d{0,128}) Required: No
ShardId
The shard ID for an individual record result.
Type: String
Pattern: shardId-[0-9]{12}
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
Record
Record
The unit of data of the Kinesis data stream, which is composed of a sequence number, a partition key, and a data blob.
Contents
ApproximateArrivalTimestamp
The approximate time that the record was inserted into the stream.
Type: Timestamp Required: No Data
The data blob. The data in the blob is both opaque and immutable to Kinesis Data Streams, which does not inspect, interpret, or change the data in the blob in any way. When the data blob (the payload before base64-encoding) is added to the partition key size, the total size must not exceed the maximum record size (1 MiB).
Type: Base64-encoded binary data object
Length Constraints: Minimum length of 0. Maximum length of 1048576.
Required: Yes EncryptionType
The encryption type used on the record. This parameter can be one of the following values:
• NONE: Do not encrypt the records in the stream.
• KMS: Use server-side encryption on the records in the stream using a customer-managed AWS KMS key.
Type: String
Valid Values: NONE | KMS Required: No
PartitionKey
Identifies which shard in the stream the data record is assigned to.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Required: Yes SequenceNumber
The unique identifier of the record within its shard.
Type: String
Pattern: 0|([1-9]\d{0,128}) Required: Yes
See Also
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
SequenceNumberRange
SequenceNumberRange
The range of possible sequence numbers for the shard.
Contents
EndingSequenceNumber
The ending sequence number for the range. Shards that are in the OPEN state have an ending sequence number of null.
Type: String
Pattern: 0|([1-9]\d{0,128}) Required: No
StartingSequenceNumber
The starting sequence number for the range.
Type: String
Pattern: 0|([1-9]\d{0,128}) Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
Shard
Shard
A uniquely identified group of data records in a Kinesis data stream.
Contents
AdjacentParentShardId
The shard ID of the shard adjacent to the shard's parent.
Type: String
Pattern: shardId-[0-9]{12}
Required: No HashKeyRange
The range of possible hash key values for the shard, which is a set of ordered contiguous positive integers.
Type: HashKeyRange (p. 112) object Required: Yes
ParentShardId
The shard ID of the shard's parent.
Type: String
Pattern: shardId-[0-9]{12}
Required: No SequenceNumberRange
The range of possible sequence numbers for the shard.
Type: SequenceNumberRange (p. 117) object Required: Yes
ShardId
The unique identifier of the shard within the stream.
Type: String
Pattern: shardId-[0-9]{12}
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
See Also
• AWS SDK for Ruby V3
ShardFilter
ShardFilter
The request parameter used to filter out the response of the ListShards API.
Contents
ShardId
The exclusive start shardID speified in the ShardFilter parameter. This property can only be used if the AFTER_SHARD_ID shard type is specified.
Type: String
Pattern: shardId-[0-9]{12}
Required: No Timestamp
The timestamps specified in the ShardFilter parameter. A timestamp is a Unix epoch date with precision in milliseconds. For example, 2016-04-04T19:58:46.480-00:00 or 1459799926.480. This property can only be used if FROM_TIMESTAMP or AT_TIMESTAMP shard types are specified.
Type: Timestamp Required: No Type
The shard type specified in the ShardFilter parameter. This is a required property of the ShardFilter parameter.
You can specify the following valid values:
• AFTER_SHARD_ID - the response includes all the shards, starting with the shard whose ID immediately follows the ShardId that you provided.
• AT_TRIM_HORIZON - the response includes all the shards that were open at TRIM_HORIZON.
• FROM_TRIM_HORIZON - (default), the response includes all the shards within the retention period of the data stream (trim to tip).
• AT_LATEST - the response includes only the currently open shards of the data stream.
• AT_TIMESTAMP - the response includes all shards whose start timestamp is less than or equal to the given timestamp and end timestamp is greater than or equal to the given timestamp or still open.
• FROM_TIMESTAMP - the response incldues all closed shards whose end timestamp is greater than or equal to the given timestamp and also all open shards. Corrected to TRIM_HORIZON of the data stream if FROM_TIMESTAMP is less than the TRIM_HORIZON value.
Type: String
Valid Values: AFTER_SHARD_ID | AT_TRIM_HORIZON | FROM_TRIM_HORIZON | AT_LATEST
| AT_TIMESTAMP | FROM_TIMESTAMP Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
See Also
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
StartingPosition
StartingPosition
The starting position in the data stream from which to start streaming.
Contents
SequenceNumber
The sequence number of the data record in the shard from which to start streaming.
To specify a sequence number, set StartingPosition to AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER.
Type: String
Pattern: 0|([1-9]\d{0,128}) Required: No
Timestamp
The time stamp of the data record from which to start reading. To specify a time stamp, set StartingPosition to Type AT_TIMESTAMP. A time stamp is the Unix epoch date with precision in milliseconds. For example, 2016-04-04T19:58:46.480-00:00 or 1459799926.480. If a record with this exact time stamp does not exist, records will be streamed from the next (later) record.
If the time stamp is older than the current trim horizon, records will be streamed from the oldest untrimmed data record (TRIM_HORIZON).
Type: Timestamp Required: No Type
You can set the starting position to one of the following values:
AT_SEQUENCE_NUMBER: Start streaming from the position denoted by the sequence number specified in the SequenceNumber field.
AFTER_SEQUENCE_NUMBER: Start streaming right after the position denoted by the sequence number specified in the SequenceNumber field.
AT_TIMESTAMP: Start streaming from the position denoted by the time stamp specified in the Timestamp field.
TRIM_HORIZON: Start streaming at the last untrimmed record in the shard, which is the oldest data record in the shard.
LATEST: Start streaming just after the most recent record in the shard, so that you always read the most recent data in the shard.
Type: String
Valid Values: AT_SEQUENCE_NUMBER | AFTER_SEQUENCE_NUMBER | TRIM_HORIZON | LATEST | AT_TIMESTAMP
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
See Also
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
StreamDescription
StreamDescription
Represents the output for DescribeStream (p. 19).
Contents
EncryptionType
The server-side encryption type used on the stream. This parameter can be one of the following values:
• NONE: Do not encrypt the records in the stream.
• KMS: Use server-side encryption on the records in the stream using a customer-managed AWS KMS key.
Type: String
Valid Values: NONE | KMS Required: No
EnhancedMonitoring
Represents the current enhanced monitoring settings of the stream.
Type: Array of EnhancedMetrics (p. 111) objects Required: Yes
HasMoreShards
If set to true, more shards in the stream are available to describe.
Type: Boolean Required: Yes KeyId
The GUID for the customer-managed AWS KMS key to use for encryption. This value can be a globally unique identifier, a fully specified ARN to either an alias or a key, or an alias name prefixed by "alias/".You can also use a master key owned by Kinesis Data Streams by specifying the alias aws/
kinesis.
• Key ARN example:
arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012
• Alias ARN example: arn:aws:kms:us-east-1:123456789012:alias/MyAliasName
• Globally unique key ID example: 12345678-1234-1234-1234-123456789012
• Alias name example: alias/MyAliasName
• Master key owned by Kinesis Data Streams: alias/aws/kinesis Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Required: No RetentionPeriodHours
The current retention period, in hours. Minimum value of 24. Maximum value of 168.
Type: Integer
Contents
Required: Yes Shards
The shards that comprise the stream.
Type: Array of Shard (p. 118) objects Required: Yes
StreamARN
The Amazon Resource Name (ARN) for the stream being described.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: arn:aws.*:kinesis:.*:\d{12}:stream/.+
Required: Yes
StreamCreationTimestamp
The approximate time that the stream was created.
Type: Timestamp Required: Yes StreamModeDetails
Specifies the capacity mode to which you want to set your data stream. Currently, in Kinesis Data Streams, you can choose between an on-demand capacity mode and a provisioned capacity mode for your data streams.
Type: StreamModeDetails (p. 130) object Required: No
StreamName
The name of the stream being described.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 128.
Pattern: [a-zA-Z0-9_.-]+
Required: Yes StreamStatus
The current status of the stream being described. The stream status is one of the following states:
• CREATING - The stream is being created. Kinesis Data Streams immediately returns and sets StreamStatus to CREATING.
• DELETING - The stream is being deleted. The specified stream is in the DELETING state until Kinesis Data Streams completes the deletion.
• ACTIVE - The stream exists and is ready for read and write operations or deletion. You should perform read and write operations only on an ACTIVE stream.
• UPDATING - Shards in the stream are being merged or split. Read and write operations continue to work while the stream is in the UPDATING state.
Type: String
See Also
Valid Values: CREATING | DELETING | ACTIVE | UPDATING Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for Ruby V3
StreamDescriptionSummary
StreamDescriptionSummary
Represents the output for DescribeStreamSummary (p. 27)
Contents
ConsumerCount
The number of enhanced fan-out consumers registered with the stream.
Type: Integer
Valid Range: Minimum value of 0. Maximum value of 1000000.
Required: No EncryptionType
The encryption type used. This value is one of the following:
• KMS
• NONE Type: String
Valid Values: NONE | KMS Required: No
EnhancedMonitoring
Represents the current enhanced monitoring settings of the stream.
Type: Array of EnhancedMetrics (p. 111) objects Required: Yes
KeyId
The GUID for the customer-managed AWS KMS key to use for encryption. This value can be a globally unique identifier, a fully specified ARN to either an alias or a key, or an alias name prefixed by "alias/".You can also use a master key owned by Kinesis Data Streams by specifying the alias aws/
kinesis.
• Key ARN example:
arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012
• Alias ARN example: arn:aws:kms:us-east-1:123456789012:alias/MyAliasName
• Globally unique key ID example: 12345678-1234-1234-1234-123456789012
• Alias name example: alias/MyAliasName
• Master key owned by Kinesis Data Streams: alias/aws/kinesis Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Required: No OpenShardCount
The number of open shards in the stream.
Type: Integer
Contents
Valid Range: Minimum value of 0. Maximum value of 1000000.
Required: Yes RetentionPeriodHours
The current retention period, in hours.
Type: Integer Required: Yes StreamARN
The Amazon Resource Name (ARN) for the stream being described.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: arn:aws.*:kinesis:.*:\d{12}:stream/.+
Required: Yes
StreamCreationTimestamp
The approximate time that the stream was created.
Type: Timestamp Required: Yes StreamModeDetails
Specifies the capacity mode to which you want to set your data stream. Currently, in Kinesis Data
Specifies the capacity mode to which you want to set your data stream. Currently, in Kinesis Data