type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-target,
Signature=4c5e7eb762ea45f292a5cd1a1cc56ed60009e19a9dba3d6e5e4e67e96d43af11 Accept: */*
{
"JobFlowId": "j-3U7TSX5GZFD8Y", "InstanceGroups": [{
"Name": "Task Instance Group", "InstanceRole": "TASK", "InstanceCount": 2,
"InstanceType": "m1.small", "Market": "ON_DEMAND"
}]
}
Sample Response
HTTP/1.1 200 OK
x-amzn-RequestId: 9da5a349-ed9e-11e2-90db-69a5154aeb8d Content-Type: application/x-amz-json-1.1
Content-Length: 71
Date: Mon, 15 Jul 2013 22:33:47 GMT
{ "InstanceGroupIds": ["ig-294A6A2KWT4WB"], "JobFlowId": "j-3U7TSX5GZFD8Y"
}
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS Command Line Interface
• AWS SDK for .NET
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for JavaScript
• AWS SDK for PHP V3
• AWS SDK for Python
• AWS SDK for Ruby V3
AddJobFlowSteps
AddJobFlowSteps
AddJobFlowSteps adds new steps to a running cluster. A maximum of 256 steps are allowed in each job flow.
If your cluster is long-running (such as a Hive data warehouse) or complex, you may require more than 256 steps to process your data. You can bypass the 256-step limitation in various ways, including using SSH to connect to the master node and submitting queries directly to the software running on the master node, such as Hive and Hadoop. For more information on how to do this, see Add More than 256 Steps to a Cluster in the Amazon EMR Management Guide.
A step specifies the location of a JAR file stored either on the master node of the cluster or in Amazon S3. Each step is performed by the main function of the main class of the JAR file. The main class can be specified either in the manifest of the JAR or by using the MainFunction parameter of the step.
Amazon EMR executes each step in the order listed. For a step to be considered complete, the main function must exit with a zero exit code and all Hadoop jobs started while the step was running must have completed and run successfully.
You can only add steps to a cluster that is in one of the following states: STARTING, BOOTSTRAPPING, RUNNING, or WAITING.
NoteThe string values passed into HadoopJarStep object cannot exceed a total of 10240 characters.
Request Syntax
{ "JobFlowId": "string", "Steps": [
For information about the parameters that are common to all actions, see Common Parameters (p. 296).
The request accepts the following data in JSON format.
JobFlowId (p. 11)
A string that uniquely identifies the job flow. This identifier is returned by RunJobFlow (p. 122) and can also be obtained from ListClusters (p. 69).
Response Syntax
Type: String
Length Constraints: Minimum length of 0. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes Steps (p. 11)
A list of StepConfig (p. 278) to be executed by the job flow.
Type: Array of StepConfig (p. 278) objects Required: Yes
Response Syntax
{ "StepIds": [ "string" ] }
Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
StepIds (p. 12)
The identifiers of the list of steps added to the job flow.
Type: Array of strings
Length Constraints: Minimum length of 0. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Errors
For information about the errors that are common to all actions, see Common Errors (p. 298).
InternalServerError
Indicates that an error occurred while processing the request and that the request was not completed.
HTTP Status Code: 400
Examples
Example
This example illustrates one usage of AddJobFlowSteps.
See Also
"JobFlowId": "j-3TS0OIYO4NFN", "Steps": [{
"Name": "Example Jar Step",
"ActionOnFailure": "CANCEL_AND_WAIT", "HadoopJarStep": {
"Jar": "s3n:\\/\\/elasticmapreduce\\/samples\\/cloudburst\\/cloudburst.jar", "Args": [
"s3n:\\/\\/elasticmapreduce\\/samples\\/cloudburst\\/input\\/s_suis.br", "s3n:\\/\\/elasticmapreduce\\/samples\\/cloudburst\\/input\\/100k.br", "s3n:\\/\\/examples-bucket\\/cloudburst\\/output",
Date: Tue, 16 Jul 2013 21:05:07 GMT {
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS Command Line Interface
See Also
• AWS SDK for .NET
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for JavaScript
• AWS SDK for PHP V3
• AWS SDK for Python
• AWS SDK for Ruby V3
AddTags
AddTags
Adds tags to an Amazon EMR resource, such as a cluster or an Amazon EMR Studio. Tags make it easier to associate resources in various ways, such as grouping clusters to track your Amazon EMR resource allocation costs. For more information, see Tag Clusters.
Request Syntax
{ "ResourceId": "string", "Tags": [
{
"Key": "string", "Value": "string"
} ]}
Request Parameters
For information about the parameters that are common to all actions, see Common Parameters (p. 296).
The request accepts the following data in JSON format.
ResourceId (p. 15)
The Amazon EMR resource identifier to which tags will be added. For example, a cluster identifier or an Amazon EMR Studio ID.
Type: String Required: Yes Tags (p. 15)
A list of tags to associate with a resource. Tags are user-defined key-value pairs that consist of a required key string with a maximum of 128 characters, and an optional value string with a maximum of 256 characters.
Type: Array of Tag (p. 294) objects Required: Yes
Response Elements
If the action is successful, the service sends back an HTTP 200 response with an empty HTTP body.
Errors
For information about the errors that are common to all actions, see Common Errors (p. 298).
InternalServerException
This exception occurs when there is an internal failure in the Amazon EMR service.
Examples
HTTP Status Code: 500 InvalidRequestException
This exception occurs when there is something wrong with user input.
HTTP Status Code: 400
Examples
Example
This example illustrates one usage of AddTags.
Sample Request
See Also
x-amzn-RequestId: 9da5a349-ed9e-11e2-90db-69a5154aeb8d
Content-Type: application/x-amz-json-1.1
Content-Length: 71
Date: Mon, 15 Jul 2013 22:33:47 GMT
{
}
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS Command Line Interface
• AWS SDK for .NET
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for JavaScript
• AWS SDK for PHP V3
• AWS SDK for Python
• AWS SDK for Ruby V3
CancelSteps
CancelSteps
Cancels a pending step or steps in a running cluster. Available only in Amazon EMR versions 4.8.0 and later, excluding version 5.0.0. A maximum of 256 steps are allowed in each CancelSteps request.
CancelSteps is idempotent but asynchronous; it does not guarantee that a step will be canceled, even if the request is successfully submitted. When you use Amazon EMR versions 5.28.0 and later, you can cancel steps that are in a PENDING or RUNNING state. In earlier versions of Amazon EMR, you can only cancel steps that are in a PENDING state.
Request Syntax
{ "ClusterId": "string",
"StepCancellationOption": "string", "StepIds": [ "string" ]
}
Request Parameters
For information about the parameters that are common to all actions, see Common Parameters (p. 296).
The request accepts the following data in JSON format.
ClusterId (p. 18)
The ClusterID for the specified steps that will be canceled. Use RunJobFlow (p. 122) and ListClusters (p. 69) to get ClusterIDs.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes
StepCancellationOption (p. 18)
The option to choose to cancel RUNNING steps. By default, the value is SEND_INTERRUPT.
Type: String
Valid Values: SEND_INTERRUPT | TERMINATE_PROCESS Required: No
StepIds (p. 18)
The list of StepIDs to cancel. Use ListSteps (p. 89) to get steps and their states for the specified cluster.
Type: Array of strings
Length Constraints: Minimum length of 0. Maximum length of 256.
Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*
Required: Yes
Response Syntax
Response Syntax
{
"CancelStepsInfoList": [ {
"Reason": "string", "Status": "string", "StepId": "string"
} ] }
Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
CancelStepsInfoList (p. 19)
A list of CancelStepsInfo (p. 167), which shows the status of specified cancel requests for each StepID specified.
Type: Array of CancelStepsInfo (p. 167) objects
Errors
For information about the errors that are common to all actions, see Common Errors (p. 298).
InternalServerError
Indicates that an error occurred while processing the request and that the request was not completed.
HTTP Status Code: 400 InvalidRequestException
This exception occurs when there is something wrong with user input.
HTTP Status Code: 400
Examples
Example
This example illustrates one usage of CancelSteps.
Sample Request
POST / HTTP/1.1
Content-Type: application/x-amz-json-1.1 X-Amz-Target: ElasticMapReduce.CancelSteps
User-Agent: aws-sdk-ruby/1.9.2 ruby/1.9.3 i386-mingw32 Host: us-east-1.elasticmapreduce.amazonaws.com
See Also
[ 11B5G7VIKHCZQ", 23PUT0NR3XF6O", 2NUYMUZ3ADACC", 10O5XO5JUY9OE", "s-CS88G2XK4N7X", "s-2M366D3KU4OTZ"
Date: Tue, 19 Jul 2016 15:31:01 GMT { "CancelStepsInfoList":
[ {"Reason": "This step cannot be cancelled.", "Status": "FAILED",
"StepId": "s-11B5G7VIKHCZQ"},
{"Reason": "Cannot cancel the step. It is already COMPLETED.", "Status": "FAILED",
"StepId": "s-23PUT0NR3XF6O"},
{"Reason": "Cannot cancel the step. It is already CANCELLED.", "Status": "FAILED",
"StepId": "s-2NUYMUZ3ADACC"},
{"Reason": "Cannot cancel the step. It is already RUNNING.", "Status": "FAILED",
"StepId": "s-10O5XO5JUY9OE"},
{"Reason": "Cannot cancel the step. It is already FAILED.", "Status": "FAILED",
"StepId": "s-CS88G2XK4N7X"}, {"Reason": "",
"Status": "SUBMITTED", "StepId": "s-2M366D3KU4OTZ"}
]
}
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
• AWS Command Line Interface
• AWS SDK for .NET
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for JavaScript
See Also
• AWS SDK for PHP V3
• AWS SDK for Python
• AWS SDK for Ruby V3