The options for monitoring an Amazon MSK cluster. You can specify which Apache Kafka metrics you want Amazon MSK to gather and send to Amazon CloudWatch. You can also configure open monitoring to gather metrics with Prometheus or Prometheus-compatible tools.
URI
/v1/clusters/clusterArn/monitoring
HTTP methods
PUT
Operation ID: UpdateMonitoring
Updates the monitoring settings for the cluster. You can use this operation to specify which Apache Kafka metrics you want Amazon MSK to send to Amazon CloudWatch. You can also specify settings for open monitoring with Prometheus. The following Python 3.6 example enables open monitoring with the Node Exporter. It also sets enhanced monitoring to PER_BROKER. For more information about monitoring, see Monitoring.
import boto3 import time
client = boto3.client('kafka')
update_monitoring_response = client.update_monitoring(
ClusterArn='arn:aws:kafka:us-east-1:0123456789019:cluster/SalesCluster/abcd1234-abcd-cafe-abab-9876543210ab-4',
CurrentVersion='K12V3IB1VIZHHY', EnhancedMonitoring='PER_BROKER',
OpenMonitoring={"Prometheus":{"JmxExporter":{"EnabledInBroker":False},"NodeExporter":
{"EnabledInBroker":True}}}
)
operation_arn = update_monitoring_response['ClusterOperationArn']
print('The ARN of the update operation is ' + operation_arn) describe_cluster_operation_response =
client.describe_cluster_operation(ClusterOperationArn=operation_arn)
operation_state = describe_cluster_operation_response['ClusterOperationInfo']
['OperationState']
print('The status of the update operation is ' + operation_state) updated = False
while not updated:
print('Sleeping for 15 seconds before checking to see if the monitoring update is done...')
time.sleep(15)
describe_cluster_operation_response =
client.describe_cluster_operation(ClusterOperationArn=operation_arn)
operation_state = describe_cluster_operation_response['ClusterOperationInfo']
['OperationState']
if 'UPDATE_COMPLETE' == operation_state:
updated = True
print('You have successfully updated the monitoring settings.')
Path parameters
Name Type Required Description
clusterArn String True The Amazon Resource
Name (ARN) that uniquely identifies the cluster.
Responses
Status code Response model Description
200 UpdateMonitoringRequest (p. 158)Successful response.
400 Error (p. 158) The request isn't valid because
the input is incorrect. Correct your input and then submit it again.
401 Error (p. 158) The request is not authorized.
The provided credentials couldn't be validated.
403 Error (p. 158) Access forbidden. Check your
credentials and then retry your request.
404 Error (p. 158) The resource could not be found
due to incorrect input. Correct the input, then retry the request.
429 Error (p. 158) 429 response
500 Error (p. 158) There was an unexpected
internal server error. Retrying your request might resolve the issue.
503 Error (p. 158) 503 response
OPTIONS
Enable CORS by returning the correct headers.
Path parameters
Name Type Required Description
clusterArn String True The Amazon Resource
Name (ARN) that uniquely identifies the cluster.
Responses
Status code Response model Description
200 None Default response for CORS
method
Schemas
Request bodies
PUT schema
{ "enhancedMonitoring": enum, "loggingInfo": {
"brokerLogs": {
}, "currentVersion": "string"
}
Response bodies
UpdateMonitoringRequest schema
{ "clusterArn": "string",
"clusterOperationArn": "string"
}
Error schema
{ "message": "string",
"invalidParameter": "string"
}
Properties
BrokerLogs
The broker logs configuration for this MSK cluster.
s3
Details of the Amazon S3 destination for broker logs.
Type: S3 (p. 161) Required: False
firehose
Details of the Kinesis Data Firehose delivery stream that is the destination for broker logs.
Type: Firehose (p. 160) Required: False
cloudWatchLogs
Details of the CloudWatch Logs destination for broker logs.
Type: CloudWatchLogs (p. 159) Required: False
CloudWatchLogs
Details of the CloudWatch Logs destination for broker logs.
logGroup
The CloudWatch log group that is the destination for broker logs.
Type: string Required: False
enabled
Specifies whether broker logs get sent to the specified CloudWatch Logs destination.
Type: boolean Required: True
EnhancedMonitoring
Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster. This property has three possible values: DEFAULT, PER_BROKER, and PER_TOPIC_PER_BROKER.
For a list of the metrics associated with each of these three levels of monitoring, see Monitoring.
DEFAULT
PER_BROKER
PER_TOPIC_PER_BROKER PER_TOPIC_PER_PARTITION
Error
Returns information about an error.
message
The description of the error.
Type: string Required: False
invalidParameter
The parameter that caused the error.
Type: string Required: False
Firehose
Firehose details for BrokerLogs.
deliveryStream
The Kinesis Data Firehose delivery stream that is the destination for broker logs.
Type: string Required: False
enabled
Specifies whether broker logs get send to the specified Kinesis Data Firehose delivery stream.
Type: boolean Required: True
JmxExporterInfo
JMX Exporter details.
enabledInBroker
JMX Exporter being enabled in broker.
Type: boolean Required: True
LoggingInfo
You can configure your MSK cluster to send broker logs to different destination types. This is a container for the configuration details related to broker logs.
brokerLogs
You can configure your MSK cluster to send broker logs to different destination types. This configuration specifies the details of these destinations.
Type: BrokerLogs (p. 159) Required: True
NodeExporterInfo
Node Exporter details.
enabledInBroker
Node Exporter being enabled in broker.
Type: boolean Required: True
OpenMonitoringInfo
JMX and Node monitoring for cluster.
prometheus
Prometheus details.Type: PrometheusInfo (p. 161) Required: True
PrometheusInfo
Prometheus details.
nodeExporter
Node Exporter details.Type: NodeExporterInfo (p. 161) Required: False
jmxExporter
JMX Exporter details.Type: JmxExporterInfo (p. 160) Required: False
S3
The details of the Amazon S3 destination for broker logs.
bucket
The name of the S3 bucket that is the destination for broker logs.
Type: string Required: False
prefix
The S3 prefix that is the destination for broker logs.
Type: string Required: False
enabled
Specifies whether broker logs get sent to the specified Amazon S3 destination.
Type: boolean Required: True
UpdateMonitoringRequest
Request body for UpdateMonitoring.
enhancedMonitoring
Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster.
Type: EnhancedMonitoring (p. 159) Required: False
loggingInfo
LoggingInfo details.Type: LoggingInfo (p. 160) Required: False
openMonitoring
The settings for open monitoring.
Type: OpenMonitoringInfo (p. 161) Required: False
currentVersion
The version of the MSK cluster to update. Cluster versions aren't simple numbers. You can describe an MSK cluster to find its version. When this update operation is successful, it generates a new cluster version.
Type: string
Required: True
UpdateMonitoringResponse
Request body for UpdateMonitoring.
clusterArn
The Amazon Resource Name (ARN) of the cluster.
Type: string Required: False
clusterOperationArn
The Amazon Resource Name (ARN) of the cluster operation.
Type: string Required: False
See also
For more information about using this API in one of the language-specific AWS SDKs and references, see the following:
UpdateMonitoring
• AWS Command Line Interface
• AWS SDK for .NET
• AWS SDK for C++
• AWS SDK for Go
• AWS SDK for Java V2
• AWS SDK for JavaScript
• AWS SDK for PHP V3
• AWS SDK for Python
• AWS SDK for Ruby V3