• Step 5: Use Amazon CloudWatch to View Amazon MSK Metrics (p. 8)
• Step 6: Delete the AWS Resources Created for This Tutorial (p. 8)
Step 1: Create an Amazon MSK Cluster
In this step of Getting Started Using Amazon MSK (p. 5), you create an Amazon MSK cluster.
To create an Amazon MSK cluster using the AWS Management Console
1. Sign in to the AWS Management Console, and open the Amazon MSK console at https://
console.aws.amazon.com/msk/home?region=us-east-1#/home/.
2. Choose Create cluster.
3. For Cluster name, enter MSKTutorialCluster.
4. From the table under All cluster settings, copy the values of the following settings and save them because you need them later in this tutorial:
• VPC
• Subnets
• Security groups associated with VPC 5. Choose Create cluster.
Next Step
Step 2: Create a Client Machine (p. 5)
Step 2: Create a Client Machine
In this step of Getting Started Using Amazon MSK (p. 5), you create a client machine. You use this client machine to create a topic that produces and consumes data. For simplicity, we'll put this client machine in the same VPC as the MSK cluster. But a client machine doesn't have to be in the same VPC as the cluster.
To create a client machine
1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
Step 3: Create a Topic 2. Choose Launch instances.
3. Choose Select to create an instance of Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type.
4. Choose the t2.xlarge instance type by selecting the check box next to it.
5. Choose Next: Configure Instance Details.
6. In the Network list, choose the VPC whose ID you saved in the section called “Step 1: Create a Cluster” (p. 5).
7. In the Auto-assign Public IP list, choose Enable.
8. In the menu near the top, choose Add Tags.
9. Choose Add Tag.
10. Enter Name for the Key and MSKTutorialClient for the Value.
11. Choose Review and Launch, and then choose Launch.
12. Choose Create a new key pair, enter MSKKeyPair for Key pair name, and then choose Download Key Pair. Alternatively, you can use an existing key pair if you prefer.
13. Read the acknowledgement, select the check box next to it, and choose Launch Instances.
14. Choose View Instances. Then, in the Security Groups column, choose the security group that is associated with the MSKTutorialClient instance.
15. Copy the name of the security group, and save it for later.
16. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.
17. In the navigation pane, choose Security Groups. Find the security group whose ID you saved in the section called “Step 1: Create a Cluster” (p. 5). Choose this row by selecting the check box in the first column.
18. In the Inbound Rules tab, choose Edit inbound rules.
19. Choose Add rule.
20. In the new rule, choose All traffic in the Type column. In the second field in the Source column, select the security group of the client machine. This is the group whose name you saved earlier in this step.
21. Choose Save rules. Now the cluster's security group can accept traffic that comes from the client machine's security group.
Next Step
Step 3: Create a Topic (p. 6)
Step 3: Create a Topic
In this step of Getting Started Using Amazon MSK (p. 5), you install Apache Kafka client libraries and tools on the client machine, and then you create a topic.
To create a topic on the client machine
1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
2. In the navigation pane, choose Instances, and then choose MSKTutorialClient by selecting the check box next to it.
3. Choose Actions, and then choose Connect. Follow the instructions to connect to the client machine MSKTutorialClient.
4. Install Java on the client machine by running the following command:
Step 4: Produce and Consume Data 5. Run the following command to download Apache Kafka.
wget https://archive.apache.org/dist/kafka/2.6.2/kafka_2.12-2.6.2.tgz
NoteIf you want to use a mirror site other than the one used in this command, you can choose a different one on the Apache website.
6. Run the following command in the directory where you downloaded the TAR file in the previous step.
tar -xzf kafka_2.12-2.6.2.tgz 7. Go to the kafka_2.12-2.6.2 directory.
8. Open the Amazon MSK console at https://console.aws.amazon.com/msk/.
9. Wait for the status of MSKTutorialCluster to become Active. This might take several minutes. After the status becomes Active, choose the cluster name. This action takes you to a page where you can see the cluster summary.
10. Choose View client information.
11. Copy the private endpoint for plaintext authentication and the Apache ZooKeeper connection string (also for plaintext communication).
12. Run the following command, replacing ZookeeperConnectString with the string that you obtained in the previous instruction.
bin/kafka-topics.sh --create --zookeeper ZookeeperConnectString --replication-factor 3 --partitions 1 --topic MSKTutorialTopic
If the command succeeds, you see the following message: Created topic MSKTutorialTopic.
Next Step
Step 4: Produce and Consume Data (p. 7)
Step 4: Produce and Consume Data
In this step of Getting Started Using Amazon MSK (p. 5), you produce and consume data.
To produce and consume messages
1. Go to the bin folder of the Apache Kafka installation on the client machine, and create a text file named client.properties with the following contents.
security.protocol=PLAINTEXT
2. Run the following command in the bin folder, replacing BootstrapBrokerString with the private endpoint string that you obtained in the section called “Step 3: Create a Topic” (p. 6).
./kafka-console-producer.sh --broker-list BootstrapBrokerString --producer.config client.properties --topic MSKTutorialTopic
3. Enter any message that you want, and press Enter. Repeat this step two or three times. Every time you enter a line and press Enter, that line is sent to your Apache Kafka cluster as a separate message.