Consuming a Kafka topic from Azure Confluent Kafka Cloud using Azure Databricks (Spark)
This post explains how to use Azure Databricks to read/consume a Kafka topic hosted on Azure Confluent Kafka Cloud.
Before you try this example, please have a look in my previous post about how to set up Azure Confluent Kafka Cloud in you own Microsoft Azure subscription.
The next step is to connect Azure Databricks together with Azure Confluent Kafka Cloud.
Push some sample data to your Kafka topic, my topic is called dc01_caiotopic1
Use Azure Databricks to read the topic, see example below:
Use the User Interface to create your Kafka API Key and change the Scala Code with your API key.