Before ingesting Confluent Kafka metadata into Dawiso, prepare your account for authentication using API credentials (API key and secret) with permissions to read metadata defined in Confluent Cloud.

Obtain your Confluent Kafka Cloud API key(s) and secret(s)

You will need two pairs of API keys and secrets to complete the setup:

  1. The first pair is used to connect your Confluent Kafka Cloud with Dawiso.
  2. The second pair is used to connect to your Confluent Kafka cluster and your schema registry.

Cluster name and service endpoint URL

First, obtain basic information about your Confluent Kafka cluster:

  1. In the Confluent Kafka Cloud console, select the cluster to ingest metadata from.

  2. In left menu, select Cluster Overview and continue to Cluster settings. Here, you will find:

    • The Cluster ID
    • The REST endpoint URL

  3. Copy and store these values to later create a Confluent Kafka connection.

Create an API key for your Cluster

From the left menu, select Cluster Overview and continue to API keys. You can choose between creating a key for your Confluent Kafka Cloud account and a Service account.

Account-bound API keyService account API key
1. Select My account., 2. Click Next and choose a descriptive name for your key, e.g. Dawiso connection API key., 3. Copy the API key and secret values and store them somewhere safe.1. Select Service account., 2. Click Next and choose a descriptive name for your key, e.g. Dawiso service account API key., 3. Create new ACLs with the following permissions to scan only the required metadata from your instance:Cluster: DescribeConfigs, 4. Consumer group: Describe, 5. Topic: Describe, DescribeConfigsCopy the API key and secret values and store them somewhere safe.
Warning

The API key and secret values are displayed only once. Copy and save them immediately, otherwise you will need to generate a new API key and secret.

Create a Schema Registry API key and obtain endpoint

To scan and parse schemas in the Confluent Kafka Schema Registry, Dawiso will also require the schema registry endpoint URL and its API key.

First, find your schema endpoint URL:

  1. Navigate to the Environments tab in Confluent Kafka.
  2. Select the environment for which you want to obtain the API key and secret.
  3. On the right side of the overview page of your environment, look for Stream Governance API.
  4. Copy and save the Endpoint URL.

Finally, create a schema registry API key-secret pair:

  1. At the bottom of the screen where you obtained the endpoint URL, click the link in the Credentials tab.
  2. Create a new Schema Registry API key and choose a descriptive name for your key, e.g. Dawiso Schema Registry API key.
  3. Copy the API key and secret values and store them somewhere safe. Note that the secret can be displayed only once.