Kafka Connector Overview

Kafka is a distributed publish-subscribe messaging system that maintains feeds of messages in topics. Producers write data to topics and consumers read from topics. For more information on Apache Kafka, see Apache Kafka documentation.

Kafka mainly operates based on a topic model. A topic is a category or feed name to which records get published. Topics in Kafka are always multi-subscriber.

To see the Kafka Connector, navigate to the connector store and search for "Kafka".

Kafka Connector Store

Compatibility

Connector Version Supported product versions
3.1.0 APIM 4.0.0, EI 7.1.0, EI 7.0.x EI 6.6.0
3.0.0 APIM 4.0.0, EI 7.1.0, EI 7.0.x EI 6.6.0
2.0.9 APIM 4.0.0, EI 7.1.0, EI 7.0.x EI 6.6.0 EI 6.5.0

For older versions, see the details in the connector store.

Kafka Connector documentation

The Kafka connector allows you to access the Kafka Producer API from the integration sequence and acts as a message producer that facilitates message publishing. The Kafka connector sends messages to the Kafka brokers.

Follow the topics given below to get started with the Kafka connector.

The following topics are specific to connector version 3.1.0 and later version:

Tip

Apache Avro Message type is supported from connector version 3.1.0 onwards.

The following topic is specific to connector version 3.0.0 and earlier versions:

Kafka Inbound Endpoint documentation

The Kafka inbound endpoint acts as a message consumer. It creates a connection to ZooKeeper and requests messages for a topic. The inbound endpoint is bundled with the Kafka connector.

  • Kafka Inbound Endpoint Example: This sample demonstrates how one way message bridging from Kafka to HTTP can be done using the inbound Kafka endpoint.

How to contribute

As an open source project, ESB extensions welcome contributions from the community.

To contribute to the code for this connector, create a pull request in one of the following repositories.

Check the issue tracker for open issues that interest you. We look forward to receiving your contributions.

Top