Kafka Connector Overview¶
Kafka is a distributed publish-subscribe messaging system that maintains feeds of messages in topics. Producers write data to topics and consumers read from topics. For more information on Apache Kafka, see Apache Kafka documentation.
Kafka mainly operates based on a topic model. A topic is a category or feed name to which records get published. Topics in Kafka are always multi-subscriber.
To see the Kafka Connector, navigate to the connector store and search for "Kafka".
Compatibility¶
| Connector Version | Supported product versions |
|---|---|
| 3.1.0 | APIM 4.0.0, EI 7.1.0, EI 7.0.x EI 6.6.0 |
| 3.0.0 | APIM 4.0.0, EI 7.1.0, EI 7.0.x EI 6.6.0 |
| 2.0.9 | APIM 4.0.0, EI 7.1.0, EI 7.0.x EI 6.6.0 EI 6.5.0 |
For older versions, see the details in the connector store.
Kafka Connector documentation¶
The Kafka connector allows you to access the Kafka Producer API from the integration sequence and acts as a message producer that facilitates message publishing. The Kafka connector sends messages to the Kafka brokers.
Follow the topics given below to get started with the Kafka connector.
-
Setting up Kafka: This includes instructions on setting up Kafka and Zookeeper.
-
Enabling Security for Kafka: This includes a variety of security-related details that will be used to secure Kafka.
-
Kafka Connector Example: This example demonstrates how to send messages to a Kafka broker via Kafka topics.
The following topics are specific to connector version 3.1.0 and later version:
Tip
Apache Avro Message type is supported from connector version 3.1.0 onwards.
-
Kafka Connector Avro Message Producer Example: This example demonstrates how to send Apache Avro messages to a Kafka broker via Kafka topics.
-
Kafka Connector Reference: This documentation provides a reference guide for the Kafka Connector.
The following topic is specific to connector version 3.0.0 and earlier versions:
- Kafka Connector Reference: This documentation provides a reference guide for the Kafka Connector.
Kafka Inbound Endpoint documentation¶
The Kafka inbound endpoint acts as a message consumer. It creates a connection to ZooKeeper and requests messages for a topic. The inbound endpoint is bundled with the Kafka connector.
- Kafka Inbound Endpoint Example: This sample demonstrates how one way message bridging from Kafka to HTTP can be done using the inbound Kafka endpoint.
How to contribute¶
As an open source project, ESB extensions welcome contributions from the community.
To contribute to the code for this connector, create a pull request in one of the following repositories.
Check the issue tracker for open issues that interest you. We look forward to receiving your contributions.
Top