Skip to main content
Kafka Connect is a data integration framework provided by the Apache Kafka community for streaming data between Apache Kafka and other data systems. AutoMQ BYOC provides a fully managed Kafka Connect service, so you can run these integrations without deploying and operating the Connect framework yourself. AutoMQ is compatible with the Apache Kafka protocol and Kafka Connect’s standard worker, plugin, and Connector model. Existing Kafka clients, Connector implementations, and operating practices can generally be reused without application or Connector rewrites. Check each plugin’s prerequisites and compatibility notes before deployment because external systems and plugin dependencies still apply. You create Connect Cluster and Connector resources in the AutoMQ Console or with Terraform. AutoMQ then provisions the Connect Workers and Connector tasks for you. The workers run as replaceable Pods in an EKS cluster inside your specified VPC, keeping integration traffic in your network while allowing worker compute capacity to be managed independently from Kafka data storage. The underlying AutoMQ data plane uses a cloud-native Shared Storage architecture with stateless brokers. Durable Kafka data remains in shared storage while broker compute can be replaced or scaled independently; Connect Workers follow the same cloud-native operating model as replaceable Kubernetes Pods.
  • Network placement: AutoMQ Connect runs on an EKS (Amazon Elastic Kubernetes Service) cluster inside your VPC. A Connect Worker Pod can access other resources in the same VPC, such as RDS and Redis, without an additional public network path.
  • Cloud-native worker architecture: Worker compute is replaceable and independently scalable. Configure worker count, worker tier, and Kubernetes placement on the Connect Cluster; plugin-specific resource or isolation requirements should be validated before production use.
  1. Manage plugins and choose the Managed or custom plugin versions for your Connect Cluster.
  2. Manage Connectors by creating a Connect Cluster and then a Connector.
  3. If you are deploying the workers on AWS, prepare the Kubernetes resources with the AutoMQ on EKS deployment guide.