Skip to main content
Table Topic is a core feature of AutoMQ, designed for modern data lake architectures. Its primary purpose is to create direct integration between streaming data and static data lakes through architectural innovation, addressing traditional issues like stream-batch separation, complex ETL processes, and high costs. This document is aimed at new users experiencing AutoMQ for the first time, focusing on how to deploy an AutoMQ cluster with Table Topic support in a local development environment (by deploying MinIO to provide object storage services), enabling users to quickly experience the features of Table Topic.
This document is intended solely for testing and learning scenarios. For production environments, please use the Table Topic feature and refer to deployment documentation such as Deploy Multi-Nodes Cluster on Linux▸.

​
Prerequisites

  • Linux/Mac/Windows Subsystem for Linux
  • Docker
  • Docker Compose version > 2.22.0
  • At least 8GB of free memory

​
Install and Start AutoMQ Cluster

Run the following command to download the Docker Compose configuration file for a single-node deployment and start the AutoMQ cluster using Docker Compose.
Once started successfully, we will send Table Topic messages and perform queries using Spark SQL to experience automatic table creation, table querying and updating, schema changes, data partitioning, and CDC capabilities. Users can open http://localhost:8888 in their browser and interactively execute commands using Jupyter Notebook.

​
Create Table

The first three steps in the Notebook involve the creation of a Table Topic, writing a single message, and triggering the creation of an Iceberg table.

​
Query Data

Note:There may be a delay between writing the message and the data being entered into the table. If an error occurs during the query, we recommend retrying after a short wait.

​
Update Records

When querying updated results, the data ops will change from I to U.

​
Delete Data

After querying the deletion results, records are empty.

​
Delete Table

​
Stop and Uninstall the AutoMQ Cluster

Once you have completed the functional tests, run the following commands to stop and uninstall the current AutoMQ cluster.
For detailed parameter configurations, please refer to the Table Topic Configuration▸ section.