> ## Documentation Index
> Fetch the complete documentation index at: https://docs.automq.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Object Storage Configuration

> Optimize your data storage with AutoMQ, the cloud-native Kafka-compatible solution utilizing leading cloud object storage for superior cost efficiency and scalability.

AutoMQ is a cloud-native, redesigned distribution of Kafka that fully utilizes cloud object storage services. It can build the storage layer using object storage from major cloud providers or storage software.

### Object Storage URI Construction

The URI is a unique identifier used by AutoMQ to represent writes to object storage, adhering to established standards to reconcile differences among various object storage service providers. The URI components are as follows: `s3` specifies the protocol; `xxx_bucket` indicates the specific bucket; `region` and `endpoint` are additional parameters used by different vendors to locate the bucket.

Ultimately, replace the variables in the configuration templates `global.config` and `global.envs` from [Deploy Multi-Nodes Cluster on Linux▸](/automq/deployment/deploy-multi-nodes-cluster-on-linux) to generate the final configuration files.

| Provider | Type | URI Example |
| - | - | - |
| AWS | Cloud Provider | s3://xxx\_bucket?region=us-east-1 |
| AWS-CN | Cloud Provider | s3://xxx\_bucket?region=cn-northwest-1\&endpoint=[https://s3.amazonaws.com.cn](https://s3.amazonaws.com.cn) |
| ALIYUN | Cloud Provider | s3://xxx\_bucket?region=oss-cn-shanghai\&endpoint=[https://oss-cn-shanghai.aliyuncs.com](https://oss-cn-shanghai.aliyuncs.com) |
| OCI  | Cloud Provider  | s3://xxx\_bucket?region=us-ashburn-1\&endpoint=[https://xxx\_namespace.compat.objectstorage.us-ashburn-1.oraclecloud.com\&pathStyle=true](https://xxx_namespace.compat.objectstorage.us-ashburn-1.oraclecloud.com\&pathStyle=true)  |
| CEPH  | Storage Software  | s3://xxx\_bucket?region=us-west-2\&endpoint=[http://127.0.0.1:80](http://127.0.0.1:80)  |
| CubeFS  | Storage Software  | s3://xxx\_bucket?region=us-west-2\&endpoint=[http://10.1.0.240:17410](http://10.1.0.240:17410)  |
| MinIO  | Storage Software  | s3://xxx\_bucket?region=us-west-2\&pathStyle=true\&endpoint=[http://10.1.0.240:9000](http://10.1.0.240:9000)  |

> [!TIP]
>
> If the `s3.wal.path` is set, you can also include the `batchInterval` and `maxBytesInBatch` parameters in the URI to control the submit interval and the maximum size of bytes per batch, respectively. The default settings are `batchInterval=250` milliseconds and `maxBytesInBatch=8388608` bytes.

### Cloud Object Storage Configuration

**AWS**

It is recommended to refer to the [Policy](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-policy-language-overview.html) and [Endpoint & Region](https://docs.aws.amazon.com/general/latest/gr/s3.html) links in the AWS documentation to get detailed configuration instructions for IAM and S3.

> [!NOTE]
>
> In a production environment, it is advisable to scope IAM Policy permissions to specific buckets to avoid unintended authorizations.

```json theme={null}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:AbortMultipartUpload",
                "s3:DeleteObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

```

**Aliyun**

It is recommended to refer to the official Alibaba Cloud documentation on [RAM Permission Policy Language Overview](https://www.alibabacloud.com/help/en/ram/user-guide/policy-overview) and [OSS Endpoints and Regions](https://www.alibabacloud.com/help/en/oss/regions-and-endpoints) for detailed instructions on configuring RAM permissions and OSS.

> [!NOTE]
>
> In a production environment, it is advisable to scope RAM policy permissions to specific buckets (OSS storage spaces) to avoid overly broad authorization and reduce security risks.

```json theme={null}
{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:PutObject",
        "oss:AbortMultipartUpload",
        "oss:GetObject",
        "oss:DeleteObject",
        "oss:PutObjectTagging"
      ],
      "Resource": [
         "acs:oss:*:*:examplebucket"
      ]
    }
  ]
}

```

**GCP**

Since the AWS S3 SDK cannot directly access GCP Cloud Storage, AutoMQ Open Source cannot run directly on Google Cloud Storage. If you need to run AutoMQ on GCP, [contact the AutoMQ team](https://www.automq.com/contact) for support. AutoMQ BYOC supports GCP Cloud Storage.

**Azure**

[Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) is not compatible with the AWS S3 network protocol; therefore, AutoMQ Open Source cannot run directly on Azure Blob Storage. Users can address this as follows:

* Set up your own S3 Proxy to access Blob Storage.

* [Contact the AutoMQ team](https://www.automq.com/contact) for support. AutoMQ BYOC supports Azure Blob Storage.

**OCI**

We recommend checking the official website for more detailed information.

It is advisable to consult the [Policy](https://docs.oracle.com/en-us/iaas/Content/Security/Reference/objectstorage_security.htm#iam-policies), [Detailed Policy](https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/objectstoragepolicyreference.htm#Details), and [Endpoint & Region](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm#usingAPI) links, and review OCI documentation for more detailed configuration instructions on IAM and Object Storage.

```text theme={null}
Allow group 'AutoMQ PoC' to manage objects in tenancy where any {target.bucket.name='automq-data', target.bucket.name='automq-ops'}
```

**Other Cloud Providers**

To deploy AutoMQ in other public cloud environments, make sure that the cloud provider's object storage service is compatible with the S3 protocol. You can contact the AutoMQ team via [this form](https://www.automq.com/contact) to obtain the necessary assistance and best practices.

### Storage Software Configuration

If you wish to deploy AutoMQ in a private data center, you can also use storage software like MinIO, Ceph, or CubeFS that supports the S3 protocol to provide object storage services.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.