Can we use S3 in Opik for Clickhouse/Mysql/Minio

[GopiSharma]

What component(s) are affected?

• Opik Python SDK
• Opik Typescript SDK
• Opik Agent Optimizer SDK
• Opik UI
• Opik Server
• Documentation

Opik version

Latest

Describe the problem

We have deployed opik in EC2 via docker-compose and mounted clickhouse/minio/mysql on ebs. Just wondering if we can mount them on S3. If yes what would be the configurations in docker-compose.

Reproduction steps and code snippets

No response

Error logs or stack trace

No response

Healthcheck results

No response

[github-actions]

Now I have a comprehensive picture of the issue. Let me write the response.

Hi, I'm Scout 🦉, the comet-ml/opik repository agent. Let me look into this.

Solution / Workaround

This is a great deployment architecture question. Here's the breakdown for each component:

---

1. MinIO → AWS S3 (most straightforward)

Opik's backend already speaks the S3 API — it uses environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, IS_MINIO, and S3_URL to configure the object store. To switch from the bundled MinIO to real AWS S3:

In docker-compose.yaml, remove the minio and mc services, and update the backend service env vars:

backend:
  environment:
    AWS_ACCESS_KEY_ID: <your-aws-key>
    AWS_SECRET_ACCESS_KEY: <your-aws-secret>
    IS_MINIO: false        # ← flip to false
    S3_BUCKET: public      # ← your target bucket name
    # Remove or leave blank S3_URL — the SDK will use AWS endpoints by default

You'll also need to pre-create the S3 bucket (the mc init container currently does this for MinIO).

---

2. ClickHouse on S3 / S3-backed disk

ClickHouse natively supports S3 as a storage disk via its S3 disk configuration. You can add this to the ClickHouse config in deployment/docker-compose/clickhouse_config/additional_config.xml:

<clickhouse>
    <storage_configuration>
        <disks>
            <s3>
                <type>s3</type>
                <endpoint>https://s3.amazonaws.com/your-bucket/clickhouse/</endpoint>
                <access_key_id>YOUR_ACCESS_KEY</access_key_id>
                <secret_access_key>YOUR_SECRET_KEY</secret_access_key>
            </s3>
        </disks>
        <policies>
            <s3_main>
                <volumes>
                    <main>
                        <disk>s3</disk>
                    </main>
                </volumes>
            </s3_main>
        </policies>
    </storage_configuration>
</clickhouse>

Then remove or reduce the clickhouse volume mount (~/opik/clickhouse/data) from docker-compose.yaml, and ClickHouse will store table data on S3. Note that ClickHouse still needs local disk for coordination/temp files, but the heavy data goes to S3.

---

3. MySQL on S3

MySQL cannot use S3 directly as its data store — it requires a POSIX-compatible block device. Your current EBS setup is the correct approach for MySQL. Common patterns for MySQL + AWS:

• Keep MySQL on EBS but take EBS snapshots for backup.
• Use Amazon RDS (managed MySQL) instead of the containerized MySQL, pointing STATE_DB_URL in the backend service to your RDS endpoint.

---

Code Investigation

The relevant configuration lives in:

• deployment/docker-compose/docker-compose.yaml — defines all services. The backend service uses IS_MINIO: true, S3_URL: http://minio:9000, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY to configure S3/MinIO object storage.
• deployment/docker-compose/clickhouse_config/additional_config.xml — ClickHouse custom config. This is where an <storage_configuration> block for S3 disks should be added.
• deployment/helm_chart/opik/values.yaml — the Helm chart already has a clickhouse.backup section that uses S3('${CLICKHOUSE_BACKUP_BUCKET}/...') for ClickHouse-native S3 backups, confirming S3 compatibility is baked in.

No test coverage gap is relevant here — this is a deployment configuration question, not a code bug.

Next Steps

There are no code changes needed — this is purely a deployment configuration task. The recommended approach for your EC2/Docker Compose setup:

1. MinIO → S3: Set IS_MINIO: false, remove MinIO containers, point credentials at your AWS S3 bucket.
2. ClickHouse → S3-backed storage: Add an <storage_configuration> block to additional_config.xml as shown above.
3. MySQL: Use Amazon RDS for MySQL and update STATE_DB_URL in the backend service, or keep on EBS with snapshot-based backups.

It would be very helpful if the Opik documentation added a dedicated "Using AWS S3 in production" guide covering these three components — if you'd like to contribute that, the docs live in apps/opik-documentation/documentation/docs/. Feel free to open a PR!

Was this helpful? React to this comment with 👍 or 👎 to rate my response.

---

View Scout trace in Opik

[GopiSharma]

Hi Team, thanks for your detailed response,

Regarding clickhouse table:

Is just adding storage configuration in deployment/docker-compose/clickhouse_config/additional_config.xml store the existing table data/future data in s3 or do we need to change clickhouse's existing table storage policy to store the data in s3?