You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This Docker Compose configuration allows you to run Tobiko Cloud hybrid executors locally or on any server that supports Docker Compose. Hybrid executors enable your own infrastructure to connect to Tobiko Cloud for operations while keeping your data warehouse credentials within your environment.
3
+
This Docker Compose configuration allows you to run Tobiko Cloud hybrid executors locally or on any server that supports Docker Compose.
4
4
5
-
## What This Setup Provides
5
+
Hybrid executors allow you to run operations on your own infrastructure while leveraging Tobiko Cloud for orchestration.
6
6
7
-
The hybrid executors connect your data warehouse to Tobiko Cloud in a secure way:
7
+
## What this setup provides
8
8
9
-
-**Apply Executor**: Executes operations that change state, like creating and updating models
10
-
-**Run Executor**: Runs read-only operations like planning and analysis
9
+
This setup deploys two hybrid executors that pass work tasks from Tobiko Cloud to your data warehouse in a secure way:
10
+
11
+
-**Apply Executor**: Handles applying changes to the data warehouse
12
+
-**Run Executor**: Handles scheduled model execution
13
+
14
+
Both executors must be properly configured with environment variables to connect to Tobiko Cloud and your data warehouse.
11
15
12
16
## Prerequisites
13
17
14
-
- Access to a data warehouse (e.g., Postgres, Snowflake, BigQuery)
18
+
- Access to a [data warehouse supported by Tobiko Cloud](../../../integrations/overview.md#execution-engines) (e.g., Postgres, Snowflake, BigQuery)
15
19
- Docker and Docker Compose
16
-
- A Tobiko Cloud account with client ID and client secret
20
+
- A Tobiko Cloud account with [client ID and client secret](../single_sign_on.md#provisioning-client-credentials)
Download the [docker-compose file](https://raw.githubusercontent.com/TobikoData/sqlmesh/refs/heads/main/docs/cloud/features/scheduler/scheduler/docker-compose.yml) and [.env.example](https://raw.githubusercontent.com/TobikoData/sqlmesh/refs/heads/main/docs/cloud/features/scheduler/scheduler/.env.example) to a local directory.
26
+
Download the [docker-compose.yml](https://raw.githubusercontent.com/TobikoData/sqlmesh/refs/heads/main/docs/cloud/features/scheduler/scheduler/docker-compose.yml) and [.env.example](https://raw.githubusercontent.com/TobikoData/sqlmesh/refs/heads/main/docs/cloud/features/scheduler/scheduler/.env.example) files to a local directory.
25
27
26
28
2.**Create your environment file**:
27
29
30
+
Copy the downloaded example environment file into a new `.env` file:
31
+
28
32
```bash
29
33
cp .env.example .env
30
34
```
31
35
32
-
3.**Edit the .env file** with your specific configuration:
36
+
3.**Edit the .env file** with your project's configuration:
33
37
34
38
- Set your Tobiko Cloud organization, project, client ID, and client secret
35
-
- Configure your database connection details
39
+
- Configure your gateway connection details
36
40
- Adjust resource limits if needed
37
41
38
42
4.**Start the executors**:
@@ -47,47 +51,48 @@ The hybrid executors connect your data warehouse to Tobiko Cloud in a secure way
47
51
docker compose logs -f
48
52
```
49
53
50
-
## Configuration Options
54
+
## Configuration options
51
55
52
-
### Database Configuration
56
+
### Gateway configuration
53
57
54
-
The default configuration is set up for Postgre, but you can use any supported database by adjusting the connection parameters in your `.env` file:
58
+
The default configuration in the `docker-compose.yml` file uses Postgres, but you can use [any supported SQL engine](../../../integrations/overview.md#execution-engines)by adjusting the connection parameters in your `.env` file.
55
59
56
-
### Multiple Gateways
60
+
####Multiple gateways
57
61
58
-
To configure multiple gateways, edit the docker compose.yml file to add additional environment variables for each gateway:
62
+
To configure multiple gateways, add additional environment variables for each gateway the `docker-compose.yml` file:
0 commit comments