You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cloud/features/scheduler/scheduler.md
+37-25Lines changed: 37 additions & 25 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -171,39 +171,57 @@ For fine-grained control, dependencies can be specified, pinned, or excluded usi
171
171
172
172
## Secret Manager
173
173
174
-
Tobiko Cloud provides a way for you to provide environment variables that will
175
-
be injected into the environment when running your python models. The values
176
-
of these variables are encrypted at rest and only available in the environment
177
-
of your running models.
174
+
Tobiko Cloud provides a secrets manager where you can define environment variables for your project's Python models.
175
+
176
+
These variables are most commonly used to provide sensitive information to Python models, such as API keys or other credentials.
177
+
178
+
Secret values are encrypted at rest and only available in the environment of your running Python models.
178
179
179
180
!!! note "Cloud Scheduler Only"
180
181
181
-
Secrets from the secret manager do not load into hybrid executors. They are
182
-
only used for cloud scheduler executors.
182
+
Secrets from the secret manager do not load into hybrid executors. They are only used for cloud scheduler executors.
183
183
184
-
In your cloud instance you can find Secrets under the Settings section and will
185
-
look like this:
184
+
Secret names have two restrictions - they must:
186
185
187
-

186
+
- Start with a letter or an underscore
187
+
- Only include letters, numbers, and underscores (no spaces or other symbols)
188
+
189
+
Secret values have no limits or restrictions. We recommend base64 encoding any secrets that contain binary data.
190
+
191
+
### Defining secrets
188
192
189
-
In this example, only one secret has been defined: `MY_SECRET`. From this panel
190
-
you can create a new secret, edit the value of your secrets, or remove them. You
191
-
can not view the value of any previously created secret.
193
+
Define a secret on the Secrets page, accessible via the Settings section in Tobiko Cloud's left side navigation bar.
194
+
195
+
The Secrets page has a single panel you use to create a new secret, edit the value of an existing secret, or remove an existing secret. You cannot view the value of any existing secret.
196
+
197
+
In this example, only one secret has been defined: `MY_SECRET`. Update its value by entering a new value in the Secret field and clicking the `Update` button, or delete it by clicking the `Remove` button.
198
+
199
+

192
200
193
-
Secret names must start with a letter or an underscore. They may only include
194
-
letters, numbers and underscores (no spaces or other symbols). There are no
195
-
limits or restrictions on the value of secrets. We recommend base64 encoding
196
-
secrets if they contain binary data.
197
201
198
202
### Python Model Example
199
203
200
-
```python
204
+
This Python model demonstrates how to read the `MY_SECRET` secret from an environment variable.
205
+
206
+
!!! danger "Protecting Secrets"
207
+
208
+
Only read environment variables from inside a Python model's `execute` function definition (not in the global scope).
209
+
210
+
If the variable is read in the global scope, SQLMesh will load the value from *your local system* when it renders the Python model instead of loading it at runtime on our executors.
211
+
212
+
This could expose sensitive information or embed an incorrect local value in the rendered model.
213
+
214
+
```python linenums="1"
201
215
import os
216
+
import pandas as pd
202
217
import typing as t
203
218
from datetime import datetime
204
219
205
220
from sqlmesh import ExecutionContext, model
206
221
222
+
# DO NOT read environment variables here.
223
+
# Only inside the `execute` function definition!
224
+
207
225
@model(
208
226
"my_model.name",
209
227
columns={
@@ -220,12 +238,6 @@ def execute(
220
238
221
239
# Read a secret from the MY_SECRET environment variable
222
240
my_secret = os.environ["MY_SECRET"]
223
-
```
224
241
225
-
!!! warning "Protecting Secrets"
226
-
227
-
It's very important that you read environment variables from inside the models
228
-
`execute` function and not in the global scope. If the variable is loaded in the
229
-
global scope than the value will be read from your local system and saved
230
-
into the rendered version of this model, instead of loaded at runtime on our
0 commit comments