doc: add cloud secrets docs (#4563)

Co-authored-by: Trey Spiller <treyspiller@gmail.com>

Author: Christopher Giroir

docs/cloud/features/scheduler/scheduler.md

@@ -167,4 +167,77 @@ Tobiko Cloud automatically manages Python dependencies of your Python macros and
 
 SQLMesh automatically infers which Python libraries are used by statically analyzing the code of your models and macros.
 
-For fine-grained control, dependencies can be specified, pinned, or excluded using the `sqlmesh-requirements.lock` file. See the [Python library dependencies](../../../guides/configuration.md#python-library-dependencies) section in the SQLMesh configuration guide for more information.
+For fine-grained control, dependencies can be specified, pinned, or excluded using the `sqlmesh-requirements.lock` file. See the [Python library dependencies](../../../guides/configuration.md#python-library-dependencies) section in the SQLMesh configuration guide for more information.
+
+## Secret Manager
+
+Tobiko Cloud provides a secrets manager where you can define environment variables for your project's Python models.
+
+These variables are most commonly used to provide sensitive information to Python models, such as API keys or other credentials.
+
+Secret values are encrypted at rest and only available in the environment of your running Python models.
+
+!!! note "Cloud Scheduler Only"
+
+    Secrets from the secret manager do not load into hybrid executors. They are only used for cloud scheduler executors.
+
+Secret names have two restrictions - they must:
+
+- Start with a letter or an underscore
+- Only include letters, numbers, and underscores (no spaces or other symbols)
+
+Secret values have no limits or restrictions. We recommend base64 encoding any secrets that contain binary data.
+
+### Defining secrets
+
+Define a secret on the Secrets page, accessible via the Settings section in Tobiko Cloud's left side navigation bar.
+
+The Secrets page has a single panel you use to create a new secret, edit the value of an existing secret, or remove an existing secret. You cannot view the value of any existing secret.
+
+In this example, only one secret has been defined: `MY_SECRET`. Update its value by entering a new value in the Secret field and clicking the `Update` button, or delete it by clicking the `Remove` button.
+
+![secrets_panel](./scheduler/secrets.png)
+
+
+### Python Model Example
+
+This Python model demonstrates how to read the `MY_SECRET` secret from an environment variable.
+
+!!! danger "Protecting Secrets"
+
+    Only read environment variables from inside a Python model's `execute` function definition (not in the global scope).
+
+    If the variable is read in the global scope, SQLMesh will load the value from *your local system* when it renders the Python model instead of loading it at runtime on our executors.
+
+    This could expose sensitive information or embed an incorrect local value in the rendered model.
+
+```python linenums="1"
+import os
+import pandas as pd
+import typing as t
+from datetime import datetime
+
+from sqlmesh import ExecutionContext, model
+
+# DO NOT read environment variables here.
+# Only inside the `execute` function definition!
+
+@model(
+    "my_model.name",
+    columns={
+        "column_name": "int",
+    },
+)
+def execute(
+    context: ExecutionContext,
+    start: datetime,
+    end: datetime,
+    execution_time: datetime,
+    **kwargs: t.Any,
+) -> pd.DataFrame:
+
+    # Read a secret from the MY_SECRET environment variable
+    my_secret = os.environ["MY_SECRET"]
+
+    ...
+```