diff --git a/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-1.png b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-1.png new file mode 100644 index 00000000..35377c97 Binary files /dev/null and b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-1.png differ diff --git a/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-2.png b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-2.png new file mode 100644 index 00000000..cbdc1d3a Binary files /dev/null and b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-2.png differ diff --git a/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-3.png b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-3.png new file mode 100644 index 00000000..4cdf7081 Binary files /dev/null and b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-3.png differ diff --git a/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-4.png b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-4.png new file mode 100644 index 00000000..49161892 Binary files /dev/null and b/docs/user-guide/account-administration/management/images/alert-notification-templates-v2-4.png differ diff --git a/docs/user-guide/account-administration/management/templates.md b/docs/user-guide/account-administration/management/templates.md index b45666a6..f9b1db49 100644 --- a/docs/user-guide/account-administration/management/templates.md +++ b/docs/user-guide/account-administration/management/templates.md @@ -1,8 +1,7 @@ --- -title: Templates -description: Create alert notification templates in OpenObserve with variable placeholders and row templates for Slack, email, webhooks, and more. +description: >- + Create alert notification templates in OpenObserve with variable placeholders, content templates, markdown rendering, chart images, and row templates for Slack, email, webhooks, and more. --- - # Alert Templates Templates are used when notification is sent for an alert, templates forms body of request being sent to destination, for eg. for slack one can create template like: @@ -16,6 +15,93 @@ Templates are used when notification is sent for an alert, templates forms body When a notification is being sent, OpenObserve will replace placeholders like {stream_name}, {alert_name} etc with actual values of stream name, alert name. +## Template kinds + +Templates have two kinds: + +- **Custom** (default): A free-form JSON or string body with `{var}` placeholders. This is the classic template format — write whatever payload your destination expects. +- **Content**: A structured template using a `ContentSpec` that OpenObserve renders into the right format for each destination automatically. Content templates give you a richer editing experience with title, Markdown body, fields, links, and optional chart images — and the same template works across Slack, Email, Teams, Discord, and webhooks without rewriting per-channel payloads. + +When you create a template from **Management > Templates**, select the kind that matches your needs. + +## Content templates + +Content templates let you design notifications using a structured specification instead of writing raw JSON for each channel. Define your content once, and OpenObserve renders it correctly for every destination. + +![creating a content template with title, body, fields, links, and chart toggle](images/alert-notification-templates-v2-1.png) + +A content template includes: + +- **Title**: A templated one-liner used as the Slack headline, email subject, or PagerDuty summary. You can set per-channel overrides for specific destinations. +- **Body**: Markdown with `{var}` placeholders. The body is rendered to HTML for rich channels and converted to plain text for channels that require it. +- **Fields**: Key-value pairs displayed as a structured field list (like Slack block fields). Each field can include a severity filter so it only appears when the alert is at a specific level (e.g., critical). +- **Links**: Custom action links with labels and URLs. Links also support severity-based visibility. +- **Rows**: Configurable row rendering — enable or disable rows, set a maximum count, select specific columns, and define a per-row format template. When disabled, the notification omits the row table entirely. +- **Chart**: Toggle to include a chart image of the alert data in the notification. See [Alert chart images](#alert-chart-images). + +Content templates are stored as a JSON `ContentSpec`. For example: + +```json +{ + "title": "{alert_name} fired", + "body": "**{alert_name}** exceeded threshold\nStream: {stream_name}\nValue: {alert_agg_value}", + "fields": [ + { "label": "Stream", "value": "{stream_name}" }, + { "label": "Runbook", "value": "https://wiki.example.com/runbooks/cpu", "show_when": { "levels": ["critical"] } } + ], + "links": [ + { "label": "View in OpenObserve", "url": "{alert_url}" } + ], + "rows": { "enabled": true, "max": 10 }, + "chart": { "enabled": false } +} +``` + +## Preview a template + +You can preview how a content template renders for any channel and severity before saving it. The preview uses the exact same rendering code path as sending a live notification, so the output matches production exactly. + +![template preview showing rendered card for Slack at critical severity](images/alert-notification-templates-v2-2.png) + +To preview a template: + +1. Navigate to **Management > Templates**. +2. Create or edit a content template. +3. Use the **Preview** panel to select a channel (Slack, Email, Teams, Discord, Webhook, PagerDuty, Opsgenie, ServiceNow, or SNS) and a severity level. +4. The preview shows: + - **Preview model**: A rendered card with title, body HTML, fields, links, and severity color. This is how the notification looks in rich clients. + - **Raw payload**: The exact JSON or email content that the send path produces. + - **Unknown variables**: Any `{variable}` in your template that isn't recognized, so you can fix typos before alert fires go out with broken placeholders. + +The preview uses deterministic sample data (e.g., "Sample CPU alert" with synthetic rows) so the same template always previews identically. + +## Alert chart images + +Content templates can include a chart image of the alert data. When you enable the **Chart** toggle in a content template, OpenObserve renders a chart of the alert query results using `plotters` and embeds it in the notification. + +![Slack notification with embedded alert chart image](images/alert-notification-templates-v2-3.png) + +Chart rendering: + +- The chart is generated on-demand at notification time using the same query results that triggered the alert. +- Charts are served via an HMAC-signed URL (`GET /api/v2/{org}/alerts/charts/render`) for security — only the notification recipient can access the chart image. +- Chart images are rendered as PNG on the node processing the notification. +- Charts are reused across multiple destinations on the same alert firing, and across retries. + +## Test a destination + +Before relying on a destination for live alerts, use the test send feature to verify it works correctly. + +1. Navigate to **Management > Alert Destinations**. +2. Select a destination from the list. +3. Click **Test Send** to dispatch a test notification through that destination. + +![destination detail page with Test Send button](images/alert-notification-templates-v2-4.png) + +The test notification is marked with `[TEST]` so recipients can distinguish it from a real alert. A test send uses the destination's configured template with synthetic sample data, verifying the full pipeline: template rendering, credential substitution, and delivery. + +## Variable reference + Variables which can be used in templates are: | Variable | Value | Description | @@ -35,6 +121,12 @@ Variables which can be used in templates are: | alert_url | URL | A link can back to the UI and check the detail data. need configure `ZO_WEB_URL` | | alert_trigger_time | _timestamp | Represents the timestamp in microseconds when the alert was evaluated | | alert_trigger_time_str | _timestamp | Formatted `alert_trigger_time`. E.g. - `2024-12-02T11:27:40` | +| alert_level | Severity level | The alert severity level: `critical`, `warning`, `ok`, `no_data`, or empty for single-level alerts | +| alert_priority | Priority string | The alert priority label, e.g. `P1`, `P2` | +| alert_tags | Tags | Comma-separated alert tags, e.g. `infra, prod` | +| alert_threshold_crit | Critical threshold | Threshold value for critical severity | +| alert_threshold_warn | Warning threshold | Threshold value for warning severity | +| alert_description | Description | The alert's description text | | rows | mutiple lines of row template values | based on `row template` in alert page | | all of the stream fields | the field value | Default we `select * from stream` if you custom sql then it will be only the fields that you selected. | @@ -97,58 +189,6 @@ And we define the `row template` in alert page: After these, the notification message will be what we expect. -### JSON row templates - -When the row template itself is valid JSON and `{rows}` (or `{rows:N}`) appears in a JSON value position, each row is injected as an element of a JSON array instead of being joined into a single newline-separated string. - -For example, with this row template: - -``` -{"pod": "{k8s_pod_name}", "count": {cnt}} -``` - -and an alert template like: - -```json -{ - "rows": "{rows}" -} -``` - -the notification body becomes a JSON array, one element per row: - -```json -{ - "rows": [ - {"pod": "pod1", "count": 1}, - {"pod": "pod2", "count": 2}, - {"pod": "pod3", "count": 1} - ] -} -``` - -If the row template is itself a JSON array, each row becomes an array element, so the result is an array of arrays. - -### Spread syntax - -Use `{...rows}` (or `{...rows:N}`) to flatten array-typed row templates into a single one-dimensional array. Where `{rows}` would produce an array of arrays, `{...rows}` merges all the inner arrays into one flat array. - -For example, with a JSON array row template like: - -``` -["{k8s_pod_name}", {cnt}] -``` - -`{...rows}` produces: - -```json -{ - "rows": ["pod1", 1, "pod2", 2, "pod3", 1] -} -``` - -As with `{rows:N}`, the `N` in `{...rows:N}` limits the number of rows that are included. - Check this video to understand more