docs: alert-notification-templates-v2 — o2-enterprise PR #2353

docs/user-guide/account-administration/management/templates.md

@@ -1,8 +1,7 @@
 ---
-title: Templates
-description: Create alert notification templates in OpenObserve with variable placeholders and row templates for Slack, email, webhooks, and more.
+description: >-
+  Create alert notification templates in OpenObserve with variable placeholders, content templates, markdown rendering, chart images, and row templates for Slack, email, webhooks, and more.
 ---
-
 # Alert Templates
 
 Templates are used when notification is sent for an alert, templates forms body of request being sent to destination, for eg. for slack one can create template like:
@@ -16,6 +15,93 @@ Templates are used when notification is sent for an alert, templates forms body
 
 When a notification is being sent, OpenObserve will replace placeholders like {stream_name}, {alert_name} etc with actual values of stream name, alert name.
 
+## Template kinds
+
+Templates have two kinds:
+
+- **Custom** (default): A free-form JSON or string body with `{var}` placeholders. This is the classic template format — write whatever payload your destination expects.
+- **Content**: A structured template using a `ContentSpec` that OpenObserve renders into the right format for each destination automatically. Content templates give you a richer editing experience with title, Markdown body, fields, links, and optional chart images — and the same template works across Slack, Email, Teams, Discord, and webhooks without rewriting per-channel payloads.
+
+When you create a template from **Management > Templates**, select the kind that matches your needs.
+
+## Content templates
+
+Content templates let you design notifications using a structured specification instead of writing raw JSON for each channel. Define your content once, and OpenObserve renders it correctly for every destination.
+
+![creating a content template with title, body, fields, links, and chart toggle](images/alert-notification-templates-v2-1.png)
+
+A content template includes:
+
+- **Title**: A templated one-liner used as the Slack headline, email subject, or PagerDuty summary. You can set per-channel overrides for specific destinations.
+- **Body**: Markdown with `{var}` placeholders. The body is rendered to HTML for rich channels and converted to plain text for channels that require it.
+- **Fields**: Key-value pairs displayed as a structured field list (like Slack block fields). Each field can include a severity filter so it only appears when the alert is at a specific level (e.g., critical).
+- **Links**: Custom action links with labels and URLs. Links also support severity-based visibility.
+- **Rows**: Configurable row rendering — enable or disable rows, set a maximum count, select specific columns, and define a per-row format template. When disabled, the notification omits the row table entirely.
+- **Chart**: Toggle to include a chart image of the alert data in the notification. See [Alert chart images](#alert-chart-images).
+
+Content templates are stored as a JSON `ContentSpec`. For example:
+
+```json
+{
+  "title": "{alert_name} fired",
+  "body": "**{alert_name}** exceeded threshold\nStream: {stream_name}\nValue: {alert_agg_value}",
+  "fields": [
+    { "label": "Stream", "value": "{stream_name}" },
+    { "label": "Runbook", "value": "https://wiki.example.com/runbooks/cpu", "show_when": { "levels": ["critical"] } }
+  ],
+  "links": [
+    { "label": "View in OpenObserve", "url": "{alert_url}" }
+  ],
+  "rows": { "enabled": true, "max": 10 },
+  "chart": { "enabled": false }
+}
+```
+
+## Preview a template
+
+You can preview how a content template renders for any channel and severity before saving it. The preview uses the exact same rendering code path as sending a live notification, so the output matches production exactly.
+
+![template preview showing rendered card for Slack at critical severity](images/alert-notification-templates-v2-2.png)
+
+To preview a template:
+
+1. Navigate to **Management > Templates**.
+2. Create or edit a content template.
+3. Use the **Preview** panel to select a channel (Slack, Email, Teams, Discord, Webhook, PagerDuty, Opsgenie, ServiceNow, or SNS) and a severity level.
+4. The preview shows:
+   - **Preview model**: A rendered card with title, body HTML, fields, links, and severity color. This is how the notification looks in rich clients.
+   - **Raw payload**: The exact JSON or email content that the send path produces.
+   - **Unknown variables**: Any `{variable}` in your template that isn't recognized, so you can fix typos before alert fires go out with broken placeholders.
+
+The preview uses deterministic sample data (e.g., "Sample CPU alert" with synthetic rows) so the same template always previews identically.
+
+## Alert chart images
+
+Content templates can include a chart image of the alert data. When you enable the **Chart** toggle in a content template, OpenObserve renders a chart of the alert query results using `plotters` and embeds it in the notification.
+
+![Slack notification with embedded alert chart image](images/alert-notification-templates-v2-3.png)
+
+Chart rendering:
+
+- The chart is generated on-demand at notification time using the same query results that triggered the alert.
+- Charts are served via an HMAC-signed URL (`GET /api/v2/{org}/alerts/charts/render`) for security — only the notification recipient can access the chart image.
+- Chart images are rendered as PNG on the node processing the notification.
+- Charts are reused across multiple destinations on the same alert firing, and across retries.
+
+## Test a destination
+
+Before relying on a destination for live alerts, use the test send feature to verify it works correctly.
+
+1. Navigate to **Management > Alert Destinations**.
+2. Select a destination from the list.
+3. Click **Test Send** to dispatch a test notification through that destination.
+
+![destination detail page with Test Send button](images/alert-notification-templates-v2-4.png)
+
+The test notification is marked with `[TEST]` so recipients can distinguish it from a real alert. A test send uses the destination's configured template with synthetic sample data, verifying the full pipeline: template rendering, credential substitution, and delivery.
+
+## Variable reference
+
 Variables which can be used in templates are:
 
 | Variable                      | Value                     | Description                               |
@@ -35,6 +121,12 @@ Variables which can be used in templates are:
 | alert_url                | URL                       | A link can back to the UI and check the detail data. need configure `ZO_WEB_URL` |
 | alert_trigger_time       | _timestamp                | Represents the timestamp in microseconds when the alert was evaluated |
 | alert_trigger_time_str   | _timestamp                | Formatted `alert_trigger_time`. E.g. - `2024-12-02T11:27:40` |
+| alert_level              | Severity level            | The alert severity level: `critical`, `warning`, `ok`, `no_data`, or empty for single-level alerts |
+| alert_priority           | Priority string           | The alert priority label, e.g. `P1`, `P2` |
+| alert_tags               | Tags                      | Comma-separated alert tags, e.g. `infra, prod` |
+| alert_threshold_crit     | Critical threshold        | Threshold value for critical severity |
+| alert_threshold_warn     | Warning threshold         | Threshold value for warning severity |
+| alert_description        | Description               | The alert's description text |
 | rows                     | mutiple lines of row template values | based on `row template` in alert page |
 | all of the stream fields | the field value                      | Default we `select * from stream` if you custom sql then it will be only the fields that you selected. |
 
@@ -97,58 +189,6 @@ And we define the `row template` in alert page:
 
 After these, the notification message will be what we expect.
 
-### JSON row templates
-
-When the row template itself is valid JSON and `{rows}` (or `{rows:N}`) appears in a JSON value position, each row is injected as an element of a JSON array instead of being joined into a single newline-separated string.
-
-For example, with this row template:
-
-```
-{"pod": "{k8s_pod_name}", "count": {cnt}}
-```
-
-and an alert template like:
-
-```json
-{
-  "rows": "{rows}"
-}
-```
-
-the notification body becomes a JSON array, one element per row:
-
-```json
-{
-  "rows": [
-    {"pod": "pod1", "count": 1},
-    {"pod": "pod2", "count": 2},
-    {"pod": "pod3", "count": 1}
-  ]
-}
-```
-
-If the row template is itself a JSON array, each row becomes an array element, so the result is an array of arrays.
-
-### Spread syntax
-
-Use `{...rows}` (or `{...rows:N}`) to flatten array-typed row templates into a single one-dimensional array. Where `{rows}` would produce an array of arrays, `{...rows}` merges all the inner arrays into one flat array.
-
-For example, with a JSON array row template like:
-
-```
-["{k8s_pod_name}", {cnt}]
-```
-
-`{...rows}` produces:
-
-```json
-{
-  "rows": ["pod1", 1, "pod2", 2, "pod3", 1]
-}
-```
-
-As with `{rows:N}`, the `N` in `{...rows:N}` limits the number of rows that are included.
-
 Check this video to understand more
 
 <iframe width="760" height="315" src="https://www.youtube.com/embed/tW8VnNnfZBg?si=9bXSzGXgPER2Gbaw" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>