diff --git a/explore-analyze/alerting.md b/explore-analyze/alerting.md index 67425e5f4b..c080782821 100644 --- a/explore-analyze/alerting.md +++ b/explore-analyze/alerting.md @@ -27,7 +27,7 @@ stack: experimental 9.5+ serverless: experimental ``` -The {{alerting-v2-system}} is built on {{esql}}. You write the query that defines what to watch for, choose how alert episodes are tracked per series, and control notifications through action policies that handle routing, frequency, and notification batching. The {{alerting-v2-system}} also adds alert episode lifecycle tracking, per-series snooze, and rules on alert episodes for correlation and escalation. It is a strong fit when you want full control over what data travels with each alert episode and how your team is notified. +The {{alerting-v2-system}} is built on {{esql}}. You write the query that defines what to watch for, choose whether matches open tracked alert episodes or are recorded as signals, and control notifications through action policies that handle routing, frequency, and notification batching. The {{alerting-v2-system}} also adds alert episode lifecycle tracking, per-series snooze, queryable signal history, and rules that can correlate signals for escalation. It is a strong fit when you want full control over what data travels with each detection and how your team is notified. :::{note} The {{alerting-v2-system}} runs next to {{kib}} alerting on {{serverless-full}} and {{stack}} 9.5 and later. You don't have to move everything at once. You can copy or rebuild rules when you're ready, and your existing {{kib}} alerting rules won't be affected. diff --git a/explore-analyze/alerting/compare-alerting-systems.md b/explore-analyze/alerting/compare-alerting-systems.md index ed02136152..3385c6f2d0 100644 --- a/explore-analyze/alerting/compare-alerting-systems.md +++ b/explore-analyze/alerting/compare-alerting-systems.md @@ -21,8 +21,9 @@ Elastic offers three alerting systems, each suited to different use cases and wo |---|---|---| | Monitor metrics, logs, or uptime with ready-made rules and no query language | [{{kib}} alerting](alerts.md) | {applies_to}`stack: ga` {applies_to}`serverless: ga` | | Use rules built for {{elastic-sec}}, {{observability}}, APM, or Maps | [{{kib}} alerting](alerts.md) | {applies_to}`stack: ga` {applies_to}`serverless: ga` | -| Write {{esql}} to define exactly what to detect and what data each alert episode carries | [{{alerting-v2-system-cap}}](system-overview.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | -| Query alert history in Discover or build dashboards from alert data | [{{alerting-v2-system-cap}}](system-overview.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | +| Write {{esql}} to define exactly what to detect and what data each alert episode or signal carries | [{{alerting-v2-system-cap}}](system-overview.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | +| Record detections as signals for later analysis without opening episodes or sending notifications | [{{alerting-v2-system-cap}}](system-overview.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | +| Query alert and signal history in Discover or build dashboards from rule event data | [{{alerting-v2-system-cap}}](system-overview.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | | Manage notification routing, grouping, and throttling in one place, reusable across rules | [{{alerting-v2-system-cap}}](system-overview.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | | Build highly custom notification logic with reusable, configurable workflows | [{{alerting-v2-system-cap}}](system-overview.md) | {applies_to}`serverless: experimental` {applies_to}`stack: experimental 9.5+` | | Build highly custom logic with scripting and chained inputs | [Watcher](watcher.md) | {applies_to}`stack: ga` {applies_to}`serverless: unavailable` | @@ -33,7 +34,7 @@ Elastic offers three alerting systems, each suited to different use cases and wo |---|---|---|---| | **Best for** | Teams using built-in rule types with form-based setup | Teams that need full control over detection and notification routing | Custom alerting logic requiring scripting | | **Rule definition** | Select a rule type and fill in parameters | [Write an {{esql}} query](experimental-alerting-system/rules/configure-rule-query.md) or use a rule builder with form-based setup | Write a JSON watch definition | -| **Alert data** | In-place updates, limited query support | [Append-only events queryable with {{esql}} in Discover](experimental-alerting-system/alerts/query-alerts-and-signals-in-discover.md) | Watch history index | +| **Alert and signal data** | In-place updates, limited query support | Append-only [rule events](experimental-alerting-system/alerts/alert-data-model.md) for alert episodes and signals, [queryable with {{esql}} in Discover](experimental-alerting-system/alerts/query-alerts-and-signals-in-discover.md) | Watch history index | | **Notifications** | Configured per action on each rule | [Centralized action policies](experimental-alerting-system/notifications-actions.md), reusable across rules; supports action-level throttling and conditions | Action-level throttling and conditions | | **Noise reduction** | Snooze per rule or per alert, maintenance windows | Per-episode acknowledge or deactivate, per-series snooze, maintenance windows, match condition routing in [action policies](experimental-alerting-system/action-policies/reduce-notification-noise.md) | Action conditions and throttling | | **Available on {{serverless-full}}** | Yes | Yes, {applies_to}`serverless: experimental` | No | diff --git a/explore-analyze/alerting/experimental-alerting-system/action-policies/about-action-policies.md b/explore-analyze/alerting/experimental-alerting-system/action-policies/about-action-policies.md index def8246f61..d4a31f203a 100644 --- a/explore-analyze/alerting/experimental-alerting-system/action-policies/about-action-policies.md +++ b/explore-analyze/alerting/experimental-alerting-system/action-policies/about-action-policies.md @@ -14,7 +14,7 @@ An action policy is the gating layer between an alert episode and a workflow in ## Why action policies are separate from rules [policies-separate-from-rules] -Action policies are independent of rules. A single action policy can cover alert episodes from many rules, so an action policy matching `severity: "critical"` applies regardless of which rule produced the alert episode. You can also update notification routing without touching any rule, and you can create rules without any action policy, which is useful for testing detection logic before wiring up notifications. +Action policies are independent of rules. A single action policy can cover alert episodes from many rules, so an action policy matching `severity: "critical"` applies regardless of which rule produced the alert episode. You can create a rule without any action policy, which is useful for testing detection logic before wiring up notifications. You can also update notification routing later without touching the rule. To scope an action policy to one rule, use a matcher expression, for example `rule.id: "my-rule-id"`. diff --git a/explore-analyze/alerting/experimental-alerting-system/alerts.md b/explore-analyze/alerting/experimental-alerting-system/alerts.md index c006f095d1..138f5ca4ef 100644 --- a/explore-analyze/alerting/experimental-alerting-system/alerts.md +++ b/explore-analyze/alerting/experimental-alerting-system/alerts.md @@ -33,9 +33,9 @@ inactive → pending → active → recovering → inactive A checkout-latency rule runs every 5 minutes. It has an activation threshold of 2 consecutive breaches and a recovery threshold of 2 consecutive clears. The episode opens only after consecutive breaches meet the activation threshold and closes only after consecutive clears meet the recovery threshold. The system waits for confirmation in both directions. 1. **14:00**: Routine check. p95 is within budget. No episode exists yet. The series is `inactive`. -2. **14:05**: p95 jumps to 3.1s. The rule detects the first breach. The system creates the episode in `pending` and starts counting consecutive breaches. +2. **14:05**: p95 jumps to 3.1s. The rule detects the first breach, creates the episode in `pending`, and starts counting consecutive breaches. 3. **14:10**: p95 is still elevated. The second consecutive breach meets the activation threshold. The episode moves from `pending` to `active`. The system pages the engineer. -4. **14:10–14:45**: Every evaluation finds high latency. The episode stays `active`. The system doesn't create new episodes. One episode tracks one problem, no matter how many times the rule evaluates while the condition holds. +4. **14:10–14:45**: Every evaluation finds high latency. The episode stays `active`. The rule doesn't create new episodes. One episode tracks one problem, no matter how many times the rule evaluates while the condition holds. 5. **14:50**: p95 drops back under 2s. The first clean check moves the episode from `active` to `recovering`. The system starts counting consecutive clears. 6. **14:55**: A second consecutive clear meets the recovery threshold. The episode moves from `recovering` to `inactive`. The engineer receives a recovery notification. diff --git a/explore-analyze/alerting/experimental-alerting-system/glossary.md b/explore-analyze/alerting/experimental-alerting-system/glossary.md index b1868cc000..49eef2bf7d 100644 --- a/explore-analyze/alerting/experimental-alerting-system/glossary.md +++ b/explore-analyze/alerting/experimental-alerting-system/glossary.md @@ -41,7 +41,7 @@ These terms appear throughout the {{alerting-v2-system}} docs. If a term is uncl : A label attached to alert episodes to indicate urgency. Severity is available as a filter in action policies so critical episodes can be routed differently from low-priority ones. To learn more, refer to [Configure rule severity](rules/configure-rule-severity.md). **Signal** -: A rule event recorded when a rule runs in Signal mode. Signals are stored and queryable in Discover, but they don't open an alert episode or trigger notifications. To learn more, refer to [Rule mode](rules/configure-rule-mode.md). +: A rule event recorded when a rule runs in Signal mode. Signals are stored in `.rule-events` and queryable in Discover, but they don't open an alert episode or trigger notifications. To learn more, refer to [Rule mode](rules/configure-rule-mode.md). **Threshold** : The condition a rule uses to decide when something is worth alerting on, including how many times the condition must be met before an alert episode opens or closes. To learn more, refer to [Alert delay](rules/configure-rule-alert-delay.md) and [Recovery condition](rules/configure-rule-recovery.md). diff --git a/explore-analyze/alerting/experimental-alerting-system/how-it-works.md b/explore-analyze/alerting/experimental-alerting-system/how-it-works.md index 77dc8b1a65..bd8af726fc 100644 --- a/explore-analyze/alerting/experimental-alerting-system/how-it-works.md +++ b/explore-analyze/alerting/experimental-alerting-system/how-it-works.md @@ -15,13 +15,13 @@ This page walks through what happens at each step after a rule runs, and broken ## Rule runs in Alert mode [how-alert-mode-works] -In Alert mode, the rule doesn't just record that a condition was found. It opens an alert episode that persists and tracks the problem until the condition clears. Each time the rule runs, it writes a rule event that can advance the episode's lifecycle state. An action policy sits between the episode and your team, deciding whether and when to trigger a workflow. +In Alert mode, the rule doesn't just record that a condition was found. Each match causes the rule to open an alert episode that persists and tracks the problem until the condition clears. Each time the rule runs, it writes a rule event that can advance the episode's lifecycle state. An action policy sits between the episode and your team, deciding whether and when to trigger a workflow. | Step | Actor | Action | |------|-------|--------| | 1 | Rule | Runs on schedule and evaluates {{esql}} against your data | | 2 | Rule | Query returns results → A rule event is written to `.rule-events` | -| 3 | System | Creates an alert episode and sets its initial state to `pending`; episode advances to `active` once the activation threshold is met | +| 3 | Rule | Creates an alert episode and sets its initial state to `pending`; episode advances to `active` once the activation threshold is met | | 4 | Action policy | Evaluates the episode against its conditions (checks for episode eligibility, match conditions, and frequency) | | 5 | Action policy | If conditions are met, triggers a workflow | | 6 | Workflow | Sends notification or runs automation | @@ -51,7 +51,7 @@ In Signal mode, the rule acts purely as a data producer. Each time the rule runs |------|-------|--------| | 1 | Rule | Runs on schedule and evaluates {{esql}} against your data | | 2 | Rule | Query returns results → Writes a rule event (signal) to `.rule-events` | -| 3 | System | Signal is immediately queryable in Discover, dashboards, and {{esql}} | +| 3 | Rule | Signal is immediately queryable in Discover, dashboards, and {{esql}} | No alert episode is opened. No action policy evaluates the result. No notification is sent. diff --git a/explore-analyze/alerting/experimental-alerting-system/rules.md b/explore-analyze/alerting/experimental-alerting-system/rules.md index 149dc7bef2..a00447d86c 100644 --- a/explore-analyze/alerting/experimental-alerting-system/rules.md +++ b/explore-analyze/alerting/experimental-alerting-system/rules.md @@ -24,7 +24,7 @@ In Signal mode, each matching row is stored as a signal document with no alert l Rules only define *what* to detect. They don't control notifications, who gets notified, or when. That's the job of action policies, which are global objects scoped to your space that match alert episodes from any rule. A rule has no say in which action policies pick it up. -This separation means you can build and test a rule without anyone getting paged, update notification routing without touching the rule, and have multiple action policies respond to the same rule independently. +This separation means you can update notification routing without touching a rule, and have multiple action policies respond to the same rule independently. ## What to do next with rules [rules-next-steps] diff --git a/explore-analyze/alerting/system-overview.md b/explore-analyze/alerting/system-overview.md index 377a27e8a2..0cc4ab263f 100644 --- a/explore-analyze/alerting/system-overview.md +++ b/explore-analyze/alerting/system-overview.md @@ -6,65 +6,72 @@ applies_to: products: - id: kibana - id: cloud-serverless -description: The experimental Kibana alerting system uses ES|QL rules to detect conditions, track problems as alert episodes, and route notifications through reusable action policies. +description: The experimental Kibana alerting system uses ES|QL rules to detect conditions, then either track matches as alert episodes with notifications or record signals for later analysis. --- # {{alerting-v2-system-cap}} overview [system-overview] -The {{alerting-v2-system}} in {{kib}} watches your {{es}} data continuously, so your team doesn't have to. You define the conditions that matter, such as when to open an issue, who should know, and how often to notify them. The system handles the rest. +The {{alerting-v2-system}} in {{kib}} watches your {{es}} data continuously, so your team doesn't have to. You define the conditions that matter, and select whether each match opens a tracked alert episode or records a signal for later analysis. The system handles the rest. ::::{note} -In the generally available {{kib}} alerting system, the term **alert** refers to a tracked occurrence of a rule condition. In the {{alerting-v2-system}}, the equivalent concept is called an **alert episode**. The two terms describe similar ideas in different systems and are not interchangeable. +In the generally available {{kib}} alerting system, the term **alert** refers to a tracked occurrence of a rule condition. In the {{alerting-v2-system}}, the equivalent concept is called an **alert episode**. Each system's APIs, UI, and instructions apply only to that system's concepts. :::: ## The core idea [core-idea] -The {{alerting-v2-system}} separates *detecting* a problem from *acting* on it: +The {{alerting-v2-system}} starts with a rule evaluating your data. When the rule detects a match, it either creates an alert episode (Alert mode) or records the match as a signal (Signal mode). -- **Detecting** - Rules focus purely on what to watch for in your data and on collecting breach and recovery events. -- **Acting** - Action policies handle who gets notified, when, and how, independently of any rule. +:::{image} /explore-analyze/images/basic-system-flow.png +:alt: Flowchart showing that after a rule detects a match, it either creates an alert episode or records a signal +::: -You can build and test detection logic before wiring up any notifications, and update notification routing across all rules in one place without editing the rules themselves. +Because acting and recording are independent, you can switch a rule between these modes as your needs change. For example, you can test a rule in Signal mode before switching it to Alert mode and setting up notifications. Notifications are handled separately by action policies, so you can update where notifications are sent for multiple rules without editing each rule individually. -## The four building blocks +## The building blocks -The {{alerting-v2-system}} is built around four objects: rules, alert episodes, action policies, and workflows, each with a distinct role. +The {{alerting-v2-system}} is built around five objects: rules, alert episodes, signals, action policies, and workflows, each with a distinct role. ### Rules -A rule defines what to watch for in your data and how often to check, and runs in one of two modes: alert, which opens and tracks an alert episode until the condition clears, or signal, which records results over time without opening episodes or sending notifications. +A rule defines what to watch for in your data and how often to check. It runs in one of two modes: Alert mode or Signal mode. The rule's mode decides how the match is handled. In Alert mode, an alert episode is created to track the match. In Signal mode, the match is recorded as a signal. Refer to [Rules](experimental-alerting-system/rules.md) to learn more. ### Alert episodes -In Alert mode, the rule opens one alert episode per problem and keeps it open until the condition clears. The alert episode moves through states (pending, active, recovering, inactive), giving you one lifecycle to triage rather than a separate item per rule check. +In Alert mode, one alert episode is created per match. The episode moves through states (pending, active, recovering, inactive), giving you one lifecycle to triage rather than a separate item per rule check. Alert episodes are passed to action policies for evaluation. Refer to [Alert episodes](experimental-alerting-system/alerts.md) to learn more. +### Signals + +In Signal mode, a match is recorded as a signal, which skips action policy evaluation entirely. As signals accumulate, you can query them in Discover, build dashboards from them, or feed them into an Alert mode rule that correlates activity across sources, feeding back into the start of the flow. + ### Action policies -An action policy is the gating layer between an alert episode and a workflow. It decides whether and when to invoke a workflow by evaluating episode eligibility, match conditions, and frequency. Policy configuration determines the scope. A policy can apply to alert episodes from a specific rule, multiple rules, or all rules in the space. +An action policy is the gating layer between an alert episode and a workflow. It decides whether and when to invoke a workflow by evaluating episode eligibility, match conditions, and frequency. A policy's configuration determines its scope, so one policy can cover alert episodes from a specific rule, multiple rules, or all rules in the space. This means you can change notification routing without touching any rule. Refer to [Notifications and actions](experimental-alerting-system/notifications-actions.md) to learn more. ### Workflows -A workflow is what actually sends the message or runs the automation, for example, posting to Slack, sending an email, calling a webhook. The {{alerting-v2-system}} invokes workflows in two ways: action policies that you configure to route alert episodes to a workflow based on match conditions and frequency, or alert episode lifecycle triggers that invoke a workflow immediately in response to a specific episode event, such as when it's activated or assigned. +A workflow is what actually sends the notification or runs the automation, for example, posting to Slack, sending an email, calling a webhook. An action policy can invoke it, or a lifecycle trigger can invoke it immediately when the episode is activated or assigned. Refer to [Connect workflows](experimental-alerting-system/workflows-alerting.md) to learn more. ## How the pieces fit together [how-pieces-fit-together] -At the simplest level: +Together, these building blocks form two main paths, which diverge based on a rule's mode: + +1. A rule evaluates your data and detects a match. +2. Depending on the rule's mode, the rule acts on the match (Alert mode) or records it (Signal mode): -1. A rule checks your data on a schedule. -2. The rule's query returns results when data matching its conditions is found. -3. The rule's mode determines what happens next: - - Alert - The rule opens an alert episode to track the problem. An action policy can route it to a workflow to perform an action or send a notification. - - Signal - Each result is recorded for querying later. Nothing else happens. + - **Alert mode**: An alert episode is created. An action policy evaluates the episode and decides whether and when to invoke a workflow. + - **Signal mode**: The match is recorded as a signal, which skips action policy evaluation and workflow invocation entirely. -For a more detailed explanation of each stage, refer to [How the {{alerting-v2-system}} works](experimental-alerting-system/how-it-works.md). +:::{image} /explore-analyze/images/detailed-system-flow.png +:alt: Flowchart showing that after a rule finds a match, it either acts by creating an alert episode that an action policy evaluates and routes to trigger notifications or actions or records a signal that doesn't trigger notifications or actions +::: ## Get started or go deeper [system-overview-next-steps] diff --git a/explore-analyze/images/basic-system-flow.png b/explore-analyze/images/basic-system-flow.png new file mode 100644 index 0000000000..6958174a80 Binary files /dev/null and b/explore-analyze/images/basic-system-flow.png differ diff --git a/explore-analyze/images/detailed-system-flow.png b/explore-analyze/images/detailed-system-flow.png new file mode 100644 index 0000000000..d0c4ee3516 Binary files /dev/null and b/explore-analyze/images/detailed-system-flow.png differ