[Security][CPS] Document detection rule behavior and correct app availability - #8064
[Security][CPS] Document detection rule behavior and correct app availability#8064nastasha-solomon wants to merge 16 commits into
Conversation
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
Co-authored-by: Cursor <cursoragent@cursor.com>
shainaraskas
left a comment
There was a problem hiding this comment.
hello thank you!
overall thoughts:
- I think it would be good if we could be clearer about when things use the space scope
- We can probably cut back/unify some of the info about API keys on rules - it reads as pretty cohesive between the two surfaces now and we should probably just document what is different about detection rules in this spot
| ::: | ||
|
|
||
| - **{{ml-cap}}:** {{anomaly-detect-cap}} job {{dfeeds}} and transforms can read data from linked projects. Jobs and results live on the origin project. {{ml-cap}} rules alert on those origin results, including anomalies produced from linked-project data. | ||
| - **Timeline:** Tables display documents from linked projects. Actions that don't apply to remote documents are disabled. |
There was a problem hiding this comment.
is this using the space scope?
There was a problem hiding this comment.
@PhilippeOberti @michaelolo24 does Timeline always use the space’s default CPS scope, or can it follow the CPS scope selector selection in the project's header? For reference, here's the latest draft.
| - **Dashboards:** The Detection & Response and Data Quality dashboards support {{cps-init}}. | ||
| - **Intelligence:** Threat intelligence indicator searches support {{cps-init}}. |
There was a problem hiding this comment.
@chuddy-elastic who can clarify whether the threat intel features always use the space’s default CPS scope, or can follow the CPS scope selector selection in the project's header? For reference, here's the latest draft.
Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
…y-security-apps.md Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
…y-security-apps.md Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
…y-security-apps.md Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
shainaraskas
left a comment
There was a problem hiding this comment.
looking nice! a couple of considerations for you about the table columns and some questions about rule scope.
the cutback on the API keys stuff for rules is looking especially nice 🎉
| - **Attack Discovery**: AI-generated attack discoveries are based on alerts from the origin project only. | ||
| - **Overview**: The Security Overview page reflects data from the origin project only. | ||
| - **Defend and Osquery**: Elastic Defend and Osquery are scoped to the origin project only. Defend and Osquery are managed through Fleet, meaning their configuration is tied to a single project. Endpoint artifacts, policies, response actions, and Osquery saved queries and packs are managed per project and are not shared across linked projects. No newline at end of file | ||
| | App | {{cps-init}} scope selector | Query-level overrides | |
There was a problem hiding this comment.
I think using the query-level overrides column for some of these details can lead to a little confusion. looking at the ML row specifically, the details in the query-level overrides are mostly about how CPS is supported in the feature. consider moving this to another column, or renaming the column to something like "Details" since only one of your features support query-level overrides (nor would we expect the others to).
the pattern observability uses is that it defers these details to a dedicated page. that's an option but perhaps too heavy
|
|
||
| **[{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md)** | ||
| : Relevant if you use {{cps}} to query data across linked {{serverless-short}} projects. Explains how detection rules use the space-level {{cps}} scope and how to use project routing to target specific projects. {{serverless-short}} only. | ||
| : Relevant if you use {{cps}} to query data across linked {{serverless-short}} projects. Explains which projects a rule searches, where its alerts are stored, how last-editor access affects linked-project searches, and how the alert limit applies across them. {{serverless-short}} only. |
There was a problem hiding this comment.
we're talking about "last-editor access" like it's a known entity but I think this might be [AI(?)] trying to come up with unnecessary shorthand - perhaps a little hard to parse for people who don't already understand. perhaps an unnecessary level of detail here as well - we don't have to name every single thing on the page
| @@ -19,7 +19,7 @@ Most users don't need these pages during initial setup. Review them if any of th | |||
| : Relevant if your data is spread across multiple {{es}} clusters and you need detection rules on one cluster to query indices on another. Covers establishing trust between clusters, remote cluster connections, and how to reference remote indices in rule index patterns. {{stack}} only. | |||
|
|
|||
| **[{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md)** | |||
There was a problem hiding this comment.
consider making the "elastic stack only" and "serverless only" notes badges on the "link" line
| **[{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md)** | |
| **[{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md)** {applies_to}`serverless: ga` {applies_to}`stack: unavailable` |
or just
| **[{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md)** | |
| **[{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md)** {applies_to}`serverless: ga` |
| @@ -1,15 +1,22 @@ | |||
| {{elastic-sec}} apps have partial {{cps-init}} support. The following features work across linked projects: | |||
| {{elastic-sec}} apps have partial {{cps-init}} support. The following table shows which apps support the {{cps-init}} scope selector and query-level overrides. | |||
There was a problem hiding this comment.
this sentence isn't necessarily describing the table contents right now
| If you create or update a rule through the API with an {{es}} API key, the rule keeps that credential and searches the origin project only: | ||
|
|
||
| - If the origin project has no matching indices, the rule doesn't run and its last-run status shows a warning. | ||
| - If those patterns exist on the origin, the rule still runs and reports success, but it searches the origin only. There's no warning for that. |
There was a problem hiding this comment.
nit: "there's no warning for that" reads a little casual to my eye
|
|
||
| The **Max alerts per run** [advanced setting](/solutions/security/detect-and-alert/common-rule-settings.md#rule-ui-advanced-params) limits the number of alerts a rule creates in a single execution. Under {{cps}}, that limit covers the combined results from the origin project and all linked projects in the space-level scope rather than each project separately. The default is 100. | ||
|
|
||
| A rule that stayed comfortably under the limit on a single project can reach it once you link projects, which leaves matches without alerts. Review the limit for rules that run across a broad scope, and consider [narrowing the scope with project routing](/explore-analyze/cross-project-search/cross-project-search-project-routing.md) instead of raising the limit. |
There was a problem hiding this comment.
"it" in "A rule that stayed comfortably under the limit on a single project can reach it" is not clear
narrowing the scope with project routing
how do you do this for rules? can you possibly connect this to where the person can input a query in rule creation (or is this very very obvious?)
| **Max alerts per run** (optional) | ||
| : Specify the maximum number of alerts the rule can create each time it executes. Default is 100. | ||
|
|
||
| {applies_to}`serverless: preview` When {{cps}} is enabled, this limit covers the combined results from the origin project and all linked projects in the space-level scope rather than each project separately. Refer to [{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md#cps-rules-max-alerts). |
There was a problem hiding this comment.
🤔 if we can limit the scope with project routing, is this exactly true? or is it the scope of the query, which is by default the same as the space-level scope unless there's a query-level override?
| @@ -1,10 +1,13 @@ | |||
|
|
|||
| When [{{cps}}](/explore-analyze/cross-project-search.md) is enabled and you have [linked projects](/deploy-manage/cross-project-search-config/cps-config-link-and-manage.md), rules query data across linked projects based on the **space-level {{cps}} scope**. | |||
| When [{{cps}}](/explore-analyze/cross-project-search.md) is enabled and you have [linked projects](/deploy-manage/cross-project-search-config/cps-config-link-and-manage.md), rules query data across linked projects based on the **space-level {{cps}} scope**. You can't set a {{cps}} scope on individual rules. | |||
There was a problem hiding this comment.
You can't set a cross-project search scope on individual rules.
this seems to clash with the paragraphs below, which say that you can override with SET project_routing, and use a qualified index expression to limit results (if not query scope). maybe easier to say "most rules can't blah blah" and treat esql and index pattern rules as exceptions?
| For prerequisites such as linking projects and configuring default scope, refer to [{{cps-cap}}](/explore-analyze/cross-project-search.md) and [Configure {{cps}} access and scope](/deploy-manage/cross-project-search-config/cps-config-access-and-scope.md). | ||
| For {{esql}} rules, you can use [`SET project_routing`](/explore-analyze/cross-project-search/cross-project-search-project-routing.md) in the rule query to target specific linked projects, overriding the space-level scope. For non-{{esql}} rules that use index patterns, you can use [qualified index expressions](/explore-analyze/cross-project-search/cross-project-search-search.md#search-expressions) to scope the rule to specific projects. | ||
|
|
||
| For prerequisites such as linking projects and configuring default scope, refer to [{{cps-cap}}](/explore-analyze/cross-project-search.md) and [Configure {{cps}} access and scope](/deploy-manage/cross-project-search-config/cps-config-access-and-scope.md). |
There was a problem hiding this comment.
no need to spell out the title of the page here
| For prerequisites such as linking projects and configuring default scope, refer to [{{cps-cap}}](/explore-analyze/cross-project-search.md) and [Configure {{cps}} access and scope](/deploy-manage/cross-project-search-config/cps-config-access-and-scope.md). | |
| For prerequisites such as linking projects and configuring default scope, refer to [](/explore-analyze/cross-project-search.md) and [](/deploy-manage/cross-project-search-config/cps-config-access-and-scope.md). |
| :::{note} | ||
| {{ml-cap}} rules don't support {{cps}}; they search data in the origin project only. Other features also have limited or no {{cps}} support. For details, refer to [{{cps-cap}} availability by app](/explore-analyze/cross-project-search/cross-project-search-manage-scope.md#cps-availability). | ||
| <!-- TODO: After https://github.com/elastic/docs-content/pull/7814 merges, restore the link on "can read linked-project data" to /explore-analyze/machine-learning/anomaly-detection/ml-ad-run-jobs.md#ml-ad-cps-scope. --> | ||
| {{ml-cap}} rules alert on {{anomaly-detect}} results stored on the origin project. {{anomaly-jobs-cap}} can read linked-project data; jobs and results stay on the origin. Other features also have limited or no {{cps}} support. For details, refer to [{{cps-cap}} availability by app](/explore-analyze/cross-project-search/cross-project-search-manage-scope.md). |
There was a problem hiding this comment.
is this note about "other features have no support" needed here?
would we consider pulling the ml rule into the main text, so we could shape it like
"most don't let you select scope, here are the exceptions:
- es|ql set project_routing
- index patterns qualified expressions
- ml rules are scoped at ad job level"
There was a problem hiding this comment.
this looks very similar to the stuff on create-manage-rules now ... should they all use a single snippet?
|
|
||
| If your data spans ECH, ECE, ECK, or self-managed clusters rather than linked {{serverless-short}} projects, refer to [{{ccs-cap}} and detection rules](/solutions/security/detect-and-alert/cross-cluster-search-detection-rules.md) instead. | ||
|
|
||
| ## Which projects a rule searches and where alerts are stored [cps-rules-scope-and-alerts] |
There was a problem hiding this comment.
is this stuff covered in the core alerting/rules docs?
Summary
Contributes to #8050.
Corrects the cross-project search (CPS) availability information for Elastic Security apps and documents rule-level CPS behavior ahead of GA.
Previews
CPS availability in Elastic Security apps - Corrected and expanded the availability list. (The same snippet also renders on Configure CPS.) Made the following changes to the section and moved the info into a table to align the format used for Kibana and Observability:
CPS and detection rules — Expands the detection-rules row. The intro snippet also renders on Observability Create and manage rules, and states space default, read-only selector, and query-level overrides inline:
Common rule settings - Max alerts per run notes that, with CPS enabled, the limit covers combined results from the origin and linked projects in the space-level scope, with a link to the full explanation.
Deliberately not included
The "Using cross-project search with Elastic Security" guidance page (item 2 of #8050) is not included as it's not a blocker to CPS GA'ing. It will be added in a separate PR.
Lifecycle note
CPS pages remain at
serverless: preview. Flipping them to GA is intentionally left to a separate PR. The one inline{applies_to}tag added here, on Max alerts per run, will need to flip with them.Generative AI disclosure