Skip to content

Allow filtering Alerts by event reason, not just message #1359

Description

@segevfiner

Description

Alert.spec.inclusionList and Alert.spec.exclusionList are documented as filtering on "message content," and in practice that's exactly what they do — both are regex lists matched only against event.Message:

// internal/server/event_handlers.go
if !s.messageIsIncluded(ctx, event.Message, alert) {
    continue
}
if s.messageIsExcluded(ctx, event.Message, alert) {
    continue
}

There's no equivalent for event.Reason (e.g. ReconciliationSucceeded, Progressing, ProgressingWithRetry, HealthCheckFailed, etc.). Today the only way to filter on reason is to write a regex against the free-text message and hope the reason string happens to appear in it verbatim — which is fragile and breaks silently if a controller's message wording changes.

This has come up before in #45 (2020) and #418 (2022), both of which were closed without adding a dedicated, structured way to filter on anything other than the message text.

Use case

I want an Alert that, for example:

  • Only forwards events where reason == HealthCheckFailed (ignore Progressing/ProgressingWithRetry noise), or
  • Excludes events where reason is in a given set, regardless of what the message text says.
    Right now this requires guessing a regex against message and re-checking it every time a controller's message format changes.

Proposed solution

Add a first-class reason filter to AlertSpec, analogous to inclusionList/exclusionList, e.g.:

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: only-failures
spec:
  providerRef:
    name: slack-bot
  eventSources:
    - kind: Kustomization
      name: '*'
  reasonInclusionList:
    - "HealthCheckFailed"
    - "ReconciliationFailed"
  reasonExclusionList:
    - "Progressing.*"

Semantics could mirror the existing message-based lists (regex, OR-matched within a list, exclusion takes precedence over inclusion), just evaluated against event.Reason instead of event.Message. Alternatively, inclusionList/exclusionList could be extended to match against reason or message, matching what several third-party guides already (incorrectly) claim is current behavior.

Alternatives considered

  • Keep encoding reason-matching in message regexes — fragile, undocumented coupling to message wording.
  • Use eventMetadata/webhook payload filtering downstream (e.g. in Alertmanager or a generic receiver) — works, but pushes filtering logic out of Flux and requires an extra hop for anyone using the built-in providers (Slack, Teams, etc.).

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions