Skip to content

[BUG] Alarm group convergence: group wrongly flips to resolved and recovery events get silently dropped #4160

Open
#4316
#4168
Open
[BUG] Alarm group convergence: group wrongly flips to resolved and recovery events get silently dropped#4160
#4316
#4168
Labels
bugSomething isn't working

Description

@mingzeh
mingzeh
opened on Jun 18, 2026

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Bug 1 — The whole group is prematurely marked resolved while other alerts in the group are still firing.

  • Push 3: group contains CPU(firing) + Memory(firing), group status = firing.
  • Push 4: CPU recovers; the pushed GroupAlert reports status = resolved and alerts contains only CPU(resolved).
  • Push 5, immediately after: Memory(firing, triggerTimes=2) is still being pushed — proving Memory never recovered, so the resolved group push in Push 4 was wrong.

Bug 2 — A recovery (resolved) event is silently dropped; the DB record stays firing forever.

  • Push 7: group contains CPU(firing) + Memory(firing, triggerTimes=6).
  • Push 8: Memory has actually recovered, but the pushed GroupAlert contains only CPU(firing). The memory resolved record disappears entirely — it is never pushed, and the alert stays firing in the database permanently, with no endAt and no auto-resolve.

Expected Behavior

Bug 1: A group's status must reflect all currently-active alerts in that group, not just the alerts received within the current push window. While Memory is still firing, the group must stay firing; CPU recovering alone must not flip the whole group to resolved.
Bug 2: Every resolved transition must be persisted, with endAt set and status changed to resolved, and then pushed. The firing repeat-interval throttle should only suppress repeated firing notifications — it must never swallow a pending resolved event.

Steps To Reproduce

Reproducibility note:

These issues are timing-sensitive and are not 100% reproducible.

  • Bug 2 has a relatively higher reproduction rate and is easier to observe when a resolved transition happens during the firing repeat-interval window.
  • Bug 1 has a lower reproduction rate, but it has been observed in actual outbound GroupAlert payloads.

Therefore, the steps below describe the reproduced lifecycle and timing conditions rather than a strictly deterministic 100%-repro procedure.

Setup:

  1. One monitor, collection interval 10s, against a mock instance instance = 127.0.0.1:2222.

  2. Two threshold rules on the same instance:

    • CPU alert: cpu_usage > 80, severity: emergency, Trigger Times: 1
    • Memory alert: memory_usage > 50, severity: critical, Trigger Times: 1

  3. One group-convergence rule, aggregate by host:

    • Group Labels: instance
    • Group Wait: 5s
    • Group Interval: 20s
    • Repeat Interval: 60s

    Both alerts collapse into the same group: groupKey = instance:127.0.0.1:2222.

Reproduce Bug 1:

  1. Drive CPU over threshold so CPU fires and is pushed, group firing.
  2. While CPU is still firing, drive Memory over threshold too, so the group now has 2 firing alerts: CPU + Memory.
  3. Let CPU recover while Memory is still firing.
  4. ❌ Observe the group push reports status = resolved with only CPU(resolved) — even though Memory is still firing, confirmed by the very next push showing Memory(firing).

Reproduce Bug 2:

  1. With CPU firing again and Memory firing, triggerTimes=6, let Memory recover during the firing repeat interval, i.e. the interval window has elapsed but less than repeatInterval has passed since the last repeat.
  2. ❌ Observe the next group push contains only CPU(firing); the Memory resolved event is never pushed and the memory alert remains firing in the DB indefinitely.

Environment

HertzBeat version(s): v1.8.0

Debug logs

Data was captured 1:1 from the actual outbound GroupAlert payloads via a custom PostAlertPlugin. The plugin just serializes each callback GroupAlert to JSON:

public class DemoPlugin implements PostAlertPlugin {
    @Override
    public void execute(GroupAlert alert, PluginContext pluginContext) {
        log.info("DemoPlugin alert: {}", JsonUtil.toJson(alert));
    }
}

Lifecycle of one group: instance:127.0.0.1:2222; tt = triggerTimes.

Push Time group status alerts in group Note
1 11:06:54 firing CPU(firing, tt=1) CPU first fires
2 11:07:54 firing CPU(firing, tt=2) repeat, 60s
3 11:08:54 firing CPU(firing, tt=3) + Memory(firing, tt=1) 2 firing alerts in group
4 11:09:43 resolved ❌ CPU(resolved) Bug 1 – whole group wrongly resolved
5 11:10:04 firing Memory(firing, tt=2) disproves Push 4
6 11:13:04 firing Memory(firing, tt=5) memory still firing
7 11:14:04 firing CPU(firing, tt=1 new) + Memory(firing, tt=6) CPU fires again
8 11:15:04 firing CPU(firing, tt=2) Bug 2 – Memory resolved vanishes
9 11:20:33 resolved CPU(resolved) normal end

Bug 1 evidence:

Push 3  status=firing    alerts=[CPU firing, Memory firing]
Push 4  status=resolved  alerts=[CPU resolved]        ← Memory dropped, group wrongly resolved
Push 5  status=firing    alerts=[Memory firing tt=2]  ← Memory was firing all along

Bug 2 evidence:

Push 7  status=firing  alerts=[CPU firing, Memory firing tt=6]
Push 8  status=firing  alerts=[CPU firing tt=2]       ← Memory resolved silently lost, DB stays firing

DemoPlugin Log

2026-06-18 11:06:54.123 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:06:54","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 98.0","status":"firing","triggerTimes":1,"startAt":1781752003603,"activeAt":1781752013600,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:06:54"}]}
2026-06-18 11:06:54.123 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:07:54.082 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:07:54","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 98.0","status":"firing","triggerTimes":2,"startAt":1781752003603,"activeAt":1781752063601,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:07:54"}]}
2026-06-18 11:07:54.083 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:08:54.115 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:08:54","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 98.0","status":"firing","triggerTimes":3,"startAt":1781752003603,"activeAt":1781752133600,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:08:54"},{"id":53965,"fingerprint":"alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical","labels":{"severity":"critical","alert-test":"hello","defineid":"17","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"内存告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server 检测出内存使用率过高，当前使用率 78.0029","status":"firing","triggerTimes":1,"startAt":1781752123640,"activeAt":1781752133641,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:08:54","gmtUpdate":"2026-06-18 11:08:54"}]}
2026-06-18 11:08:54.116 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:09:43.639 [alerter-reduce-worker-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"resolved","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:09:43","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 98.0","status":"resolved","triggerTimes":3,"startAt":1781752003603,"activeAt":1781752133600,"endAt":1781752183602,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:09:43"}]}
2026-06-18 11:09:43.639 [alerter-reduce-worker-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:10:04.144 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:10:04","alerts":[{"id":53965,"fingerprint":"alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical","labels":{"severity":"critical","alert-test":"hello","defineid":"17","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"内存告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server 检测出内存使用率过高，当前使用率 78.0029","status":"firing","triggerTimes":2,"startAt":1781752123640,"activeAt":1781752203640,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:08:54","gmtUpdate":"2026-06-18 11:10:04"}]}
2026-06-18 11:10:04.144 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:11:04.103 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:11:04","alerts":[{"id":53965,"fingerprint":"alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical","labels":{"severity":"critical","alert-test":"hello","defineid":"17","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"内存告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server 检测出内存使用率过高，当前使用率 78.0029","status":"firing","triggerTimes":3,"startAt":1781752123640,"activeAt":1781752253640,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:08:54","gmtUpdate":"2026-06-18 11:11:04"}]}
2026-06-18 11:11:04.104 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:12:04.114 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:12:04","alerts":[{"id":53965,"fingerprint":"alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical","labels":{"severity":"critical","alert-test":"hello","defineid":"17","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"内存告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server 检测出内存使用率过高，当前使用率 78.0029","status":"firing","triggerTimes":4,"startAt":1781752123640,"activeAt":1781752323640,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:08:54","gmtUpdate":"2026-06-18 11:12:04"}]}
2026-06-18 11:12:04.115 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:13:04.107 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:13:04","alerts":[{"id":53965,"fingerprint":"alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical","labels":{"severity":"critical","alert-test":"hello","defineid":"17","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"内存告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server 检测出内存使用率过高，当前使用率 78.0029","status":"firing","triggerTimes":5,"startAt":1781752123640,"activeAt":1781752383639,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:08:54","gmtUpdate":"2026-06-18 11:13:04"}]}
2026-06-18 11:13:04.108 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:14:04.111 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:14:04","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"firing","triggerTimes":1,"startAt":1781752433599,"activeAt":1781752443599,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:14:04"},{"id":53965,"fingerprint":"alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical","labels":{"severity":"critical","alert-test":"hello","defineid":"17","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"内存告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server 检测出内存使用率过高，当前使用率 78.0029","status":"firing","triggerTimes":6,"startAt":1781752123640,"activeAt":1781752443640,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:08:54","gmtUpdate":"2026-06-18 11:14:04"}]}
2026-06-18 11:14:04.112 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:15:04.107 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:15:04","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"firing","triggerTimes":2,"startAt":1781752433599,"activeAt":1781752503600,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:15:04"}]}
2026-06-18 11:15:04.108 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:16:04.107 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:16:04","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"firing","triggerTimes":3,"startAt":1781752433599,"activeAt":1781752563599,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:16:04"}]}
2026-06-18 11:16:04.107 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:17:04.148 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:17:04","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"firing","triggerTimes":4,"startAt":1781752433599,"activeAt":1781752623599,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:17:04"}]}
2026-06-18 11:17:04.148 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:18:04.084 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:18:04","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"firing","triggerTimes":5,"startAt":1781752433599,"activeAt":1781752683600,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:18:04"}]}
2026-06-18 11:18:04.085 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:19:04.098 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:19:04","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"firing","triggerTimes":6,"startAt":1781752433599,"activeAt":1781752743601,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:19:04"}]}
2026-06-18 11:19:04.099 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:20:04.117 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"firing","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:20:04","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"firing","triggerTimes":7,"startAt":1781752433599,"activeAt":1781752803599,"endAt":null,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:20:04"}]}
2026-06-18 11:20:04.117 [alarm-group-calculate-0] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}
2026-06-18 11:20:33.615 [alerter-reduce-worker-1] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:19  - DemoPlugin alert: {"id":53842,"groupKey":"instance:127.0.0.1:2222","status":"resolved","groupLabels":{"instance":"127.0.0.1:2222"},"commonLabels":{"alert-test":"hello","instancename":"Mock Server","instance":"127.0.0.1:2222"},"commonAnnotations":{},"alertFingerprints":["alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","alert-test:hello,alertname:内存告警,defineid:17,instance:127.0.0.1:2222,instancename:Mock Server,severity:critical"],"creator":null,"modifier":null,"gmtCreate":"2026-06-08 17:00:32","gmtUpdate":"2026-06-18 11:20:33","alerts":[{"id":53964,"fingerprint":"alert-test:hello,alertname:CPU告警,defineid:18,instance:127.0.0.1:2222,instancename:Mock Server,severity:emergency","labels":{"severity":"emergency","alert-test":"hello","defineid":"18","instancename":"Mock Server","instance":"127.0.0.1:2222","alertname":"CPU告警"},"annotations":{"annotation":"hello","test-annotation":"world"},"content":"Mock Server CPU使用率过高，当前值 88.0","status":"resolved","triggerTimes":7,"startAt":1781752433599,"activeAt":1781752803599,"endAt":1781752833602,"creator":null,"modifier":null,"gmtCreate":"2026-06-18 11:06:54","gmtUpdate":"2026-06-18 11:20:33"}]}
2026-06-18 11:20:33.616 [alerter-reduce-worker-1] INFO  org.apache.hertzbeat.plugin.impl.DemoPlugin Line:20  - DemoPlugin pluginContext: {"params":null,"paramHolder":null}

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Apache HertzBeat
    Status
    To do

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions