Detect hardware watchdog resets as a boot reason

[eriknordmark]

Motivation: opportunity for this improvement was discovered during kvm-to-k offline-resize watchdog stress testing — hardware-watchdog resets shouldbe surfaced as a boot reason if possible so that users (and tests) can distinguish a watchdog-forced reboot from other reboots.

Description

On a reboot for which EVE recorded no reason of its own, nodeagent guesses
the cause: if the storage controller's SMART power-cycle counter increased it
reports a dirty power-off (BootReasonPowerFail), otherwise it reports a
kernel panic (BootReasonKernel) or, when SMART is unavailable,
BootReasonUnknown. A device reset by its hardware watchdog (counter
unchanged) is therefore indistinguishable from a kernel bug.

This PR adds a BootReasonHWWatchdog and the signal needed to set it. The
watchdog container reads the watchdog boot status (WDIOC_GETBOOTSTATUS) once
at startup, before it arms the device, and records the set flag names to
/persist/hw_watchdog_bootstatus. When nodeagent reaches the
counter-unchanged / unknown branches and that file shows CARDRESET, it
reports BootReasonHWWatchdog instead of guessing a kernel panic.

BootReasonHWWatchdog.StartWithSavedConfig() returns false — like the
software watchdog reasons, a device returning from an unexplained hard hang
should wait for the controller rather than immediately restart saved
application config. This is an operator-visible behavior change: resets
that previously fell to BootReasonKernel/BootReasonUnknown and auto-resumed
apps will, on CARDRESET-reporting platforms, no longer auto-resume.

The flag is only reported by some watchdog drivers (e.g. AMD sp5100_tco,
many ARM SoC watchdogs); Intel iTCO always reports a zero boot status, so on
that hardware behavior is unchanged.

PR dependencies

• Depends on Add BOOT_REASON_HW_WATCHDOG and KUBE_TRANSITION eve-api#148 (adds BOOT_REASON_HW_WATCHDOG = 16 and
  BOOT_REASON_KUBE_TRANSITION = 15). Once that merges, a Bump eve-api
  commit will be added here to vendor the new enum value. Pillar already
  compiles without it (the reason maps to the proto via a numeric cast).

How to test and validate this PR

• Unit tests: BootReasonFromString / String / StartWithSavedConfig in
  pkg/pillar/types/zedagenttypes_test.go cover the new value.
• On hardware whose watchdog driver reports WDIOF_CARDRESET: trigger a
  watchdog reset, then confirm /persist/hw_watchdog_bootstatus contains
  CARDRESET and the device info message reports BootReasonHWWatchdog
  (rather than BootReasonKernel).
• The wdctl-before-arm read needs per-platform validation: opening
  /dev/watchdog can arm the timer on some drivers, so confirm the device is
  not left ticking before watchdog(8) starts petting it.

Changelog notes

A device reset by its hardware watchdog is now reported with the dedicated
boot reason "hardware watchdog" instead of being attributed to a kernel panic,
on platforms whose watchdog driver supports it.

PR Backports

• 16.0-stable: No — new enhancement, not a bug fix.
• 14.5-stable: No — new enhancement, not a bug fix.
• 13.4-stable: No — new enhancement, not a bug fix.

Checklist

• I've provided a proper description
• I've added the proper documentation
• I've tested my PR on amd64 device
• I've tested my PR on arm64 device
• I've written the test verification instructions
• I've set the proper labels to this PR

[codecov]

Codecov Report

❌ Patch coverage is 23.07692% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 21.09%. Comparing base (dc7f587) to head (ee96ff5).
⚠️ Report is 75 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/pillar/cmd/nodeagent/nodeagent.go	0.00%	20 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6068      +/-   ##
==========================================
+ Coverage   20.68%   21.09%   +0.40%     
==========================================
  Files         492      502      +10     
  Lines       91950    93674    +1724     
==========================================
+ Hits        19018    19758     +740     
- Misses      71321    72125     +804     
- Partials     1611     1791     +180     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.