lim.test TestInfo times out matching dinfo.systemAdapter.status.ports.ifname even though EVE publishes valid systemAdapter info

[eriknordmark]

Summary

tests/lim/testdata/info_test.txt (the info_test step in the smoke suite) times out at the 5-minute -timewait waiting for the pattern

InfoContent.dinfo.systemAdapter.status.ports.ifname:.*eth[01].*

…even though the device is healthy, posting info messages, and those messages contain exactly that data.

This test passed in <5 s on April 22 against the same eve-api/go proto version (v0.0.0-20260114160322-5feda2767927 in eden, v0.0.0-20260420100351-253db6a73b76 in pillar). It now hangs and times out against the same proto versions, in a fresh smoke run.

Repro

Standard smoke run against a COVER=y EVE built from lf-edge/eve master + the test-coverage PRs (#5901, #5914, #5915, #5922, #5923, #5925), with eden master + #1152, #1162, #1163 (ctrl_encrypt_cert_change), #1164, #1165 (nim-eden-tests), #1161 merged. (None of these touch the systemAdapter info-publish path.)

Adam: locally-built lfedge/adam:0.0.65-pr152 (lf-edge/adam#152 merged on top of master).

Diagnostics

EVE-side: 7 info POSTs to adam during the 5-min test window. 26 of those across the run are ZiDevice messages, every one of them contains dinfo.systemAdapter.

Decoded JSON of a stored ZiDevice info entry from redis (xrange INFO_EVE_<uuid>):

"systemAdapter": {
  "status": [
    {
      "version": 1,
      "key": "lastresort",
      "lastSucceeded": "2026-05-10T07:48:27...",
      "ports": [
        { "ifname": "eth0", "name": "eth0", "isMgmt": true, ... },
        { "ifname": "eth1", "name": "eth1", "isMgmt": true, ... }
      ]
    }
  ]
}

So dinfo.systemAdapter.status[0].ports[0].ifname == "eth0" and ports[1].ifname == "eth1" — exactly what the test pattern asks for.

lim.test log:

=== RUN   TestInfo
lim_test.go:274: time: 2026-05-10T09:53:44+02:00 out: Wait for info of ... number=1 timewait=5m0s
testContext.go:291: WaitForProc terminated by timeout 5m0s
testContext.go:274: WaitForProcWithErrorCallback terminated by timeout 5m0s
--- FAIL: TestInfo (300.00s)

Hypotheses

• The proto-path resolver in lim.test (or one of its einfo helpers) treats systemAdapter as a repeated message (i.e. expects dinfo.systemAdapter[].status...) when the proto actually wraps it as a singular message containing a repeated status field. The wrapper would explain why the path no longer resolves even though the data is present.
• Alternatively, a recent change to einfo.InfoNew (or wherever the live-stream watcher demultiplexes incoming entries from adam's redis stream) may be skipping ZiDevice messages or applying a stricter type check.

Both other lim.test tests in smoke (log_test, ssh, eve_restart) passed normally, so basic adam ↔ lim.test ↔ controller wiring is fine — the regression appears specific to dinfo.systemAdapter path resolution / matching.

Workaround

For coverage runs, the failure is cosmetic: the rest of smoke and downstream suites continue without it. Treating it as a known flake until it's diagnosed.

Asks

1. Identify whether the regression lives in the proto-path resolver or in einfo.InfoNew (or in the walker that decides which paths inside systemAdapter are addressable).
2. Add a unit test for the resolver against a recorded ZiDevice fixture so future regressions on this path don't bisect-bisect through 5-minute timeouts.

[eriknordmark]

Update — root cause is on the EVE side, not eden.

After deeper investigation the diagnosis in this issue is wrong. The eden path resolver and lim.test are not regressed; the proto-path resolver in pkg/utils/lookup.go correctly walks dinfo.systemAdapter.status.ports.ifname (mcuadros/go-lookup auto-iterates nested slices via aggregableLookup + mergeValue.AppendSlice).

The actual failure: the EVE under test never sends /info messages to the controller for an extended period because EVE never POSTs its EdgeNodeCerts (POST /api/v2/edgedevice/id/<uuid>/certs) — controller-side attestation is therefore stuck in a Z_ATTEST_RESPONSE_CODE_NO_CERT_FOUND retry loop. Once the deferred-event queue in controllerconn saturates with the failed publish, info publishing degrades and stops. lim.test TestInfo then times out on every query (including the simplest InfoContent.dinfo.machineArch:.+), regardless of whether the resolver path is correct, because no info messages are arriving in the test window.

Filed as lf-edge/eve#5936 with the full evidence and a suspected first-bad-commit list (the suspect master commits all landed 2026-05-05 → 2026-05-07 and all touch pkg/pillar/controllerconn/).

The query change earlier in the eden tree (tests/lim/testdata/info_test.txt switching from the deprecated dinfo.network.devName to dinfo.systemAdapter.status.ports.ifname in commit 7247e99) is unrelated and correct — it passed in 0.16 s in the April 22 baseline run with byte-identical eden code.

Closing this issue. The "asks" listed at the bottom of the original report (resolver investigation, fixture-based unit test for the lookup) are not invalidated — they're just not the cause of the observed test timeout. If you want a separate eden-side hardening PR for fixture-based resolver tests, please open a new issue.

[eriknordmark]

Final note: lf-edge/eve#5936, which I filed and pointed this issue at, has also been retracted and closed as wrong. The actual root cause of the original info_test timeout turned out to be unrelated to either eden or EVE master: the eden test session inherited a stale eden_adam_volume from a prior session that had baseos.base_os_version=12.1.0-kvm-amd64 set in adam's controller config. EVE silently downloaded 12.1.0 and rebooted into it ~5 min after onboard, and from that point every test ran against stock 12.1.0 instead of the COVER build.

There is no bug to track here. Closing the redirect chain.

Lesson learned and now codified in my local run-eden-test skill: always docker volume rm eden_adam_volume eden_redis_volume eden_eserver_volume eden_registry_volume as part of fresh-setup, and verify cat /run/eve-release matches the configured eve.tag after onboard and between every suite. docker rm -f eden_adam alone leaves the named volume behind.