Repository navigation
Report swap growth and wallpaper refreshes in lane health, and find SpringBoard - #196
Merged
Merged
Conversation
…pringBoard Run 39862150429's iOS discovery lane lost its runner a minute into the flow: Maestro's connections timed out while the runner, the tap proxy and xcodebuild were all still running. In that minute PosterBoard ran a wallpaper gallery refresh, which started about ten wallpaper extensions of 100 to 200 MB each, and the host went from no swap to 536 MB. Passing discovery lanes in #193's and #192's runs logged no refresh. Lane health now reports how far swap grew during the flow and how many wallpaper gallery refreshes PosterBoard ran, with the first five refresh lines, so the next lane that stalls this way says so. Its SpringBoard spindump never ran on this runtime: the lookup expected no space in the process path, and the iOS 26.4 runtime lives under "iOS 26.4.simruntime". It matches the whole path now.
Pull request footprint
User-delivered artifact sizes
Most complex product function
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure
#195's run 39862150429 failed
native-e2e-ios-light (discovery)(job 120115822649) before any tap. Maestro'sdeviceInforaisedSocketTimeoutException: Connect timed out, followed byDevice became unreachable. The artifacts show:The runner and the tap proxy started and answered.
maestro-runner.loghasdeviceInfo: 200and thendeviceInfo through the tap proxy: 200at 22:11:30. Maestro's status check,deviceInfo,setPermissionsandterminateAppall reached the runner through the proxy between 22:11:56 and 22:11:57. So it is neither a start-up race nor the driver dying at start-up.maestro-runner.mjsreports ready only after both answer.After 22:12:02 no request reached the runner, but nothing had died. Maestro started
assertVisibleat 22:12:04.9, and its firstdeviceInfogave up at 22:12:49.9 after 45 s of connect timeouts. The runner's log has no connection after 22:12:02. The process list taken after the flow still shows the proxy's node (7593), its xcodebuild (7802), the runner (8433) and Cue (10576). The proxy logged no unreachable-runner error, so the requests never reached it either.The host started swapping hard at that moment.
Lane health logged 1,239 MB paged in during the flow. The proxy's node process sat at 14 MB resident, mostly paged out.
PosterBoard ran a wallpaper gallery refresh just before. At 22:12:01 it started
pushUpdatedPosterDescriptors("Push To Proactive"), and about ten wallpaper extensions came up: Gradient, Mercury, Kaleidoscope, Aegir, Unity, Rhizome, Weather, Pride, Extragalactic. Lane health's process census shows PosterBoard at 193 MB, MercuryPosterExtension at 161 MB, and so on. The wallpaper processes wrote 29,305 of the run's 57,910 simulator log lines. The discovery lanes in Deduplicate CI setup and drop stale configuration #193's run 39855874496 and Prune tautological, outmoded and duplicated tests #192's run 39850551247 logged no gallery refresh, and 24 and 284 wallpaper lines.I could not prove from the artifacts how the swap turned into connect timeouts, as opposed to slow replies. The timing and the 29,305 against 24 or 284 wallpaper lines point at the gallery refresh, which is simulator behaviour rather than ours.
Change
So the next lane that stalls this way says so in its health report:
sysctl vm.swapusagebefore and after) and how many wallpaper gallery refreshes PosterBoard ran, recording the first five refresh lines.iOS 26.4.simruntime. The lookup now matches the whole path.A possible mitigation for later is turning off PosterBoard's gallery refresh the way
ios-disabled-daemons.tsvturns off other daemons. I didn't do it here: SpringBoard depends on PosterBoard for the home and lock screen, and this one failure isn't evidence enough to justify the risk.Tests
sysctl;iOS 26.4.simruntimepath.Mutation check, each applied alone:
Footprint
No product code changes. CI and tooling:
scripts/lane-health.mjsgains about 20 lines. Configuration: one cspell word. Tests: about 30 lines.pnpm checkpasses on Node 22.23.4.