CI table improvements - #5129
Open
ptomato wants to merge 3 commits into
Open
Conversation
ptomato
commented
Sep 3, 2026
ptomato
commented
Member
- Make the failed test emoji less big and red
- Drop irrelevant info from test artifacts
- Print test results in CI log
I've had some anecdotal feedback that the big red X still makes it look like it's a problem you have to solve before the PR can be merged. That's not the intention, so maybe a less big and red emoji makes that clearer.
These are the only keys used by tools/misc/build-results-comment.mjs. We can drop everything else (including the test file contents, which is very long because it includes the harness.) Also print the results to stdout before writing them to the file, so that the results are visible in the CI logs. Closes: tc39#5127
Member
Author
|
I guess we can't test the ➖ emoji on this PR because the privileged comment-writing workflow is always going to be run from main. |
Ms2ger
approved these changes
Sep 3, 2026
Ms2ger
left a comment
Ms2ger
left a comment
Contributor
There was a problem hiding this comment.
Consider using ➕ for passing tests for consistency
gibson042
approved these changes
Sep 3, 2026
| features: [Reflect,Intl.Locale,Intl.Locale-info] | ||
| ---*/ | ||
|
|
||
|
|
Member
Author
There was a problem hiding this comment.
This is just temporary, in order to get a test to run.
| function formatFail(message) { | ||
| if (!message) return '❌'; | ||
| return `[❌](## "${sanitizeTableMarkdown(message)}")`; | ||
| if (!message) return '➖'; |
Member
There was a problem hiding this comment.
I'd also like the emoji to be part of an obvious pair—(👍, 👎), (🙆, 🙅), (✅, ❎), etc.
| -- ${{ needs.changed-tests.outputs.all_changed_files }} \ | ||
| > "results-${{ matrix.engine.key }}.json" | ||
| | jq 'map({relative, scenario, result})' \ | ||
| | tee "results-${{ matrix.engine.key }}.json" |
Member
There was a problem hiding this comment.
We can do better than JSON in this log output, e.g.
Suggested change
| | tee "results-${{ matrix.engine.key }}.json" | |
| | tee "results-${{ matrix.engine.key }}.json" | |
| | jq -r ' | |
| map( | |
| "\n\(if .result.pass then "PASS" else "FAIL" end) \(.relative) [\(.scenario)]\(if .result.pass then "" else "\n\(.result.message)" end)" | |
| ) | |
| | .[] | |
| ' |
to produce text like
PASS path/to/foo [default]
FAIL path/to/bar [default]
failure message
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.