Parallel tests#2190
Conversation
Johan-Liebert1
added
the
ci/merge
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the tmt test runner to support parallel execution of test plans using std::thread. It introduces a RunPlanResult struct and extracts the core test logic into a run_plan function. The review feedback identifies several opportunities for improvement, including addressing interleaved console output from concurrent threads, optimizing the thread-joining logic to avoid waiting for entire batches, handling potential panics when querying system parallelism, and removing redundant clones of owned objects.
| } | ||
| } | ||
|
|
||
| fn run_plan( |
There was a problem hiding this comment.
Running tests in parallel will cause their console output (from println!, eprintln!, and xshell commands) to interleave, making the logs difficult to read. Consider capturing the output per thread and printing it only when the test completes, or using a structured logging approach.
There was a problem hiding this comment.
Yeah this is an issue, but the final pass vs fail output is logged sequentially
Johan-Liebert1
force-pushed
the
parallel-tests
branch
from
122ad1a to
93f4abf
Compare
cgwalters
left a comment
cgwalters
left a comment
There was a problem hiding this comment.
Without any kind of deeper review, I think what we really want to do is upstream into https://github.com/teemtee/tmt support for bcvk.
I believe it already has support for concurrency (I mean I'd hope) and such - and that would make it a lot more sustainable for other projects to use tmt+bcvk (and we have many in the ecosystem that would make sense to do so)
| - uses: actions/checkout@v6 | ||
| - name: Bootc Ubuntu Setup | ||
| uses: bootc-dev/actions/bootc-ubuntu-setup@main | ||
| uses: Johan-Liebert1/bootc-actions/bootc-ubuntu-setup@main |
There was a problem hiding this comment.
I think we could add an option to the action which fetches bcvk's binary built from git main or so
There was a problem hiding this comment.
Yeah, that'd help
| plan: String, | ||
| vm_name: String, | ||
| image: String, |
Johan-Liebert1
force-pushed
the
parallel-tests
branch
from
93f4abf to
f2bbb18
Compare
|
One or two test failures, but not as good of a speedup as I'd had locally. GA seems to have 4cpu |
|
Yes, we can bump to larger runners though. I was experimenting with that previously in bootc-dev/ci-sandbox#1 but it's been a while |
Johan-Liebert1
added
ci/tier-1
Johan-Liebert1
force-pushed
the
parallel-tests
branch
3 times, most recently
from
5230a9f to
a98f50a
Compare
|
Alright, so composefs integration tests now take ~30-40 mins instead of ~1hr - 1hr30min Tests are still quite inconsistent tough. Especially the GC one randomly takes over 1000s to complete, dunno why. Needs some investigation |
|
The ostree one takes around ~1hr instead of ~2h15min Some tests are randomly taking a bit too long though I'll try to debug this |
Johan-Liebert1
added
ci/merge
Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Signed-off-by: Johan-Liebert1 <pragyanpoudyal41999@gmail.com>
We share host container storage in the VM, so we usually already have localhost/bootc as an image in the VM. If we don't find it, only then do we copy from ostree/composefs storage Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
So we don't spawn VMs for tests like "composefs-gc" just to do nothing and exit Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Sort tests in descending order of time taken for completion so longer tests get scheduled together. Also, update to use mpsc channels for communication between threads Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Johan-Liebert1
force-pushed
the
parallel-tests
branch
from
a98f50a to
cedbd85
Compare
2 participants
No description provided.