Skip to content

Split CI into multiple workflows and use more granular steps - #489

Merged
fghoussen merged 3 commits into
opencollab:masterfrom
mmuetzel:ci
Aug 24, 2026
Merged

Split CI into multiple workflows and use more granular steps#489
fghoussen merged 3 commits into
opencollab:masterfrom
mmuetzel:ci

Conversation

@mmuetzel

Copy link
Copy Markdown
Contributor

Pull request purpose

GitHub allows temporarily disabling workflows (e.g., in forks of the repository). It doesn't have built-in granularity to disable jobs inside the same workflow though.
To do that currently, a user could, e.g., remove jobs manually and then rebase their actual changes on a branch where the jobs are still intact.

Additionally, GitHub truncates the output for long steps in its web interface.

Detailed changes proposed in this pull request

Simplify disabling parts of the CI temporarily by splitting the workflow files into three (one per platform).

Additionally, use more granular steps to make it easier to catch where the runs potentially fail. This also reduces the risk of the output being truncated by GitHub's web interface.

There is no difference in coverage by the CI with these changes. The exact same configurations as before still exists after this change. They are only slightly re-organized to help with future changes.

@mergify

mergify Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@mmuetzel

Copy link
Copy Markdown
Contributor Author

Hm...
One of the runners failed for a workflow that I didn't even touch:
https://github.com/opencollab/arpack-ng/actions/runs/30544381285/job/90876712775?pr=489#step:3:140

Install packages
    ▷ Installing bash build-base ccache cmake gfortran eigen-dev lapack-dev 
    ERROR: Unable to read database: file format is invalid or inconsistent
    ERROR: Failed to open apk database: file format is invalid or inconsistent
    
    Error occurred at line 318:
      315 | 		echo '▷ Installing $pkgs'
      316 | 		apk add --update-cache $pkgs
      317 | 	SHELL
    > 318 | 	abin/"$INPUT_SHELL_NAME" --root /.setup.sh
      319 | fi
      320 | 
      321 | 
    Error: Error occurred at line 318: 	abin/"$INPUT_SHELL_NAME" --root /.setup.sh (see the job log for more information)
    Error: Process completed with exit code 1.

Maybe, a network error on the runner which caused it to download a corrupt file?

@fghoussen

Copy link
Copy Markdown
Collaborator

Nope not a network error... I re-triggered the job: repeated fail.
#483 was OK but CI fails ont the same job now on master after merge...

@mmuetzel: can you have a look at it? I guess the problem lies around the github action

@mmuetzel

mmuetzel commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Nope not a network error... I re-triggered the job: repeated fail. #483 was OK but CI fails ont the same job now on master after merge...

@mmuetzel: can you have a look at it? I guess the problem lies around the github action

I'm not sure why that suddenly started failing. I'd guess that it is an issue with the packages database of Alpine Linux for that platform.

Thinking about that again, I'd probably do that entire workflow a bit differently today: Instead of using emulation to run a "native" compiler for each respective target platform, it would probably make more sense to use a cross-compiler to build without emulation for the respective target platforms. And then use qemu only to run the tests.
Also, GitHub started providing runners on 64-bit ARM hardware. So, that part of the workflow could run natively entirely.

I'll try to overhaul that part of the CI soonish, and then come back to this PR.

Marking this PR as a draft for the time being.

@mmuetzel
mmuetzel marked this pull request as draft August 4, 2026 06:57
@fghoussen

Copy link
Copy Markdown
Collaborator

it would probably make more sense to use a cross-compiler to build without emulation for the respective target platforms. And then use qemu only to run the tests.
Also, GitHub started providing runners on 64-bit ARM hardware. So, that part of the workflow could run natively entirely.

@sylvestre: what do you think?

GitHub allows temporarily disabling workflows (e.g., in forks of the
repository). It doesn't have built-in granularity to disable jobs inside
the same workflow though.
To do that currently, a user could, e.g., remove jobs manually and then
rebase their actual changes on a branch where the jobs are still intact.

Simplify that workflow by splitting the workflow files into three (one
per platform).
Additionally, use more granular steps to make it easier to catch where the
runs potentially fail. That also might help with GitHub truncating the output
for long steps.
Additionally, use more granular steps to make it easier to catch where the
runs potentially fail. That also might help with GitHub truncating the output
for long steps.
@mmuetzel

Copy link
Copy Markdown
Contributor Author

Thanks for merging #491.
I rebased this PR on top of a current head of the master branch.

Marking as ready for review again.

@mmuetzel
mmuetzel marked this pull request as ready for review August 18, 2026 10:41
@sylvestre

Copy link
Copy Markdown
Contributor

Sounds good

@fghoussen
fghoussen merged commit 947a004 into opencollab:master Aug 24, 2026
24 checks passed
@fghoussen

Copy link
Copy Markdown
Collaborator

@mmuetzel: thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants