Add ekore_capi by AkshatRai07 · Pull Request #537 · NNPDF/eko

AkshatRai07 · 2026-06-23T19:16:50Z

A step for #519 ; we explicitly shape our public API along the way #145

In this PR we:

Add the ekore_capi crate.
Expose anomalous_dimensions and operator_matrix_elements as C functions.
Change visibility of functions in ekore crate.

TODO

Create and merge a PR for:

Deploy of maturin wheels failed #538. Fix Maturin workflow #540
--release tag. Fix poe compile #541
No malloc in ekore. Remove mallocs and pass single buffer in run_op_integration #542
Single buffer passing throughout the integration. Remove mallocs and pass single buffer in run_op_integration #542
ekore visibility. Changing visibility of ekore and Splitting constants.rs #543
Split crates/ekore/src/constants.rs #539 Changing visibility of ekore and Splitting constants.rs #543
spacelike.rs tests. Add tests to spacelike.rs files #546
Bump Rust and add msrv.yml Add msrv.yml #545
Define version of a dependency in workspace and make crates point to that. Add centralised workspace dependencies #544

Only after all that, in this PR:

Copilot

Pull request overview

This PR introduces a new ekore_capi crate to expose ekore’s anomalous dimensions and operator matrix elements through a C ABI, and tightens internal visibility in ekore to keep implementation details private while preserving higher-level APIs.

Changes:

Added crates/ekore_capi with C-ABI entrypoints for unpolarized/polarized anomalous dimensions and unpolarized space-like OMEs, plus Cache allocation/free helpers.
Adjusted visibility in ekore (many pub → pub(super) / pub(crate), and pub mod → mod) to reduce exposed internals.
Added cbindgen/cargo-c metadata and updated the lockfile to include the new crate.

Reviewed changes

Copilot reviewed 29 out of 30 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
crates/ekore/src/operator_matrix_elements/unpolarized/spacelike/as2.rs	Restricts OME helper function visibility to module scope.
crates/ekore/src/operator_matrix_elements/unpolarized/spacelike/as1.rs	Restricts OME helper function visibility to module scope.
crates/ekore/src/operator_matrix_elements/unpolarized/spacelike.rs	Makes `as1`/`as2` internal modules.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gqg.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gps.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gnsv.rs	Narrows visibility of `gamma_nsv`.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gnsp.rs	Narrows visibility of `gamma_nsp`.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/gnsm.rs	Narrows visibility of `gamma_nsm`.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/ggq.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4/ggg.rs	Restricts N3LO component function visibility.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as4.rs	Removes public re-exports and makes singlet entry internal.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as3.rs	Restricts NNLO functions to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as2.rs	Narrows visibility of NLO functions and singlet helper pieces.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as1aem1.rs	Restricts mixed QCD×QED pieces to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/as1.rs	Narrows visibility of LO helper pieces.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/aem2.rs	Restricts NLO QED pieces to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike/aem1.rs	Restricts LO QED pieces to parent module.
crates/ekore/src/anomalous_dimensions/unpolarized/spacelike.rs	Makes most submodules internal; keeps `as1`/`as2` crate-visible.
crates/ekore/src/anomalous_dimensions/polarized/spacelike/as2.rs	Restricts polarized NLO functions to parent module.
crates/ekore/src/anomalous_dimensions/polarized/spacelike/as1.rs	Restricts polarized LO functions to parent module.
crates/ekore/src/anomalous_dimensions/polarized/spacelike.rs	Makes polarized `as1`/`as2` internal modules.
crates/ekore_capi/src/lib.rs	Defines C-facing `ComplexF64` and Cache lifecycle API; wires C API modules.
crates/ekore_capi/src/ad_us.rs	Adds C ABI wrappers for unpolarized space-like anomalous dimensions (QCD and QCD×QED).
crates/ekore_capi/src/ad_ps.rs	Adds C ABI wrappers for polarized space-like anomalous dimensions (QCD).
crates/ekore_capi/src/ome_us.rs	Adds C ABI wrappers for unpolarized space-like OME towers.
crates/ekore_capi/README.md	Documents crate purpose and citation policy.
crates/ekore_capi/cbindgen.toml	Configures C header generation.
crates/ekore_capi/Cargo.toml	Declares new crate, crate-type, and cargo-c metadata.
Cargo.lock	Adds `ekore_capi` to the workspace lockfile.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

AkshatRai07 · 2026-06-23T19:55:48Z

For change in visibility of ekore functions, I changed all of them to pub(super) and only the functions in the spacelike.rs files are now pub. Those who caused errors on being pub(super) (because they are called by functions outside their mod but inside the crate), their visibility was changed to pub(crate).

What I will do later:

Create C, C++, and maybe Fortran files for running tests.
Add ccompile, ctest, and other relevant tasks in pyproject.toml for poethepoet.
Add CI/CD files.

There are a few doubts for which I want your opinion:

For ekore:

What should be the visibility for functions in harmonics?

For ekore_capi:

Should we expose constants.rs?
Should we expose harmonics? If yes then which functions?
In ekore crate, tests are only for the ad_us, should I write tests for all spacelike.rs (as they are there on the python side), and hence write the same tests for capi?
How do we want to publish this? Do we want to push this crate to crates.io? In pineappl it is pointed out that cargo-c cannot fetch over the internet, and cargo install won't work. What I've been doing is:

cargo cinstall --release -p ekore_capi --destdir=./crates/ekore_capi/dist --prefix=/

We can then publish the dist as a GitHub release (as it is done in pineappl).

Should we write extensive doc comments, detailing the function's purpose and parameters?

My initial thought is to make only the harmonics::cache::Cache public, expose that through c_api. If required, we can change visibility of others later. We should also expose constants.rs, write more tests, publish the way pineappl does it, and add doc comments.

Other things I thought we could've done for #518, but can be done here:

Add a --release tag in poethepoet's compile task.
Remove all the mallocs inside the ekore and eko crate. For eg. replacing all the vec! with constant size arrays.
Pass a buffer through rust_quad_ker such that it doesn't malloc at every quadrature point, so one buffer for the entire integration loop.

Do let me know your thoughts.

AkshatRai07 · 2026-06-23T20:27:04Z

CoPilot reviews Summary:

There will be panic if we pass order_qcd as 0 (As you cannot initialize a 0 dimension vec). I don't think you'll ever pass that, there was no such guard in ekore, but still no problem in adding a guard here.
CoPilot is stating that there might be accidental panics (by say accidentally passing a wrong value), and these panics can cause Undefined Behaviour, so we should add safeguards. This is partially True. This behavior occurred for versions before 1.81. For 1.81+, it results in the crashing of code instead of undefined behaiour. We can either bump the 1.70 to 1.81 in our workspace Cargo.toml or add graceful handling.
CoPilot says to use checked_mul. This is just noise, there's no way order_qcd * 9 or something will get close to max for usize.

My initial thought is to add safeguards and not bump the Rust version. Please let me know your thoughts too.

AkshatRai07 · 2026-06-24T08:24:29Z

Small note, the data.nnpdf.science site is down, assets.sh is failing locally too, and that is why these tests are failing. Do you know the status of the site?

felixhekhorn · 2026-06-24T08:33:45Z

Small note, the data.nnpdf.science site is down, assets.sh is failing locally too, and that is why these tests are failing. Do you know the status of the site?

Indeed, I noticed. I hope the relevant people become aware quickly and are able to do something ...

AkshatRai07 · 2026-06-24T09:35:54Z

Should I fix #538 here or in a separate branch?

felixhekhorn

many comments generalize to other cases
I think it would be a good idea to split cf61e01 into a separate PR (since it affects lot's of files and does effectively nothing) and then put this one with just the C API on top

felixhekhorn · 2026-06-24T10:11:09Z

I wonder if we should apply some kind of meta programming ... because the function body of ad_ps_gamma_ns_qcd will look the same in all variations. The only thing that might change (and I'm not sure how that would manifest in the interface) is the maximum available order ...

We can explore this after most of the stuff is done.

Using macro_rules!, we can remove duplicate code. Do tell me if I should start working on that.

I would say yes

felixhekhorn · 2026-06-24T10:21:37Z

Other things I thought we could've done for #518, but can be done here:

* Add a `--release` tag in poethepoet's compile task.

* Remove all the mallocs inside the ekore and eko crate. For eg. replacing all the `vec!` with constant size arrays.

* Pass a buffer through rust_quad_ker such that it doesn't malloc at every quadrature point, so one buffer for the entire integration loop.

Do let me know your thoughts.

Should I fix #538 here or in a separate branch?

all these go into separate PRs; the --release tag and fixing #538 should have priority over this one I would say

felixhekhorn · 2026-06-24T10:37:18Z

For ekore:
* What should be the visibility for functions in harmonics?
For ekore_capi:
* Should we expose harmonics? If yes then which functions?
My initial thought is to make only the harmonics::cache::Cache public, expose that through c_api. If required, we can change visibility of others later.

I would say, let's follow our established logic, that we first expose as little as possible. In this case we have to expose Cache since it is an explicit parameter, but everything else should be hidden if possible. Since C and Rust should have the same surface, this implies making Rust more private. Since we think here for the first time seriously about visibility, I'm not worried about hiding stuff.

For ekore_capi:
* Should we expose constants.rs?

I think this requires some work, because again I want to expose as little as possible. We need to expose the PID as they are an explicit input, but I want to hide all the other things. I suggest to split the files into several parts corresponding to different math/physics and then we only expose what we need.

* In ekore crate, tests are only for the `ad_us`, should I write tests for all spacelike.rs (as they are there on the python side), and hence write the same tests for capi?

write more tests

yes, please. Please import all tests from the Python side which are not implemented yet. My idea is to use these same tests also in the CI to test the capo.

* How do we want to publish this? Do we want to push this crate to crates.io? In pineappl it is pointed out that cargo-c cannot fetch over the internet, and cargo install won't work. What I've been doing is:

cargo cinstall --release -p ekore_capi --destdir=./crates/ekore_capi/dist --prefix=/

We can then publish the dist as a GitHub release (as it is done in pineappl).

publish the way pineappl does it

yes, let's follow what other people do (and make our life easier)

* Should we write extensive doc comments, detailing the function's purpose and parameters?

and add doc comments

yes, we must. If we want to provide an API we must provide the documentation along side it else it is pointless

felixhekhorn · 2026-06-24T10:44:36Z

* CoPilot is stating that there might be accidental panics (by say accidentally passing a wrong value), and these panics can cause Undefined Behaviour, so we should add safeguards. This is partially True. This behavior occurred for versions before `1.81`. For `1.81+`, it results in the crashing of code instead of undefined behaiour. We can either bump the `1.70` to `1.81` in our workspace Cargo.toml or add graceful handling.

and not bump the Rust version

We can safely raise the MSRV I would say. Again, let's follow PineAPPL and they are currently at "raised MSRV to 1.91.0
raised Rust edition to 2024" - see Changelog . We also should add an MSRV workflow to our repo

CoPilot reviews Summary:

* There will be `panic` if we pass order_qcd as 0 (As you cannot initialize a 0 dimension vec). I don't think you'll ever pass that, there was no such guard in `ekore`, but still no problem in adding a guard here.

* CoPilot says to use `checked_mul`. This is just noise, there's no way `order_qcd * 9` or something will get close to max for `usize`.

My initial thought is to add safeguards

I agree - I think it is the rust way to add guards

felixhekhorn · 2026-06-24T10:50:56Z

What I will do later:
* Create C, C++, and maybe Fortran files for running tests.
* Add ccompile, ctest, and other relevant tasks in pyproject.toml for poethepoet.
* Add CI/CD files.

remember that Fortran has very low priority, but all the other stuff is crucial.

Now, looking at the flood of comments above I think this PR is a good start, but we should split some stuff from this PR and address problems that got revealed here first, before pushing further on this front (when we will surely discover more problems). In short, let's try to make a bunch of small PRs first instead of one big one

AkshatRai07 · 2026-06-24T11:40:31Z

let's try to make a bunch of small PRs first instead of one big one

Let me add a TODO as a comment here:

EDIT: moved to top

felixhekhorn · 2026-07-07T11:03:05Z

Is this relevant for us NNPDF/pineappl@823a4f7 ?

felixhekhorn

some of the comments obviously generalise to other cases, if you agree

felixhekhorn · 2026-07-09T08:17:12Z

+      - name: Get test data
+        id: cache-test-data
+        uses: actions/cache@v4
+        with:
+          path: test-data
+          key: test-data-v2
+      - name: Download test data
+        if: steps.cache-test-data.outputs.cache-hit != 'true'
+        run: |
+          ./tests/data/assets.sh


We don't need any data, no? so drop

felixhekhorn · 2026-07-09T08:21:01Z

+/// # Returns
+/// * Returns the fixed required buffer length of `3`.
+#[unsafe(no_mangle)]
+pub extern "C" fn ad_ps_gamma_ns_qcd_n3lo_len() -> usize {


I don't think this function makes sense: n3lo_variation in ad_ps_gamma_ns_qcd must be exactly 3 and no other and users must provide exactly those and they must know what it refers to - drop

felixhekhorn · 2026-07-09T08:22:02Z

+///
+/// # Safety
+/// * `c` must be a valid, non-null pointer to an initialized `Cache`.
+/// * `n3lo_variation` must be a valid, non-null pointer to a buffer of `u8` elements.


Suggested change

/// * `n3lo_variation` must be a valid, non-null pointer to a buffer of `u8` elements.

/// * `n3lo_variation` must be a valid, non-null pointer to a buffer of `u8` three elements.

felixhekhorn · 2026-07-09T08:23:08Z

+    c: *mut Cache,
+    nf: u8,
+    n3lo_variation: *const u8,
+    n3lo_len: usize,


drop as discussed above - the size is (rightfully) forced here

eko/crates/ekore_capi/src/ad_us.rs

Line 82 in 76ae5bf

let var: [u8; 3] = slice::from_raw_parts(n3lo_variation, 3).try_into().unwrap();

felixhekhorn · 2026-07-09T08:24:48Z

+/// Maximum QCD coupling power implemented.
+pub const MAX_ORDER_QCD: usize = 4;
+/// Maximum QED coupling power implemented.
+pub const MAX_ORDER_QED: usize = 2;
+
+/// singlet-like non-singlet |PID|.
+pub const PID_NSP: u16 = 10101;
+/// valence-like non-singlet |PID|.
+pub const PID_NSM: u16 = 10201;
+/// non-singlet all-valence |PID|.
+pub const PID_NSV: u16 = 10200;
+/// singlet-like non-singlet up-sector |PID|.
+pub const PID_NSP_U: u16 = 10102;
+/// singlet-like non-singlet down-sector |PID|.
+pub const PID_NSP_D: u16 = 10103;
+/// valence-like non-singlet up-sector |PID|.
+pub const PID_NSM_U: u16 = 10202;
+/// valence-like non-singlet down-sector |PID|.
+pub const PID_NSM_D: u16 = 10203;
+
+// Compile-time assertions to ensure C API constants remain synced with `ekore`
+const _: () = {
+    assert!(MAX_ORDER_QCD == ekore::constants::MAX_ORDER_QCD);
+    assert!(MAX_ORDER_QED == ekore::constants::MAX_ORDER_QED);
+    assert!(PID_NSP == ekore::constants::PID_NSP);
+    assert!(PID_NSM == ekore::constants::PID_NSM);
+    assert!(PID_NSV == ekore::constants::PID_NSV);
+    assert!(PID_NSP_U == ekore::constants::PID_NSP_U);
+    assert!(PID_NSP_D == ekore::constants::PID_NSP_D);
+    assert!(PID_NSM_U == ekore::constants::PID_NSM_U);
+    assert!(PID_NSM_D == ekore::constants::PID_NSM_D);
+};


why are they not just re-imported in the first place? no need to define them twice, no?

felixhekhorn · 2026-07-09T08:33:35Z

should we add a proper unit testing framework?

felixhekhorn · 2026-07-09T08:34:07Z

+#include <stdio.h>
+#include <stdlib.h>
+
+static int check_len(const char *label, size_t got, size_t expected)


if we don't go for a proper framework, surely this belongs into utils

felixhekhorn · 2026-07-09T08:37:38Z

this is (quasi) verbatim the same file as for .c, no? why would we do such a thing? I think C is enough and users who write C++ can use that just the same, I'd say

felixhekhorn · 2026-07-09T08:38:54Z

I would say yes

felixhekhorn · 2026-07-09T08:41:55Z

+      - name: Enforce tag-only execution
+        if: github.ref_type != 'tag'
+        run: |
+          echo "Error: Manual dispatches must be run against a tag, not a branch."
+          exit 1


This is inconvenient for debugging - can't why change it such that if it is a tag the files are uploaded there, if it is not, the files are just stored as artifacts to the job?

Copilot AI review requested due to automatic review settings June 23, 2026 19:16

Copilot started reviewing on behalf of AkshatRai07 June 23, 2026 19:17 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

felixhekhorn added rust Rust extension related gsoc26 labels Jun 24, 2026

felixhekhorn marked this pull request as draft June 24, 2026 08:34

felixhekhorn requested changes Jun 24, 2026

View reviewed changes

AkshatRai07 force-pushed the ekore-capi branch from a6fc224 to 1d0f98f Compare June 24, 2026 11:11

felixhekhorn reviewed Jun 24, 2026

View reviewed changes

Comment thread crates/dekoder/README.md Outdated

This was referenced Jun 24, 2026

Fix poe compile #541

Merged

Remove mallocs and pass single buffer in run_op_integration #542

Merged

Changing visibility of ekore and Splitting constants.rs #543

Merged

Add centralised workspace dependencies #544

Merged

Add msrv.yml #545

Merged

felixhekhorn mentioned this pull request Jun 30, 2026

GSoC 26 #515

Open

AkshatRai07 mentioned this pull request Jun 30, 2026

Add tests to spacelike.rs files #546

Merged

felixhekhorn closed this Jul 7, 2026

felixhekhorn reopened this Jul 7, 2026

ekore_capi initial commit

8336106

AkshatRai07 added 6 commits July 7, 2026 17:08

Added cbindgen

645bac9

Added ad and ome apis

3e869ae

Added ekore_capi to READMEs and release.json

a6c625a

Fixed ad_ps heading and ekore_capi link

facc5fe

Changed code to comply with Rust 2024

532444b

Changed deps to point to root Cargo.toml

bec1381

AkshatRai07 force-pushed the ekore-capi branch from c7e1819 to bec1381 Compare July 7, 2026 12:38

AkshatRai07 added 10 commits July 7, 2026 22:37

Expose Cache and constants

9eee043

Add Guards

0aa4b6c

Init capi tests

05942ce

Added CI for capi

31b3624

Fixed capi py ver

512983e

Added CD for capi

79c6e56

Add helpers and docs

f935a75

Changed to pub mod to show docs

9a56b65

Add utils to ctest

071a7fb

Added remaining tests

76ae5bf

felixhekhorn requested changes Jul 9, 2026

View reviewed changes

	/// * `n3lo_variation` must be a valid, non-null pointer to a buffer of `u8` elements.
	/// * `n3lo_variation` must be a valid, non-null pointer to a buffer of `u8` three elements.

Uh oh!

Conversation

AkshatRai07 commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AkshatRai07 commented Jun 23, 2026

Uh oh!

AkshatRai07 commented Jun 23, 2026

Uh oh!

AkshatRai07 commented Jun 24, 2026

Uh oh!

felixhekhorn commented Jun 24, 2026

Uh oh!

AkshatRai07 commented Jun 24, 2026

Uh oh!

felixhekhorn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

felixhekhorn commented Jun 24, 2026

Uh oh!

felixhekhorn commented Jun 24, 2026

Uh oh!

felixhekhorn commented Jun 24, 2026

Uh oh!

felixhekhorn commented Jun 24, 2026

Uh oh!

Uh oh!

AkshatRai07 commented Jun 24, 2026 • edited by felixhekhorn Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felixhekhorn commented Jul 7, 2026

Uh oh!

felixhekhorn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AkshatRai07 commented Jun 23, 2026 •

edited

Loading

AkshatRai07 commented Jun 24, 2026 •

edited by felixhekhorn

Loading