feat: make determinism available even when in parallel mode - #33
feat: make determinism available even when in parallel mode#33MilesCranmer wants to merge 4 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if controller.should_stop(state.pools.total_evals) { | ||
| stop_dispatching = true; | ||
| controller.cancel(); | ||
| break; |
There was a problem hiding this comment.
In deterministic mode, completed results are buffered until their turn in release_in_task_order, so apply_task_result (which increments pools.total_evals) can be delayed even when later tasks have already finished. The stop check here (controller.should_stop(state.pools.total_evals)) therefore uses a stale eval count and can keep dispatching additional tasks, exceeding the configured evaluation budget by multiple populations if an early task is slow. This regression is introduced by the new ordered-commit logic and shows up specifically when deterministic+parallel scheduling is used.
Useful? React with 👍 / 👎.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
dynamic_expressions / eval
Details
| Benchmark suite | Current: f90d486 | Previous: 18944d1 | Ratio |
|---|---|---|---|
evaluation/BuiltinF32/derivative |
17.594068 ms/iter (± 0.310106) |
17.535021 ms/iter (± 0.287446) |
1.00 |
evaluation/BuiltinF32/eval |
3.040477 ms/iter (± 0.02865) |
3.038753 ms/iter (± 0.030606) |
1.00 |
evaluation/BuiltinF64/derivative |
23.859484 ms/iter (± 0.082799) |
23.871291 ms/iter (± 0.050008) |
1.00 |
evaluation/BuiltinF64/eval |
4.102817 ms/iter (± 0.01151) |
4.057799 ms/iter (± 0.00922) |
1.01 |
evaluation/Float32/derivative |
11.432367 ms/iter (± 0.025123) |
11.425006 ms/iter (± 0.018174) |
1.00 |
evaluation/Float32/eval |
2.373496 ms/iter (± 0.033477) |
2.37269 ms/iter (± 0.046834) |
1.00 |
evaluation/Float64/derivative |
27.507251 ms/iter (± 0.05643) |
27.517805 ms/iter (± 0.099308) |
1.00 |
evaluation/Float64/eval |
5.238016 ms/iter (± 0.016965) |
5.260476 ms/iter (± 0.01249) |
1.00 |
utilities/Float32/clone |
7.316 us/iter (± 0.037) |
7.214 us/iter (± 0.016) |
1.01 |
utilities/Float32/combine_operators |
61.889 us/iter (± 0.154) |
61.606 us/iter (± 0.249) |
1.00 |
utilities/Float32/compress_constants |
6.84 us/iter (± 0.022) |
6.943 us/iter (± 0.027) |
0.99 |
utilities/Float32/count_depth_sizes/128 |
44.125 us/iter (± 6.307) |
51.015 us/iter (± 2.143) |
0.86 |
utilities/Float32/count_depth_sizes/20 |
4.38 us/iter (± 0.016) |
4.349 us/iter (± 0.015) |
1.01 |
utilities/Float32/count_depth_sizes/64 |
13.115 us/iter (± 0.04) |
13.075 us/iter (± 0.085) |
1.00 |
utilities/Float32/count_depth_sizes/8 |
2.021 us/iter (± 0.004) |
1.992 us/iter (± 0.024) |
1.01 |
utilities/Float32/counting |
4.178 us/iter (± 0.021) |
4.142 us/iter (± 0.077) |
1.01 |
utilities/Float32/simplify |
99.276 us/iter (± 0.491) |
101.068 us/iter (± 0.498) |
0.98 |
utilities/Float64/clone |
7.819 us/iter (± 0.016) |
7.807 us/iter (± 0.024) |
1.00 |
utilities/Float64/combine_operators |
61.538 us/iter (± 0.086) |
61.596 us/iter (± 0.105) |
1.00 |
utilities/Float64/compress_constants |
6.564 us/iter (± 0.025) |
6.419 us/iter (± 0.034) |
1.02 |
utilities/Float64/count_depth_sizes/128 |
52.476 us/iter (± 0.391) |
50.683 us/iter (± 0.304) |
1.04 |
utilities/Float64/count_depth_sizes/20 |
4.381 us/iter (± 0.014) |
4.372 us/iter (± 0.051) |
1.00 |
utilities/Float64/count_depth_sizes/64 |
13.108 us/iter (± 0.077) |
13.092 us/iter (± 0.079) |
1.00 |
utilities/Float64/count_depth_sizes/8 |
2.02 us/iter (± 0.058) |
1.986 us/iter (± 0.008) |
1.02 |
utilities/Float64/counting |
4.2 us/iter (± 0.013) |
4.077 us/iter (± 0.032) |
1.03 |
utilities/Float64/simplify |
100.927 us/iter (± 0.408) |
102.535 us/iter (± 0.304) |
0.98 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
symbolic_regression / optim
Details
| Benchmark suite | Current: f90d486 | Previous: 18944d1 | Ratio |
|---|---|---|---|
search/equation_search |
1.793755429 s/iter (± 0.100934861) |
2.02475953 s/iter (± 0.094823103) |
0.89 |
utils/best_of_sample |
377 ns/iter (± 1) |
374 ns/iter (± 1) |
1.01 |
utils/check_constraints_x10 |
6.648 us/iter (± 0.008) |
6.573 us/iter (± 0.108) |
1.01 |
utils/compute_complexity_x10/u16 |
0 ns/iter (± 0) |
0 ns/iter (± 0) |
1 |
utils/insert_random_op_x10 |
3.588 us/iter (± 0.158) |
3.489 us/iter (± 0.492) |
1.03 |
utils/next_generation_x100 |
194.228 us/iter (± 1.665) |
195.651 us/iter (± 1.384) |
0.99 |
utils/optimize_constants_x10 |
16.667524 ms/iter (± 0.698121) |
16.559171 ms/iter (± 0.635116) |
1.01 |
utils/randomly_rotate_tree_x10 |
7.7 us/iter (± 0.093) |
7.646 us/iter (± 0.055) |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
1 participant
No description provided.