Repository navigation
Mask the loss weight alongside the error (use_f_weights is broken) - #47
Merged
Merged
Conversation
`tf.boolean_mask` flattens what it selects, so a force error of shape
[n_atoms, 3] comes back as [n_atoms*3]. The weight did not travel through the
same selection, so `error**2 * weight` tried to broadcast [n_atoms*3] against
[n_atoms, 3] and the graph failed to build:
Incompatible shapes: [579] vs. [193,3]
The force error is always masked -- `f_mask` is an all-true fill -- so this hit
every run with `use_f_weights: True`, i.e. the option has never worked since
boolean masking arrived in 1586487 (2020). Scalar weights broadcast fine, which
is why the energy path (mask None unless `max_energy` is set) never showed it.
Select the weight with the same mask when it is a tensor of the error's rank,
and add a regression test that trains and evaluates with per-component force
weights.
Roy-Kid
force-pushed
the
fix/masked-loss-weight
branch
from
September 16, 2026 13:59
c5d85b3 to
d94cdd9
Compare
zzy2014
approved these changes
Sep 17, 2026
zzy2014
left a comment
zzy2014
left a comment
Contributor
There was a problem hiding this comment.
Everything looks good to me.
2 of 4 tasks
Roy-Kid
added a commit
to Roy-Kid/PiNN
that referenced
this pull request
Sep 17, 2026
The docs deploy pushes to gh-pages, a single shared ref, so two master merges landing seconds apart deploy in parallel and the slower job's `mike deploy -p` is rejected as non-fast-forward. That is what broke run 35215109985: Teoroo-CMC#47 and Teoroo-CMC#46 merged 36 seconds apart, Teoroo-CMC#47 won the race, and Teoroo-CMC#46's docs fix never reached the site while everything else in the run was green. Waiting for a PR's docs deploy to finish before merging the next one avoids the race, so this does not serialize deploys automatically. It only opens the repair path: the workflow already accepted workflow_dispatch, but deploy-docs was gated on `github.event_name == 'push'`, so a manual run rebuilt and tested without ever redeploying the docs. Accepting workflow_dispatch there makes "Run workflow" on master actually replay the deploy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
use_f_weights: Truehas never been able to train: the graph fails to build withIncompatible shapes: [n_atoms*3] vs [n_atoms, 3].tf.boolean_maskflattens what it selects, so the force error of shape[n_atoms, 3]comes back as[n_atoms*3], whilefeatures['f_weights']keeps its[n_atoms, 3]shape — anderror**2 * weightcannot broadcast the two. The force error is always masked (f_maskis an all-true fill), so every run that turns the option on hits it. Scalar weights broadcast fine, which is why the energy path — wheremaskisNoneunlessmax_energyis set — never showed the problem, and why this survived since boolean masking arrived in 1586487 (2020-09-12).This is independent of #45; the bug reproduces identically on the commit before it.
Fix: select the weight with the same mask when it is a tensor of the error's rank, in both the TRAIN and EVAL branches. Scalar multipliers are passed through untouched.
Test:
test_use_f_weights_trainstrains and evaluates a potential with per-component force weights. It fails on master with the exact error above and passes with the fix.