Skip to content

Protect weighted mean and weighted total by ensuring double precision - #246

Open
kanchukaitis wants to merge 1 commit into
JonKing93:mainfrom
kanchukaitis:fix/weighted-mean-single-double-cell2mat
Open

kanchukaitis wants to merge 1 commit into
JonKing93:mainfrom
kanchukaitis:fix/weighted-mean-single-double-cell2mat

Conversation

@kanchukaitis

Copy link
Copy Markdown
Collaborator

Summary

This PR addresses this issue: #242

stateVectorVariable.weightedMean and weightedTotal store the user's weights with their input class:

obj.weights{d} = weights{k}(:);

When those weights are single — e.g. cosd() of single-precision lat/lon metadata from a gridfile — the per-dimension obj.weights cell ends up mixing single (the weighted dimension) with double [] (the non-weighted dimensions). stateVectorVariable.serialize then stacks that cell with cell2mat (via stackVectors), which prior to R2025a throws:

All contents of the input cell array must be of the same data type.

So stateVector.build fails whenever a variable combines a spatial field with a pre-computed weighted (e.g. cosine-latitude hemispheric) mean and the coordinates/weights are single precision.

Fix

Cast the stored weights to double in both weightedMean.m and weightedTotal.m:

obj.weights{d} = double(weights{k}(:));

This guarantees serialize's cell2mat always sees one class (double), matching the double [] of the non-weighted dimensions. This doesn't matter on R2025a+ (where cell2mat became more forgiving for this). weightedTotal is also included because it has the identical storage line and the same potential for a precision mismatch.

Reproduction

sv = stateVector;
sv = sv.add('T', 'field.grid');        % gridfile with single-precision lat/lon
meta = gridfile('field.grid').metadata;
w  = cosd(meta.lat(extratropical));    % single weights (single lat)
sv = sv.mean('T', 'lat', w);           % pre-computed weighted mean over lat
sv = sv.build(nEns, ...);              % pre-R2025a: cell2mat data-type error

Casting the weights (or, user-side, w = double(cosd(meta.lat(...)))) resolves it.

weightedMean and weightedTotal stored weights with their input class. Single weights (e.g. cosd of single-precision lat/lon) then mix single with the double [] of non-weighted dimensions in obj.weights, and serialize's cell2mat rejects the mixture of precisions in pre-R2025a ("All contents of the input cell array must be of the same data type"), leading to a failed stateVector.build. Solution is to cast the stored weights to double.
@JonKing93

JonKing93 commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Hey @kanchukaitis, I think this makes sense. One thing I want to check though - will upgrading the weights always promote the built ensemble from single to double?

For example, consider a state vector that's designed from some single precision variable. And that state vector has a weighted mean that uses single precision weights. In that case, I think the built ensemble should also be single precision - all our data so far has been single precision, and it seems like a memory waste to force everything to be twice as large. Would you be able to test that scenario with the new code and see what happens? (and if it differs from the current implementation)

Brainstorming a possible alternate approach - maybe stateVectorVariable needs to handle the data type of the weights more carefully. Something along the lines of:

  • stateVectorVariable.weightedMean, line 45: Checks the data type of the variable, and casts the empty array to the same precision
  • stateVectorVariable.serialize, near lines 99-103: Would need to check data types and upgrade as needed.

Very open to suggestions here, let me know what you think

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants