Repository navigation
Carry the track length and start frame through to Insight3 - #63
Merged
Merged
Conversation
bin_to_lmchallenge_format.py expands a molecule that was on for several
frames back into one row per frame:
track_length = i3_block['tl'][i]
for j in range(track_length):
fr = i3_block['fr'][i] + j
That worked against an Insight3 era file. tracker.c wrote
object_data_int[TLEN] = cur->track_length, avemlist.c flagged TLEN as
NOAVERAGE so it survived averaging, and insight.h defines FRAME as "frame
the peak first occured in".
hdf5ToBin(), which replaced that path in Dec 2017, wrote neither.
createFromMultiFit() never touched 'tl', so it kept its default of 1, and
every track was written into frame 1. The expansion was silently a no-op:
in the HDF5 exported
track_length / tl 1, 3, 10 1, 1, 1
frame_number / fr 0, 4, 20 1, 1, 1
rows out of lmchallenge 14 3
Map both fields in createFromMultiFit(), which is where every other HDF5
field is translated. Insight3 frame numbers start at 1 and HDF5 frame
numbers start at 0, hence the +1. Localizations carry neither field so
that path is unchanged.
Restoring 'tl' on its own would have made things worse, which is why
2e43489 deliberately left it alone: each expanded row was given the
whole of 'a', so a ten frame track would have reported its full photon
count ten times. 'a' is a track total in our files, and in Insight3 era
ones too since avemlist flagged AREA as TOTAL, so each row now gets its
share.
bin_to_lmchallenge_format.py was executable-only, all module level code
reading sys.argv at import, so nothing could test it. Wrapped in
binToLMChallenge() with a __main__ guard, like every other utility here,
and it returns the row count.
Note that a track with a gap in it is still expanded into consecutive
frames. The .bin format records only where a track started and how long
it was, so the information needed to do better is not there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bin_to_lmchallenge_format.pyexpands a molecule that was on for severalframes back into one row per frame:
That worked against an Insight3 era file, where
tracker.cwroteTLEN,avemlist.cflagged itNOAVERAGEso it survived averaging, andinsight.hdefinesFRAMEas "frame the peak first occured in".hdf5ToBin(), which replaced that path in Dec 2017, wrote neither field, sothe expansion was silently a no-op:
track_length/tlframe_number/frBoth halves have to change together, which is why #59 deliberately left
tlalone: each expanded row is given the whole of
a, so restoring the tracklength on its own would have reported a ten frame track's full photon count
ten times over.
ais a track total in the files this package writes, and inInsight3 era ones too since
avemlistflaggedAREAasTOTAL, so each rownow gets its share.
Changes
createFromMultiFit()mapsframe_numberontofrandtrack_lengthonto
tl, which is where every other HDF5 field is already translated.Insight3 frames are 1-based and HDF5 frames are 0-based, hence the
+1.Localizations carry neither field, so that path is untouched.
bin_to_lmchallenge_format.pydividesaby the track length.The converter was executable-only — all module level code reading
sys.argvat import — so nothing could test it. It is now
binToLMChallenge()with a__main__guard, like every other utility here, and returns the row count.That accounts for most of its diff.
The new test runs the whole path, HDF5 to
.binto challenge text, andchecks that each track occupies consecutive frames from its own start frame
and that each row carries its per-frame photons. It fails on the parent
commit.
Note that a track with a gap in it is still expanded into consecutive frames.
The
.binformat records only where a track started and how long it was, sothe information needed to do better is not there.
Full test suite passes, 278 tests.