Repository navigation
Fix names that do not resolve, and drop Python 2 - #66
Merged
Merged
Conversation
That commit re-enabled the first drift correction test by renaming
_test_drift_correction_1() to test_drift_correction_1(), and did not
update the __main__ block, which still called the old name. Running the
file directly has raised NameError ever since:
NameError: name '_test_drift_correction_1' is not defined.
Did you mean: 'test_drift_correction_1'?
The full test suite stayed green throughout, because pytest collects
test_* functions and never executes __main__. That is also why this was
not caught: the verification for that commit ran pytest, which cannot
reach the broken line.
Also lists test_drift_correction_1_xy_failure(), added by the same
commit and likewise never added to the block.
Verified by running the file the way __main__ intends rather than
through pytest, which exercises all eleven tests and now exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two names in it were wrong, and it has never been called, so neither
showed up:
sin_theta_2 = (n1/n2)*sin_theta_1 NameError, no n1 or n2
return self.appylNARestriction(pf_ab) AttributeError, appyl
The first is what the linter found. The second it cannot see, since an
attribute is not a bare name, and fixing only the first moves the
failure four lines down rather than removing it. applyNARestriction is
spelled correctly at the file's five other call sites.
For Snell's law here, n1 is the immersion medium and n2 the sample, as
the sibling aberration() does it eight lines above:
sin_theta_2 = (self.imm_index/smp_index)*sin_theta_1
which also matches the Gibson-Lanni model this function cites.
The test calls the function, which is all that is needed to catch both
names, and then checks two things that are true of it by construction:
everything beyond the NA is zero, which is what makes the second fix
observable rather than assumed, and at zero defocus with the sample
index matching the immersion index the aberration is identically 1.
That test uses geo.r > 1.0 for "outside the NA", because that is what
applyNARestriction() masks on. geo.r is a normalized radius while
geo.r_max is the NA edge in grid units, and confusing the two, as I did
at first, gives a mask that quietly includes part of the pupil.
The geometry is chosen so that 691 of 1024 pixels fall outside the NA.
At the geometry the other tests in this file use, nothing does, and the
restriction check would pass without testing anything.
This does not verify the physics, only that the function runs and obeys
those two properties. Nothing calls it.
Note that pupil_math.py has CRLF line endings, so it was edited in
binary; its line ending count is unchanged at 765.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three names wrong in one four line block, all from copying it out of
MPPeakFinderArb.setVariances():
print("psf max", numpy.max(psf)) no psf here
filename = "psf_z0.0_c{1:d}.tif".format(j) no j, and no arg 1
tifffile.imwrite(filename, psf.astype(...))
In the version it came from, the loop runs over z and then channel, so
the PSF is 'psf' and the channel index is 'j', and the filename carries
both z and channel: "psf_z{0:.3f}_c{1:d}".format(mfilter_z, j). The
Gaussian version has a single z, so it hardcodes z in the name and loops
over channels with 'i', and its PSF is 'psf_norm'.
So the block referenced two names that do not exist and kept a format
string for two arguments while passing one. Fixing only the names would
have moved the failure to IndexError on {1:d}. That is the same shape of
defect as the Spliner check_mode bug fixed in the fitters branch: a
debugging path that crashes the moment anyone turns it on.
There was no test file for fitting_mp at all. This adds one that builds
the finder, turns check_mode on and calls setVariances(), which is
enough to reach the block. It checks that one image is written per
channel and named by the channel index, and that the contents match a
Gaussian PSF computed independently in the test rather than read back
out of the finder.
The mapping file and parameters the finder needs are built the way
diagnostics/multiplane_dao/configure.py does it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rest of what the linter found, all of the same kind: a name used
that is not defined, or defined twice. None of these paths runs in the
test suite, which is why they have survived.
multi_plane/analysis_io.py static_background is used but never
imported. sa_library/analysis_io.py
has the same block and does import it,
as
import storm_analysis.sa_library.\
static_background as static_background
Guarded by static_background_estimate,
with a FIXME saying it is untested.
admm/admm_lasso_c.py def run(self, a_lamba, iterations)
against a body that uses a_lambda. The
parameter is the misspelt one. Nothing
calls this method, so renaming it
breaks no caller.
multi_plane/mp_fit_arb_c.py super(MPSFFnFit, ...) inside class
MPPSFFnFit. A dropped P.
simulator/emitters_on_lines.py argls.nemitters for args.nemitters, in
the argument list of the __main__
call, so the script fails for everyone
who runs it.
jupyter_examples/ __main__ calls sCMOSSinglePlaneXML().
multiplane_measure_psf.py The function is sCMOSSingleFrameXML().
And two things defined twice:
rolling_ball_bgr/ imports numpy twice, six lines apart.
rolling_ball_lib_c.py
diagnostics/ imports spliner's settings module at
spliner_configure_common.py the top and then shadows it with the
settings parameter of configure(),
which is the only function in the
file. Every settings. reference in it
is the parameter. Worse than unused:
the module is meant to be
parameterized, and fista_decon's
configure.py calls it with its own
settings. Removing the import is safer
than renaming the parameter, which two
callers pass positionally.
Not touched, and still reported: unicode in loadclib.py, which sits in
the else of a sys.version_info > (3,0) test and so is unreachable rather
than broken, unicode twice in TSFProto_pb2.py, which is generated
protobuf output and should be regenerated rather than hand edited, and
raw_input in spline1D.py. Removing those means deciding to drop Python 2
from those files, which is a separate question.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Same omission as 052781e, in the other three files where the audit added or re-enabled tests. pytest collects test_* functions and ignores __main__, so running the file directly ran a subset of it and nobody noticed. test_clusters_sa_h5py.py test_cl_sa_h5py_z_is_loaded and test_cl_sa_h5py_drops_invalid_z, added with the clustering z fix. test_fitz_c.py test_fitz_c_marker_survives_drift and test_fitz_c_marker_propagates_to_tracks, added with the -infinity marker. test_spliner.py test_psf_to_spline and test_psf_to_spline_2D were re-enabled by renaming them, and the block still had the old names commented out as _test_psf_to_spline(). Uncommented under the names they now have. Verified by running all three files directly rather than through pytest, which is the path that was wrong. All exit 0. Four files still do not list everything they define: test_clustering.py, test_dataio.py, test_matched_filter.py and test_measure_psf_utils.py. Those omissions predate this work and none of them raises, so they are left alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pyproject.toml already says requires-python = '>=3.10', and CI builds on
3.10, 3.11 and 3.12 only, so none of this has been a supported path for
some time. It was dead code rather than a policy question.
Twenty five sys.version_info guards, in two shapes.
Five outside the tests, where the Python 2 branch is simply deleted and
the Python 3 branch kept:
sa_utilities/bin_to_hdf5.py ElementTree.tostring(..., 'unicode')
sa_library/analysis_io.py rather than 'ISO-8859-1'
multi_plane/analysis_io.py
sa_library/loadclib.py SetDllDirectoryW without unicode()
L1H/setup_A_matrix.py pickle.load(fp, encoding='latin1')
The loadclib one is where the linter's "undefined name unicode" came
from. It was never reachable, since it sits in the else of the version
test, but it was the reason that file was flagged.
Twenty in the tests, all of the form
# Only test for Python3 due to pickle incompatibility issues.
if (sys.version_info < (3, 0)):
return
which on any supported interpreter never fires. Two of those twenty were
mine, copied from their neighbours when I added the py_f tests.
Removing the guards left 'import sys' unused in seven files, so those go
too. loadclib.py keeps its import, it still uses sys.platform. This is
not a general unused-import cleanup, only the ones this commit made
dead.
Also fixes the two Python 2 idioms in spline1D.py's __main__ demo,
range(x.size/10), which is a float in Python 3 and raises, and
raw_input().
Not touched: sa_utilities/TSFProto_pb2.py, which is generated protobuf
output, says "DO NOT EDIT" at the top, and uses unicode("", "utf-8") in
two default values. It should be regenerated rather than hand edited,
and it cannot run in any case: google.protobuf is not a dependency of
this package, and its two consumers, read_tagged_spot_file.py and
bin_to_tagged_spot_file.py, use the Python 2 implicit relative form
"import TSFProto_pb2", which fails outright on Python 3. That corner is
three files that do not run, and untangling it is a separate job. Those
two names are the only F821 findings left in the tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ruff (
--select F, the pyflakes rules) reports 486 findings. This fixes theones that mean "this raises if the path runs", plus the Python 2 code that
several of them turn out to be.
None of these paths is covered by the test suite, which is how they survived.
The crash class
simulator/pupil_math.pysin_theta_2 = (n1/n2)*sin_theta_1— neither name existsmulti_plane/fitting_mp.pypsfandjundefined, and"...c{1:d}".format(j)passes one argument for index 1multi_plane/analysis_io.pystatic_backgroundused but never importedadmm/admm_lasso_c.pydef run(self, a_lamba, ...)against a body usinga_lambdamulti_plane/mp_fit_arb_c.pysuper(MPSFFnFit, ...)insideclass MPPSFFnFitsimulator/emitters_on_lines.pyargls.nemittersforargs.nemittersjupyter_examples/multiplane_measure_psf.py__main__callssCMOSSinglePlaneXML(); it issCMOSSingleFrameXML()test/test_drift_correction.py__main__calls_test_drift_correction_1(), renamed in #59Plus two things defined twice:
rolling_ball_lib_c.pyimports numpy twice,and
spliner_configure_common.pyimports spliner's settings module and thenshadows it with
configure()'s parameter — which matters becausefista_decon/configure.pycalls that function with its own settings.Two of these needed a second fix the linter cannot see.
pupil_math.pyalso calls
self.appylNARestriction()— an attribute, not a bare name, sopyflakes' rules do not reach it, and fixing only the
NameErrormoves thefailure four lines down.
fitting_mp.pylikewise would have gone fromNameErrortoIndexErroron the format string. Both were caught only bywriting a test that actually calls the code.
Tests
test_fitting_mp.pyis new — there was no test file for that module. Itbuilds the multiplane finder, turns
check_modeon and reaches the block,then checks the written TIFFs against a Gaussian PSF computed independently
in the test. The
pupil_mathtest uses a geometry chosen so that 691 of 1024pixels fall outside the NA; at the geometry the other tests in that file use,
nothing does, and the NA assertion would pass while testing nothing.
Python 2
pyproject.tomlalready declaresrequires-python = '>=3.10'and CI builds3.10 through 3.12, so the 25
sys.version_infoguards were dead code ratherthan a supported path. Removed, along with the seven
import sysstatementsthat became unused as a result, and the two Python 2 idioms in
spline1D.py'sdemo block.
Deliberately not done
TSFProto_pb2.pykeeps its twounicodecalls. It is generated protobufoutput marked "DO NOT EDIT",
google.protobufis not a dependency, and itstwo consumers use the Python 2 implicit relative
import TSFProto_pb2.Those three files do not run at all; untangling them is a separate job.
These are the only
F821findings left in the tree, down from 19.them means rewriting files wholesale for no functional gain, which deserves
its own decision rather than riding along here.
Full test suite passes, 280 tests. The four test files whose
__main__blocks changed were also run directly, since that is the path pytest never
executes and therefore never checked.