Repository navigation
2026 3 production - #423
Shekar V (vshekar) wants to merge 101 commits into
Conversation
Cleaned up some of the function calls
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings remain across validation, startup, GUI, concurrency, and environment setup.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
This pull request consolidates production LSDC updates across request modeling, Bluesky/Tiled integration, Ophyd devices, GUI workflows, camera handling, command workers, and deployment tasks.
Changes:
- Adds typed request/result models and Bluesky metadata generation.
- Introduces device abstractions, Qt/EPICS bridges, camera workers, and recovery plans.
- Updates GUI, collection processing, runtime environments, and beamline tooling.
File summaries
| File | Reviewed changes |
|---|---|
utils/models.py |
Adds typed request, result, sample, and collection models; validation findings remain. |
utils/custom_pv.py |
Adds Qt/EPICS signal bridging and coalescing; callback lifecycle and synchronization findings remain. |
utils/command_execution.py |
Adds parallel-safety command annotations. |
utils/bluesky.py |
Generates Bluesky and proposal metadata; serialization and timeout findings remain. |
threads.py |
Adds camera fetching and worker updates; timeout and decode-handling findings remain. |
start_bs.py |
Configures Tiled writing and beamline devices; an FMX initialization finding remains. |
setenergy_lsdc.py |
Provides FMX energy-scan support. |
server.py |
Adds normal and immediate command workers; worker-coordination findings remain. |
runFastDPH5.py |
Integrates fast-DP processing. |
robot_lib.py |
Provides robot control support. |
raddoseLib.py |
Integrates RADDOSE calculations. |
pixi.toml |
Defines runtime tasks and dependencies; environment propagation findings remain. |
mxbluesky/plans/top_view.py |
Adds top-view alignment plans. |
mxbluesky/plans/loop_detection.py |
Adds loop-detection plans. |
mxbluesky/plans/__init__.py |
Exports Bluesky plans. |
mxbluesky/devices/zebra.py |
Defines Zebra devices. |
mxbluesky/devices/vector_program.py |
Defines vector-program devices. |
mxbluesky/devices/top_align.py |
Defines top-alignment devices. |
mxbluesky/devices/sample_environment.py |
Defines sample-environment devices. |
mxbluesky/devices/optics.py |
Defines optics devices. |
mxbluesky/devices/misc.py |
Defines miscellaneous beamline devices. |
mxbluesky/devices/md2.py |
Defines MD2 devices. |
mxbluesky/devices/md2_flyers.py |
Adds MD2 flyer support. |
mxbluesky/devices/ioc_control.py |
Defines IOC controls. |
mxbluesky/devices/governor.py |
Defines governor controls. |
mxbluesky/devices/goniometer.py |
Defines goniometer devices. |
mxbluesky/devices/generic.py |
Adds generic device abstractions. |
mxbluesky/devices/diagnostics.py |
Defines diagnostic devices. |
mxbluesky/devices/detector.py |
Defines detector devices. |
mxbluesky/devices/cryostream.py |
Defines cryostream devices. |
mxbluesky/devices/comm.py |
Defines communication PVs. |
mxbluesky/devices/click_center.py |
Adds click-centering support. |
mxbluesky/devices/cameras.py |
Defines camera devices. |
mxbluesky/devices/beamline_devices.py |
Provides the beamline device factory. |
mxbluesky/devices/auto_recovery.py |
Adds automatic recovery support. |
mxbluesky/devices/attenuation.py |
Defines attenuation devices. |
mxbluesky/devices/__init__.py |
Exports device definitions. |
lsdcServer |
Provides the server launcher. |
lsdcGui.py |
Provides the GUI launcher. |
gui/widgets/log_widget.py |
Updates the log display widget. |
gui/vector.py |
Updates vector controls. |
gui/raster.py |
Updates raster controls. |
gui/dialog/user_screen.py |
Updates the user screen. |
gui/dialog/staff_screen.py |
Adds staff controls; checkbox state handling findings remain. |
gui/dialog/set_energy.py |
Updates the set-energy dialog. |
gui/dialog/multicol.py |
Updates multi-collection controls. |
gui/dialog/dewar.py |
Updates dewar controls. |
gui/dewar_tree.py |
Adds threaded refresh and proposal checks; retry and throttling findings remain. |
gui/data_loc_info.py |
Updates data-location information. |
gui/albula/interface.py |
Updates Albula integration. |
gov_lib.py |
Provides governor library support. |
gon_lib.py |
Provides goniometer library support. |
Gen_Traj_Square.py |
Generates square trajectories. |
Gen_Commands.py |
Generates beamline commands. |
epics_det.py |
Provides EPICS detector support. |
embl_robot.py |
Provides EMBL robot support. |
db_lib.py |
Provides database access helpers. |
daq_utils.py |
Updates DAQ initialization and camera settings; configuration findings remain. |
daq_mainAux.py |
Updates auxiliary DAQ operations. |
daq_main2.py |
Updates primary DAQ operations. |
daq_main_common.py |
Updates shared DAQ dispatch. |
config_params.py |
Adds cycle configuration discovery; startup request handling findings remain. |
bin/lsdcServer_nyx.cmd |
Adds the NYX server launcher. |
bin/lsdcServer_fmx.cmd |
Adds the FMX server launcher. |
bin/lsdcServer_amx.cmd |
Adds the AMX server launcher. |
bin/lsdcRemote_fmx.cmd |
Adds the FMX remote launcher. |
bin/lsdcRemote_amx.cmd |
Adds the AMX remote launcher. |
bin/lsdcGui_nyx |
Adds the NYX GUI launcher. |
bin/lsdcGui_fmx |
Adds the FMX GUI launcher. |
bin/lsdcGui_amx |
Adds the AMX GUI launcher. |
bin/fmx_env.sh |
Sets the FMX runtime environment. |
bin/amx_env.sh |
Sets the AMX runtime environment. |
beamline_support.py |
Updates beamline support utilities. |
.gitignore |
Updates ignored files. |
Review details
Suppressed comments (11)
daq_lib.py:721
- The autoproc eligibility check is on the post-collection processing path but performs this proposal API request without a timeout. An unavailable API can block
collectDataindefinitely after acquisition, preventing the queue from advancing; use a bounded connect/read timeout.
r = requests.get(f"{os.environ['NSLS2_API_URL']}/v1/proposal/{currentRequest['proposalID']}")
daq_utils.py:69
init_environment()now setsunitScalingandsampleCameraCountbut no longer loads the configuredmag1ViewAngle–mag4ViewAnglevalues. Those globals remain at their module defaults (CAMERA_ANGLE_BEAM), soControlMain.focusTweakCBand the related centering path ignore beamline camera-orientation configuration and use the beam view for every camera. Restore the per-cameragetBlConfigreads with the existing fallback behavior.
unitScaling = 1
sampleCameraCount = 4
gui/dewar_tree.py:174
- This throttle also rejects explicit
hard_refreshandget_latest_pucksrequests made within one second of any prior refresh. Those user-triggered operations are then silently dropped, so the tree can remain stale; apply the interval only to ordinary refreshes or queue the special request.
if (
time.monotonic() - self._last_refresh_started_monotonic
< self._min_refresh_interval_s
):
return
pixi.toml:12
- This task executes
fmx_env.shas a child process, so its exports are lost beforeipythonstarts; the remote shell then lacksBEAMLINE_ID,CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
pixi.toml:15 - This task executes
fmx_env.shas a child process, so its exports are lost beforeipythonstarts; the server then lacksBEAMLINE_ID,CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
pixi.toml:21 - This task executes
amx_env.shas a child process, so its exports are lost beforeipythonstarts; the remote shell then lacksBEAMLINE_ID,CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
pixi.toml:24 - This task executes
amx_env.shas a child process, so its exports are lost beforeipythonstarts; the server then lacksBEAMLINE_ID,CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
threads.py:77 loadFromDatareturnsFalsefor a non-image response, but that result is ignored here. A camera endpoint that returns an HTML/error body with HTTP 200 will be marked healthy and a nullQImagewill be emitted instead of entering the recovery path; check the return value and raise on decode failure.
utils/bluesky.py:24- The successful path serializes the validated request with Python field names (
detector_distance,raster_def, etc.), while the validation fallback returns the raw database object using aliases (detDist,rasterDef). The same request therefore produces two incompatiblecollection_metadataschemas depending on whether validation happens to succeed; serialize withby_alias=True(or normalize both paths) consistently.
utils/custom_pv.py:146 - For an ophyd
Signal,subscribe()registers the callback and does not return an unsubscribe token;close()consequently leaves_subscription_tokenasNoneand never removes this callback. Closed bridges will keep receiving updates and retain their GUI slots. Store the callback passed tosubscribeand pass that same callback tounsubscribe.
utils/custom_pv.py:138 - All bridges in
ControlMain.initCallbacksare built from ophyd Signals, so this branch is used andrun=Falsesuppresses the initial subscription callback. The GUI therefore never receives the current state for its checkboxes/status fields until a PV changes; the existing UI subscription pattern usesrun=Truefor initial values. Default this toTrueor set it explicitly for state bridges.
- Files reviewed: 65/79 changed files
- Comments generated: 14
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| self.set_energy_checkbox.setChecked(False) | ||
| self.set_energy_checkbox.stateChanged.connect(self.set_energy_check_cb) | ||
| self.set_energy_checkbox.setChecked(True if getBlConfig(SET_ENERGY_CHECK) else False) | ||
| self.set_energy_checkbox.clicked.connect(self.set_energy_check_cb) |
| version = "0.1.0" | ||
|
|
||
| [tasks.fmx-gui] | ||
| cmd = "./bin/fmx_env.sh && cd \"$INIT_CWD\" && setsid -f python /nsls2/software/mx/daq/lsdc_fmx/lsdcGui.py" |
| cmd = "./bin/fmx_env.sh && cd \"$INIT_CWD\" && ipython -i /nsls2/software/mx/daq/lsdc_fmx/server.py" | ||
|
|
||
| [tasks.amx-gui] | ||
| cmd = "./bin/amx_env.sh && cd \"$INIT_CWD\" && setsid -f python /nsls2/software/mx/daq/lsdc_amx/lsdcGui.py" |
| if command_name == RUN_QUEUE_COMMAND: | ||
| run_dc_queue_active.set() | ||
| try: | ||
| process_input(cmd) | ||
| finally: | ||
| if command_name == RUN_QUEUE_COMMAND: | ||
| run_dc_queue_active.clear() |
| @@ -173,6 +173,8 @@ class SampleXYZ(Device): | |||
| from embl_robot import EMBLRobot | |||
| import setenergy_lsdc | |||
| ] | ||
| except Exception as e: | ||
| logger.exception(e) | ||
| proposal_membership_updates[proposal_id] = False |
| cap = cv2.VideoCapture(url, cv2.CAP_FFMPEG) | ||
| cap.set(cv2.CAP_PROP_OPEN_TIMEOUT_MSEC, 2000) | ||
| cap.set(cv2.CAP_PROP_READ_TIMEOUT_MSEC, 2000) |
| bs_md["collection_metadata"] = request.request_def.model_dump(mode="json") # request["request_obj"] | ||
| except ValidationError as e: | ||
| logger.exception("Could not validate request") | ||
| proposal_id = request_dict.get("proposal_id") |
|
|
||
|
|
||
| try: | ||
| r = requests.get(f"{os.environ['NSLS2_API_URL']}/v1/proposal/{proposal_id}") |
| if hasattr(self.source, "add_callback"): | ||
| self._epics_callback_index = self.source.add_callback( | ||
| self._on_epics_changed, | ||
| **callback_kwargs, | ||
| ) | ||
| elif hasattr(self.source, "subscribe"): | ||
| event_type = callback_kwargs.pop("event_type", "value") | ||
| run = callback_kwargs.pop("run", False) | ||
| if callback_kwargs: | ||
| unknown = ", ".join(sorted(callback_kwargs)) | ||
| raise TypeError(f"Unsupported ophyd subscribe kwargs: {unknown}") | ||
| self._subscription_token = self.source.subscribe( | ||
| self._on_ophyd_changed, | ||
| event_type=event_type, | ||
| run=run, | ||
| ) | ||
| else: | ||
| raise TypeError( | ||
| "signal_or_name must be EPICS PV name, EPICS PV object, or ophyd Signal-like object" | ||
| ) | ||
|
|
||
| self._pending_value: Any = _NOTHING | ||
| self._coalesce_timer = QtCore.QTimer(self) | ||
| self._coalesce_timer.setInterval(coalesce_interval_ms) | ||
| self._coalesce_timer.timeout.connect(self._flush_coalesced) |
RobertSchaffer1
left a comment
There was a problem hiding this comment.
15 files in, might as well start here.
| conda activate lsdc-server-2023-2-latest | ||
| $LSDCHOME/daq_mainAux.py | ||
| export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt | ||
| export SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt |
There was a problem hiding this comment.
I wonder if it's worth putting these (and above line) all somewhere, so we can change them simultaneously if needed.
|
|
||
| def addMultiRequestLocation(self, parentReqID, hitCoords, locIndex, wedge=10.0): | ||
| def addMultiRequestLocation(self, parentReqID, hitCoords, locIndex, wedge=None): #rough proto of what to pass here for details like how to organize data | ||
| print(wedge) |
All previous PRs combined into one