Skip to content

2026 3 production - #423

Open
Shekar V (vshekar) wants to merge 101 commits into
masterfrom
2026-3-production
Open

Shekar V (vshekar) wants to merge 101 commits into
masterfrom
2026-3-production

Conversation

@vshekar

Copy link
Copy Markdown
Collaborator

All previous PRs combined into one

Shekar V (vshekar) and others added 30 commits September 15, 2026 17:00
Cleaned up some of the function calls
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 16, 2026 15:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical and moderate findings remain across validation, startup, GUI, concurrency, and environment setup.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This pull request consolidates production LSDC updates across request modeling, Bluesky/Tiled integration, Ophyd devices, GUI workflows, camera handling, command workers, and deployment tasks.

Changes:

  • Adds typed request/result models and Bluesky metadata generation.
  • Introduces device abstractions, Qt/EPICS bridges, camera workers, and recovery plans.
  • Updates GUI, collection processing, runtime environments, and beamline tooling.
File summaries
File Reviewed changes
utils/models.py Adds typed request, result, sample, and collection models; validation findings remain.
utils/custom_pv.py Adds Qt/EPICS signal bridging and coalescing; callback lifecycle and synchronization findings remain.
utils/command_execution.py Adds parallel-safety command annotations.
utils/bluesky.py Generates Bluesky and proposal metadata; serialization and timeout findings remain.
threads.py Adds camera fetching and worker updates; timeout and decode-handling findings remain.
start_bs.py Configures Tiled writing and beamline devices; an FMX initialization finding remains.
setenergy_lsdc.py Provides FMX energy-scan support.
server.py Adds normal and immediate command workers; worker-coordination findings remain.
runFastDPH5.py Integrates fast-DP processing.
robot_lib.py Provides robot control support.
raddoseLib.py Integrates RADDOSE calculations.
pixi.toml Defines runtime tasks and dependencies; environment propagation findings remain.
mxbluesky/plans/top_view.py Adds top-view alignment plans.
mxbluesky/plans/loop_detection.py Adds loop-detection plans.
mxbluesky/plans/__init__.py Exports Bluesky plans.
mxbluesky/devices/zebra.py Defines Zebra devices.
mxbluesky/devices/vector_program.py Defines vector-program devices.
mxbluesky/devices/top_align.py Defines top-alignment devices.
mxbluesky/devices/sample_environment.py Defines sample-environment devices.
mxbluesky/devices/optics.py Defines optics devices.
mxbluesky/devices/misc.py Defines miscellaneous beamline devices.
mxbluesky/devices/md2.py Defines MD2 devices.
mxbluesky/devices/md2_flyers.py Adds MD2 flyer support.
mxbluesky/devices/ioc_control.py Defines IOC controls.
mxbluesky/devices/governor.py Defines governor controls.
mxbluesky/devices/goniometer.py Defines goniometer devices.
mxbluesky/devices/generic.py Adds generic device abstractions.
mxbluesky/devices/diagnostics.py Defines diagnostic devices.
mxbluesky/devices/detector.py Defines detector devices.
mxbluesky/devices/cryostream.py Defines cryostream devices.
mxbluesky/devices/comm.py Defines communication PVs.
mxbluesky/devices/click_center.py Adds click-centering support.
mxbluesky/devices/cameras.py Defines camera devices.
mxbluesky/devices/beamline_devices.py Provides the beamline device factory.
mxbluesky/devices/auto_recovery.py Adds automatic recovery support.
mxbluesky/devices/attenuation.py Defines attenuation devices.
mxbluesky/devices/__init__.py Exports device definitions.
lsdcServer Provides the server launcher.
lsdcGui.py Provides the GUI launcher.
gui/widgets/log_widget.py Updates the log display widget.
gui/vector.py Updates vector controls.
gui/raster.py Updates raster controls.
gui/dialog/user_screen.py Updates the user screen.
gui/dialog/staff_screen.py Adds staff controls; checkbox state handling findings remain.
gui/dialog/set_energy.py Updates the set-energy dialog.
gui/dialog/multicol.py Updates multi-collection controls.
gui/dialog/dewar.py Updates dewar controls.
gui/dewar_tree.py Adds threaded refresh and proposal checks; retry and throttling findings remain.
gui/data_loc_info.py Updates data-location information.
gui/albula/interface.py Updates Albula integration.
gov_lib.py Provides governor library support.
gon_lib.py Provides goniometer library support.
Gen_Traj_Square.py Generates square trajectories.
Gen_Commands.py Generates beamline commands.
epics_det.py Provides EPICS detector support.
embl_robot.py Provides EMBL robot support.
db_lib.py Provides database access helpers.
daq_utils.py Updates DAQ initialization and camera settings; configuration findings remain.
daq_mainAux.py Updates auxiliary DAQ operations.
daq_main2.py Updates primary DAQ operations.
daq_main_common.py Updates shared DAQ dispatch.
config_params.py Adds cycle configuration discovery; startup request handling findings remain.
bin/lsdcServer_nyx.cmd Adds the NYX server launcher.
bin/lsdcServer_fmx.cmd Adds the FMX server launcher.
bin/lsdcServer_amx.cmd Adds the AMX server launcher.
bin/lsdcRemote_fmx.cmd Adds the FMX remote launcher.
bin/lsdcRemote_amx.cmd Adds the AMX remote launcher.
bin/lsdcGui_nyx Adds the NYX GUI launcher.
bin/lsdcGui_fmx Adds the FMX GUI launcher.
bin/lsdcGui_amx Adds the AMX GUI launcher.
bin/fmx_env.sh Sets the FMX runtime environment.
bin/amx_env.sh Sets the AMX runtime environment.
beamline_support.py Updates beamline support utilities.
.gitignore Updates ignored files.
Review details

Suppressed comments (11)

daq_lib.py:721

  • The autoproc eligibility check is on the post-collection processing path but performs this proposal API request without a timeout. An unavailable API can block collectData indefinitely after acquisition, preventing the queue from advancing; use a bounded connect/read timeout.
            r = requests.get(f"{os.environ['NSLS2_API_URL']}/v1/proposal/{currentRequest['proposalID']}")

daq_utils.py:69

  • init_environment() now sets unitScaling and sampleCameraCount but no longer loads the configured mag1ViewAngle–mag4ViewAngle values. Those globals remain at their module defaults (CAMERA_ANGLE_BEAM), so ControlMain.focusTweakCB and the related centering path ignore beamline camera-orientation configuration and use the beam view for every camera. Restore the per-camera getBlConfig reads with the existing fallback behavior.
  unitScaling = 1
  sampleCameraCount = 4

gui/dewar_tree.py:174

  • This throttle also rejects explicit hard_refresh and get_latest_pucks requests made within one second of any prior refresh. Those user-triggered operations are then silently dropped, so the tree can remain stale; apply the interval only to ordinary refreshes or queue the special request.
        if (
            time.monotonic() - self._last_refresh_started_monotonic
            < self._min_refresh_interval_s
        ):
            return

pixi.toml:12

  • This task executes fmx_env.sh as a child process, so its exports are lost before ipython starts; the remote shell then lacks BEAMLINE_ID, CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
    pixi.toml:15
  • This task executes fmx_env.sh as a child process, so its exports are lost before ipython starts; the server then lacks BEAMLINE_ID, CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
    pixi.toml:21
  • This task executes amx_env.sh as a child process, so its exports are lost before ipython starts; the remote shell then lacks BEAMLINE_ID, CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
    pixi.toml:24
  • This task executes amx_env.sh as a child process, so its exports are lost before ipython starts; the server then lacks BEAMLINE_ID, CONFIGDIR, and the DAQ environment. Source the environment script in the task shell.
    threads.py:77
  • loadFromData returns False for a non-image response, but that result is ignored here. A camera endpoint that returns an HTML/error body with HTTP 200 will be marked healthy and a null QImage will be emitted instead of entering the recovery path; check the return value and raise on decode failure.
    utils/bluesky.py:24
  • The successful path serializes the validated request with Python field names (detector_distance, raster_def, etc.), while the validation fallback returns the raw database object using aliases (detDist, rasterDef). The same request therefore produces two incompatible collection_metadata schemas depending on whether validation happens to succeed; serialize with by_alias=True (or normalize both paths) consistently.
    utils/custom_pv.py:146
  • For an ophyd Signal, subscribe() registers the callback and does not return an unsubscribe token; close() consequently leaves _subscription_token as None and never removes this callback. Closed bridges will keep receiving updates and retain their GUI slots. Store the callback passed to subscribe and pass that same callback to unsubscribe.
    utils/custom_pv.py:138
  • All bridges in ControlMain.initCallbacks are built from ophyd Signals, so this branch is used and run=False suppresses the initial subscription callback. The GUI therefore never receives the current state for its checkboxes/status fields until a PV changes; the existing UI subscription pattern uses run=True for initial values. Default this to True or set it explicitly for state bridges.
  • Files reviewed: 65/79 changed files
  • Comments generated: 14
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

self.set_energy_checkbox.setChecked(False)
self.set_energy_checkbox.stateChanged.connect(self.set_energy_check_cb)
self.set_energy_checkbox.setChecked(True if getBlConfig(SET_ENERGY_CHECK) else False)
self.set_energy_checkbox.clicked.connect(self.set_energy_check_cb)
Comment thread pixi.toml
version = "0.1.0"

[tasks.fmx-gui]
cmd = "./bin/fmx_env.sh && cd \"$INIT_CWD\" && setsid -f python /nsls2/software/mx/daq/lsdc_fmx/lsdcGui.py"
Comment thread pixi.toml
cmd = "./bin/fmx_env.sh && cd \"$INIT_CWD\" && ipython -i /nsls2/software/mx/daq/lsdc_fmx/server.py"

[tasks.amx-gui]
cmd = "./bin/amx_env.sh && cd \"$INIT_CWD\" && setsid -f python /nsls2/software/mx/daq/lsdc_amx/lsdcGui.py"
Comment thread server.py
Comment on lines +89 to +95
if command_name == RUN_QUEUE_COMMAND:
run_dc_queue_active.set()
try:
process_input(cmd)
finally:
if command_name == RUN_QUEUE_COMMAND:
run_dc_queue_active.clear()
Comment thread start_bs.py
@@ -173,6 +173,8 @@ class SampleXYZ(Device):
from embl_robot import EMBLRobot
import setenergy_lsdc
Comment thread gui/dewar_tree.py
]
except Exception as e:
logger.exception(e)
proposal_membership_updates[proposal_id] = False
Comment thread threads.py
Comment on lines +63 to +65
cap = cv2.VideoCapture(url, cv2.CAP_FFMPEG)
cap.set(cv2.CAP_PROP_OPEN_TIMEOUT_MSEC, 2000)
cap.set(cv2.CAP_PROP_READ_TIMEOUT_MSEC, 2000)
Comment thread utils/bluesky.py
bs_md["collection_metadata"] = request.request_def.model_dump(mode="json") # request["request_obj"]
except ValidationError as e:
logger.exception("Could not validate request")
proposal_id = request_dict.get("proposal_id")
Comment thread utils/bluesky.py


try:
r = requests.get(f"{os.environ['NSLS2_API_URL']}/v1/proposal/{proposal_id}")
Comment thread utils/custom_pv.py
Comment on lines +131 to +155
if hasattr(self.source, "add_callback"):
self._epics_callback_index = self.source.add_callback(
self._on_epics_changed,
**callback_kwargs,
)
elif hasattr(self.source, "subscribe"):
event_type = callback_kwargs.pop("event_type", "value")
run = callback_kwargs.pop("run", False)
if callback_kwargs:
unknown = ", ".join(sorted(callback_kwargs))
raise TypeError(f"Unsupported ophyd subscribe kwargs: {unknown}")
self._subscription_token = self.source.subscribe(
self._on_ophyd_changed,
event_type=event_type,
run=run,
)
else:
raise TypeError(
"signal_or_name must be EPICS PV name, EPICS PV object, or ophyd Signal-like object"
)

self._pending_value: Any = _NOTHING
self._coalesce_timer = QtCore.QTimer(self)
self._coalesce_timer.setInterval(coalesce_interval_ms)
self._coalesce_timer.timeout.connect(self._flush_coalesced)

@RobertSchaffer1 RobertSchaffer1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

15 files in, might as well start here.

Comment thread bin/lsdcRemote_amx.cmd
conda activate lsdc-server-2023-2-latest
$LSDCHOME/daq_mainAux.py
export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
export SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it's worth putting these (and above line) all somewhere, so we can change them simultaneously if needed.

Comment thread gui/dialog/multicol.py

def addMultiRequestLocation(self, parentReqID, hitCoords, locIndex, wedge=10.0):
def addMultiRequestLocation(self, parentReqID, hitCoords, locIndex, wedge=None): #rough proto of what to pass here for details like how to organize data
print(wedge)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants