Skip to content

Create large-image tiles in a job instead of on every upload request - #2008

Open
romleiaj wants to merge 1 commit into
mainfrom
fix/large-image-tiles-job
Open

romleiaj wants to merge 1 commit into
mainfrom
fix/large-image-tiles-job

Conversation

@romleiaj

Copy link
Copy Markdown
Collaborator

girder_large_image's auto-set runs createImageItem synchronously when each file is saved, so every uploaded or imported image -- including JPG/PNG frames of plain image sequences, which never use tiles -- was probed for a tile source inside its own upload request. On a multicam flight with thousands of frames per camera, uploads queued behind that work until requests hit the gateway timeout.

The DIVE plugin now defaults large_image.auto_set to off (an explicit admin setting still wins). postprocess queues a create_large_image_tiles job when a folder holds large-image files (TIFF, NITF, ...) that have no tiles yet; the job creates them one item at a time with progress and marks the folder as a dataset when done. Uploads and filesystem/S3 imports both reach postprocess, so both are covered. The existing per-item tile check is shared with convert_large_images.

The multicam upload's wait for camera processing now restarts its 10-minute clock whenever a job reports progress, so a long tiling job that keeps moving is not cut off.

girder_large_image's auto-set runs createImageItem synchronously when each
file is saved, so every uploaded or imported image -- including JPG/PNG
frames of plain image sequences, which never use tiles -- was probed for a
tile source inside its own upload request. On a multicam flight with
thousands of frames per camera, uploads queued behind that work until
requests hit the gateway timeout.

The DIVE plugin now defaults large_image.auto_set to off (an explicit admin
setting still wins). postprocess queues a create_large_image_tiles job when a
folder holds large-image files (TIFF, NITF, ...) that have no tiles yet; the
job creates them one item at a time with progress and marks the folder as a
dataset when done. Uploads and filesystem/S3 imports both reach postprocess,
so both are covered. The existing per-item tile check is shared with
convert_large_images.

The multicam upload's wait for camera processing now restarts its 10-minute
clock whenever a job reports progress, so a long tiling job that keeps moving
is not cut off.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant