Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
7d61dcc
Spike; create master crop file via vips.
tonytw1 May 21, 2025
3500d24
Naive resize crops to jpegs only step.
tonytw1 May 23, 2025
19ef1ab
createCrops and resizeImageVips in same Future and arena to allow sha…
tonytw1 May 23, 2025
4ebd4e0
Try to use 1 load of the source image across all resizes; do the resi…
tonytw1 May 23, 2025
e2d2782
createCrops takes the masterCrop as a VImage rather than a file.
tonytw1 May 23, 2025
abd14d9
createMasterCrop moving to same arena.
tonytw1 May 23, 2025
5aa962f
Remove colourModel check from createMasterCrop.
tonytw1 May 23, 2025
116621c
Logging. Show when we reload the master crop from disk.
tonytw1 May 23, 2025
69e9701
Correct arena close timing.
tonytw1 May 23, 2025
6a10e38
createMasterCrop no Futures.
tonytw1 May 23, 2025
dc06d90
arena scope?
tonytw1 May 23, 2025
90e40fe
arena scope?
tonytw1 May 23, 2025
4d16b8e
arena scope?
tonytw1 May 23, 2025
7911a65
arena scope?
tonytw1 May 23, 2025
199b2c3
Spike; no reload of master crop image.
tonytw1 May 23, 2025
bb45497
Marking TODO; why local file for master crop.
tonytw1 May 24, 2025
290cfd1
Log local resize file location.
tonytw1 May 24, 2025
e5e4c86
PNG specific optimiseImage steps can move straight into the resizeIma…
tonytw1 May 24, 2025
287f378
Logging.
tonytw1 May 24, 2025
36891b4
TODO Strip true is needed to stop exif auto correction until we can s…
tonytw1 May 24, 2025
12f4932
Revert "Revert "Master image converted to sRBG colour space.""
tonytw1 May 24, 2025
3ed64e6
Resize uses save file.
tonytw1 Feb 10, 2026
d7ccb6f
Log crop type decision.
tonytw1 May 24, 2025
13b8122
Colour correct not effective if metadata stripped. Bake it in?
tonytw1 May 24, 2025
bb53966
Refactor; master crop image file write pushes up out of the create ma…
tonytw1 May 24, 2025
5526b62
createMasterCrop uses saveToFile and can now save PNGs.
tonytw1 Feb 10, 2026
fa45bb7
Refactor. S3 store of master crop pushes up. MasterCrop is a simpler …
tonytw1 May 24, 2025
d4005b8
Refactor. File create for mastercrop pushes up to be next to the s3 s…
tonytw1 May 24, 2025
f445198
Master file create order; before arena close.
tonytw1 May 24, 2025
a41d4e1
Comment; master crop if different from the the full dimensions crop; …
tonytw1 May 24, 2025
9cabcd5
Refactor; split creation of crop files from publishing to S3.
tonytw1 May 25, 2025
5bb187c
quantise crops.
tonytw1 Feb 11, 2026
4ee3eb9
isGraphic was sendint all TIFFs to PNG.
tonytw1 May 25, 2025
64cc3c3
isGraphic was sendint all TIFFs to PNG.
tonytw1 May 25, 2025
7c21b5c
Correct CMYK renders too light in crops.
tonytw1 May 25, 2025
2edce02
Unused parameters.
tonytw1 May 26, 2025
db2af8b
Unused parameters.
tonytw1 May 26, 2025
436bbd9
Bypass icc_transform for LAB images.
tonytw1 May 26, 2025
96feee0
Remove non used non vips functions.
tonytw1 Feb 28, 2026
2639d2a
Remove non used runConvert command.
tonytw1 May 26, 2025
0510af7
Remove unused iccColourSpace val.
tonytw1 Jun 29, 2025
527240e
Reapply png master quality.
tonytw1 Jun 29, 2025
e56de4d
Crop quality values can be int.
tonytw1 Jan 17, 2026
a5ab4a1
CropType decision can be deferred until after the master crop has bee…
tonytw1 Nov 24, 2025
8a8202e
Refactor. Push the isGraphic? decision up out of cropType so that is …
tonytw1 Dec 29, 2025
2bdbe4c
Spike. vips based implementation of isGraphic? is likely to involve t…
tonytw1 Dec 29, 2025
f91001c
[libvips] Testing around hasAlpha.
tonytw1 Feb 1, 2026
3a4eb9e
Test to exercise ImageOperations resize.
tonytw1 Feb 18, 2026
6664923
Clarify master crop quality values.
tonytw1 Jan 17, 2026
2408595
[libvips-cropping] Allow embedded icc profile to be used in crop icc …
tonytw1 Jan 19, 2026
27a189b
[libvips-cropping] Image operations resize takes the output file as a…
tonytw1 Jan 21, 2026
9995acc
[libvips-cropping] Cropping of LAB fixed by making non icc_transform …
tonytw1 Jan 19, 2026
d79be09
[libvips-cropping] Show that it is possible to manually trim metadata…
tonytw1 Feb 12, 2026
63311b1
[libvips-cropping] Bake the credit, copyright and supplier transmissi…
tonytw1 Jan 24, 2026
8765512
XMP Credit -> photoshop:Credit
tonytw1 Feb 24, 2026
fd0666d
XMP Byline -> creator.
tonytw1 Feb 24, 2026
3fa195c
[libvips-cropping] While manually stripping metadata might be useful,…
tonytw1 Feb 12, 2026
197458b
[libvips-cropping] Use vip ffm 1.9.8 VImage set blob.
tonytw1 Jul 18, 2026
b7a47e8
[libvips-cropping] Additional resize tests.
tonytw1 Jan 31, 2026
afb36c1
[libvips-cropping] Test for correct resize of png with alpha.
tonytw1 Feb 16, 2026
264defd
[libvips-cropping] Provide an example of the vips can't render a LAB …
tonytw1 Jan 31, 2026
53cd197
[libvips-cropping] Cleanup; Lower the input of apiSource to imageId f…
tonytw1 Jan 26, 2026
07ba9ac
[libvips-cropping] resizeImageVips does not need the MasterCrop object
tonytw1 Mar 15, 2026
70d60e6
[libvips-cropping] Metadata for crop call.
tonytw1 Feb 1, 2026
e55fa09
[libvips-cropping] Crop metadata in XMP via vips removes exiftool as …
tonytw1 Jan 24, 2026
fc340fe
[libvips-cropping] createCrops moves to image operations for testing.
tonytw1 Jan 26, 2026
8827583
[libvips-cropping] Tests to exercise crops.
tonytw1 Jan 31, 2026
33a1bfa
[libvips-cropping] createCrops can execute it's resizeImage calls in …
tonytw1 Mar 15, 2026
c0de9d1
[libvips-cropping] Master crop save in parallel.
tonytw1 Jan 26, 2026
c8bd114
[libvips-cropping] Logging.
tonytw1 Jan 26, 2026
e74bed3
[libvips-cropping] Logging and imports.
tonytw1 Jan 26, 2026
cb52d99
[libvips-cropping] Do not send resizes until master has successfully …
tonytw1 Jan 27, 2026
ab19c45
[libvips-cropping] getImageInformation uses imageOperations.hasAlpha …
tonytw1 Feb 1, 2026
195dd7d
[libvips-cropping] Crops uses master hasAlpha image operation rather …
tonytw1 Feb 1, 2026
b4c0838
[libvips-cropping] Do not repeat calls to image_get_interpretation
tonytw1 Feb 24, 2026
57568dc
Removes exiftool from base image.
tonytw1 Feb 20, 2026
46df447
Remove pnquant from base image.
tonytw1 Jan 13, 2026
36d0981
CropsTest - wait for Future before closing arena!
tonytw1 Jul 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ lazy val commonLib = project("common-lib").settings(
// declare explicit dependency on desired version of aws sdk v2 bedrock runtime
"software.amazon.awssdk" % "bedrockruntime" % awsSdkV2Version,
"software.amazon.awssdk" % "s3vectors" % awsSdkV2Version,
"com.adobe.xmp" % "xmpcore" % "6.1.11",
ws,
"org.testcontainers" % "testcontainers-elasticsearch" % "2.0.2" % Test,
),
Expand Down

Large diffs are not rendered by default.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,6 @@ object ImageMagick extends GridLogging {
op
}

def runConvertCmd(op: IMOperation, useImageMagick: Boolean)(implicit logMarker: LogMarker): Future[Unit] = {
Stopwatch.async(s"Using ${if(useImageMagick) "imagemagick" else "graphicsmagick"} for imaging conversion operation '$op'") {
Future {
new ConvertCmd(!useImageMagick).run(op)
}
}
}

def runIdentifyCmd(op: IMOperation, useImageMagick: Boolean)(implicit logMarker: LogMarker): Future[List[String]] = {
Stopwatch.async(s"Using ${if (useImageMagick) "imagemagick" else "graphicsmagick"} for imaging identification operation '$op'") {
Future {
Expand Down
Binary file not shown.
25 changes: 25 additions & 0 deletions common-lib/src/test/resources/schaik.com_pngsuite/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Test PNG images
===============

These images are taken from http://www.schaik.com/pngsuite/pngsuite_bas_png.html

basn0g08 - 8 bit (256 level) grayscale
basn2c08 - 3x8 bits rgb color
basn3p08 - 8 bit (256 color) paletted
basn6a08 - 3x8 bits rgb color + 8 bit alpha-channel

LICENCE
-------

At the time of downloading these images the licence file at http://www.schaik.com/pngsuite/PngSuite.LICENSE contained the following text:

```
PngSuite
--------

Permission to use, copy, modify and distribute these images for any
purpose and without fee is hereby granted.


(c) Willem van Schaik, 1996, 2011
```
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
package com.gu.mediaservice.lib.imaging

import app.photofox.vipsffm.jextract.VipsRaw
import app.photofox.vipsffm.{VImage, Vips}
import app.photofox.vipsffm.Vips
import com.gu.mediaservice.lib.BrowserViewableImage
import com.gu.mediaservice.lib.logging.{LogMarker, MarkerMap}
import com.gu.mediaservice.model.{Bounds, Dimensions, ImageMetadata, Instance, Jpeg, OrientationMetadata, Png, Tiff}
import org.scalatest.time.{Millis, Span}
import com.gu.mediaservice.model._
import org.scalatest.concurrent.ScalaFutures
import org.scalatest.funspec.AnyFunSpec
import org.scalatest.matchers.should.Matchers
import org.scalatest.time.{Millis, Span}

import java.io.File
import java.lang.foreign.Arena
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{Await, Future}
import scala.concurrent.duration.{Duration, SECONDS}
import scala.concurrent.Future

// This test is disabled for now as it doesn't run on our CI environment, because GraphicsMagick is not present...
class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
Expand All @@ -24,6 +23,12 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
implicit override val patienceConfig: PatienceConfig = PatienceConfig(timeout = Span(1000, Millis), interval = Span(25, Millis))
implicit val logMarker: LogMarker = MarkerMap()

private val metadata = ImageMetadata(
credit = Some("Tony McCrae"),
copyright = Some("Eel Pie Consulting Ltd"),
suppliersReference = Some("eelpie-123")
)

describe("thumbnail") {
it("should write thumbnail to output file") {
val image = fileAt("IMG_4403.jpg")
Expand Down Expand Up @@ -99,6 +104,118 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
}
}

describe("resize") {
it("should output resized image to file in chosen format") {
implicit val arena: Arena = Arena.ofShared()
val fullSizedImage = VImage.newFromFile(arena, fileAt("IMG_4403.jpg").getAbsolutePath)
val imageOperations = new ImageOperations("")

val outputFile = new File("/Users/tony/Desktop/out5.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(1000, 800), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB colour spaces correctly in sRGB") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out6.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB colour spaces correctly as PNG") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out7.png")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Png)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB 16 bit colour spaces correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB16.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out8.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render PNG with alpha correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val image = fileAt("with-alpha.png")
val fullSizedImage = VImage.newFromFile(arena, image.getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/resized-png-with-alpha.png")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Png)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB TIFF with alpha correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val image = fileAt("lab8-with-alpha.tif")
val fullSizedImage = VImage.newFromFile(arena, image.getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out13.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}
}

describe("alpha") {
it("should return false for RGB for a Jpeg with no alpha") {
implicit val arena: Arena = Arena.ofShared
val image = VImage.newFromFile(arena, fileAt("rgb-wo-profile.jpg").getAbsolutePath)
val hasAlpha = ImageOperations.hasAlpha(image)
arena.close()
hasAlpha should be(false)
}

it("should return true for PNG with alpha") {
implicit val arena: Arena = Arena.ofShared
val image = VImage.newFromFile(arena, fileAt("with-alpha.png").getAbsolutePath)
val hasAlpha = ImageOperations.hasAlpha(image)
arena.close()
hasAlpha should be(true)
}
}

describe("identifyColourModel") {
it("should return RGB for a JPG image with RGB image data and no embedded profile") {
val image = fileAt("rgb-wo-profile.jpg")
Expand Down Expand Up @@ -230,7 +347,93 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
}
}

// TODO: test cropImage and its conversions
describe("graphic detection") {
it("should return not graphic for true colour jpeg") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("exif-orientated-no-rotation.jpg").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(false)
arena.close()
}

it("should return is graphic for depth 2 tiff") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("flower.tif").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

it("should return is graphic for depth 4 png with alpha") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("schaik.com_pngsuite/tbbn0g04.png").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

it("should return is graphic for depth 8 indexed png") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("schaik.com_pngsuite/basn3p08.png").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

}

describe("cropping") {
val operations = new ImageOperations("")

it("should create unscaled master crop to resize from full sized images") {
implicit val arena: Arena = Arena.ofConfined
//val fullsizedImage = fileAt("Lab 16bpc (7d0b7c7b8e890d7e5d369093aa437bd833e20f71).tiff")
val fullsizedImage = fileAt("IMG_4403.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 2000, 2400), metadata, None)

val outputFile = new File("/Users/tony/Desktop/master.jpg")
operations.saveImageToFile(masterCrop, Jpeg, 95, outputFile, keep = Some(VipsRaw.VIPS_FOREIGN_KEEP_XMP))
arena.close()
}

it("should create unscaled master crop from CMYK full sized image") {
implicit val arena: Arena = Arena.ofConfined
val fullsizedImage = fileAt("CMYK-with-profile.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 2000, 2400), metadata, None)

val outputFile = new File("/Users/tony/Desktop/master-from-cmyk.jpg")
operations.saveImageToFile(masterCrop, Jpeg, 95, outputFile, keep = Some(VipsRaw.VIPS_FOREIGN_KEEP_XMP))

arena.close()
}

it("should create files foreach crop size") {
implicit val arena: Arena = Arena.ofShared()
val fullsizedImage = fileAt("CMYK-with-profile.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 3000, 2000), metadata, None)
val landscapeCropSizingWidths = Seq(
Dimensions(140, 100),
Dimensions(320, 200),
Dimensions(800, 600),
Dimensions(1000, 1200),
Dimensions(2000, 3000),
)
implicit val i: Instance = Instance("id")

val eventualCrops = operations.createCrops(masterCrop, landscapeCropSizingWidths.toList, "test-image-id",
Bounds(0, 0, 1000, 1200),
Jpeg,
new File("/Users/tony/tmp/crops"),
75
)

whenReady(eventualCrops) { crops =>
arena.close()
}
}
}

def fileAt(resourcePath: String): File = {
new File(getClass.getResource(s"/$resourcePath").toURI)
Expand Down
2 changes: 0 additions & 2 deletions container-images/jdk-vips/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,6 @@ RUN rm /tmp/vips-8.18.4.tar.xz
RUN rm -r /tmp/vips-8.18.4/

RUN apt -y --no-install-suggests install \
pngquant \
libimage-exiftool-perl \
libjemalloc-dev \
graphicsmagick \
graphicsmagick-imagemagick-compat
Loading
Loading