Reference

Package for computing the Structural Similarity Index Measure (SSIM) and its gradients on NVIDIA GPUs.

Quick start

julia> using FastCUDASSIM, CUDA

julia> img1 = CUDA.rand(Float32, 3, 512, 768); img2 = similar(img1);

julia> ssim(img1, img2)  # Actual value depends on RNG above
4.7778763f-6

Key points

• CUDA only
• We use zero-padding in the convolutions. The convolution kernel is the typical Gaussian with standard deviation $\sigma = 1.5$ and window size 11 x 11. We represent it using Float32.
• We expect Float-like image intensities in [0, 1]. This includes N0f8.
• Internally we use channels x height x width x batch size format for the input image batches, and height x channels x width x batch size for internal buffers (N_dssims_dQ and friends). Supplied inputs need not have this explicit size, but must have an equivalent memory layout.

Image formats

• To expand on the last point, consider a CuMatrix{RGB{N0f8}} of size (256, 384). It is equivalent (via channelview) to a CuArray{N0f8, 3} of size (3, 256, 384), and (via a further reshape) to a CuArray{N0f8, 4} with size (3, 256, 384, 1). We will perform these conversions automatically. Therefore, this input is supported.
• In these conversions we will (if necessary) first expand the channels dimension, followed by the batch dimension. Therefore, a CuArray{Float32} of size (x, y, z) will be considered to be a single x-channel image of height y and width z. To input a grayscale batch consisting of z images of size (x, y), either use a CuArray{Gray{Float32}, 3} with size (x, y, z), or explicitly set the number of color channels to 1 via a CuArray{Float32, 4} of size (1, x, y, z).
• When we internally allocate an output (D)SSIM or gradient batch (either because an allocating method like dssim_with_gradient is called, or because in one of the mutating methods like dssim_with_gradient! the supplied value was nothing), we will convert this back to a format similar to that of the first input image (batch). For example, if no batch axis appears here, then we will return a scalar (D)SSIM value instead of a CuVector of length 1. There might be some promotions, though. For example, ssim_gradient(imgs1, imgs2) for imgs1 a CuArray{RGB{N0f8}, 3} of size (256, 384, 16) and imgs2 similar, will return a CuArray{RGB{Float32}, 3} gradient batch, again of size (256, 384, 16).

dssim(
    imgs1, imgs2, 
    divide_by_two_in_dssim = false
)

Compute the Structural DisSimilarity Index Measure (DSSIM) values between two image batches.

Like dssim_with_gradient, but without the gradient.

See also ssim, and dssim! for the non-allocating version of dssim.

dssim!(
    dssims, 
    imgs1, imgs2, 
    should_zero = true,
    divide_by_two_in_dssim = false
)
-> dssims

Compute the DSSIMs between two image batches.

Like ssim!, but for the structural dissimilarity DSSIM = 1 - SSIM if !divide_by_two_in_dssim (the default) or DSSIM = (1 - SSIM) / 2 otherwise.

See also dssim_with_gradient!.

dssim_gradient(
    imgs1, imgs2, 
    divide_by_two_in_dssim = false
)

Compute the gradients of the DSSIMs between two image batches with respect to the first.

Like dssim_with_gradient, but does not compute the DSSIM itself.

Similar to ssim_gradient, but for the DSSIM = 1 - SSIM when !divide_by_two_in_dssim (the default), or DSSIM = (1 - SSIM) / 2 otherwise. This method allocates (and frees) a number of buffers and of course also allocates the output gradient 'image' batch. See dssim_gradient! for the non-allocating version.

dssim_gradient!(
    dL_dimgs1, 
    imgs1, imgs2,
    N_dssims_dQ, N_dssims_dM, N_dssims_dP,
    divide_by_two_in_dssim = false
)
-> dL_dimgs1

Compute the gradients of the DSSIMs between two image batches with respect to the first one.

Like dssim_with_gradient!, but without the computation of the DSSIM itself, and like ssim_gradient!, but for the DSSIM = 1 - SSIM when !divide_by_two_in_dssim (the default) or DSSIM = (1 - SSIM) / 2 otherwise.

dssim_with_gradient(
    imgs1, imgs2, 
    divide_by_two_in_dssim = false
)

Compute the DSSIMs between two image batches, as well as their gradients with respect to the first one.

Like ssim_with_gradient, but for the structural dissimilarity DSSIM = 1 - SSIM when !divide_by_two_in_dssim (the default), or DSSIM = (1 - SSIM) / 2 otherwise. See also dssim_with_gradient! for the non-allocating version.

dssim_with_gradient!(
    dssims, dL_dimgs1, 
    imgs1, imgs2,
    N_dssims_dQ, N_dssims_dM, N_dssims_dP,
    should_zero = true,
    divide_by_two_in_dssim = true
)
-> (dssims, dL_dimgs1)

Compute the DSSIMs between two image batches, as well as the gradients with respect to the first one.

Like ssim_with_gradient!, but for the structural dissimilarity DSSIM = 1 - SSIM when !divide_by_two_in_dssim (by default), or DSSIM = (1 - SSIM) / 2.

ssim!(
    ssims, 
    imgs1, imgs2, 
    should_zero = true
)
-> ssims

Compute the SSIMs between two image batches.

Like ssim_with_gradient! but does not compute the gradient. See this function for the meaning of the arguments.

ssim(imgs1, imgs2)

Compute the Structural Similarity Index Measure (SSIM) values between two image batches.

Like ssim_with_gradient, but without the gradient.

See the module documentation for the input and output formats. Also see ssim! for the non-allocating version.

ssim_gradient!(
    dL_dimgs1, 
    imgs1, imgs2,
    N_dssims_dQ, N_dssims_dM, N_dssims_dP,
)
-> dL_dimgs1

Compute the gradients of the SSIM between two image batches with respect to the first one.

Like ssim_with_gradient!, but without the computation of the SSIM itself.

ssim_gradient(imgs1, imgs2)

Compute the gradients of the SSIMs between two image batches with respect to the first.

Like ssim_with_gradient, but does not compute the SSIM itself.

This method allocates (and frees) a number of buffers and of course also allocates the output gradient 'image' batch. See ssim_gradient! for the non-allocating version.

ssim_with_gradient!(
    ssims, dL_dimgs1, 
    imgs1, imgs2,
    N_dssims_dQ, N_dssims_dM, N_dssims_dP,
    should_zero = true
)
-> (ssims, dL_dimgs1)

Compute the SSIMs between two image batches, as well as the gradients with respect to the first one.

See the module documentation for the input and output formats.

Arguments

• ssims: The (output) CuVector which will contain the SSIM values.
• dL_dimgs1: The (output) gradients of imgs1 with respect to the SSIM value L.
• imgs1: The first image batch, with respect to which we compute the gradients.
• imgs2: The second image batch.
• N_dssims_dQ, N_dssims_dM and N_dssims_dP: Internal buffers for intermediate gradients.
• should_zero: To get the correct SSIM value we need ssims to start with values of 0. When set to true we will perform this zeroing inside the method. Otherwise it is the responsibility of the caller to have done this in advance. Defaults to true.

Returns

(ssims, dL_dimgs1)

ssim_with_gradient(imgs1, imgs2)

Compute the SSIMs between two image batches, as well as the gradients with respect to the first one.

This method allocates (and frees) a number of buffers and of course also allocates the output gradient 'image' batch. See ssim_with_gradient! for the non-allocating version.

See the module documentation for expected formats for the inputs and output.

Arguments

• imgs1: The first image batch.
• imgs2: The second image batch.

Returns

1. The SSIM values.
2. The gradients of these values with respect to imgs1.