Error diffusion using a second image as threshold

Error diffusion is a popular halftoning method for rendering a continuous-tone image on a binary device. It works by thresholding each pixel value with a constant (usually set to medium gray) to set the output pixel to 0 or 1. The error, or the difference between the output and in the input, is then diffused to the pixel's neighbors.
Original "Congress" 8-bit grayscale image.	Original "Mary" 8-bit grayscale image.
Halftone of "Congress" using a constant threshold.	Halftone of "Mary" using a constant threshold.
If we use a spatially varying threshold, such as a different image, the second image is visible in the final halftone.
Halftone of "Congress" using "Mary" as threshold.	Halftone of "Mary" using "Congress" as threshold.
Halftone of constant medium gray using "Congress" as threshold.	Halftone of constant medium gray using "Mary" as threshold.
A practical use of this might be like this. If we can estimate the illumination of a scene, then we can adjust for the illumination variation by varying the halftoning threshold
Original "grating" with even illumination, 8-bit grayscale image.	Halftone of "grating" using constant threshold.
Original "grating" with uneven illumination, 8-bit grayscale image.	Halftone of "grating" with uneven illumination using constant threshold.
Uneven illumination, 8-bit grayscale image.	Halftone of "grating" with uneven illumination using varying threshold derived from illumination.
But how do we estimate the illumination? One way is to estimate the local mid-gray of an illuminated surface. Then assume the illumination is twice the mid-gray. The local mid-gray can be formed by low pass filtering the image. Using this line of reasoning, we can use the lowpass version of an image as the threshold to halftone the image itself.
"Grating" with uneven illumination after lowpass filter, 8-bit grayscale image. The lowpass filter is implemented by 5 levels of an integer wavelet transform decomposition, followed by a reconstruction using the lowpass coefficients only. It corresponds roughly to a 32-tap filter in each dimension.	Halftone of "grating" with uneven illumination using the lowpass image as the threshold image. The illumination effect is mostly evened out but some errors, notably those in the heavy shadows, appear.
Estimate of illumination using a lowpass filter with lower cutoff frequency, 8-bit grayscale image. The lowpass filter is implemented by 6 levels of an integer wavelet transform decomposition, followed by a reconstruction using the lowpass coefficients only. It corresponds roughly to a 64-tap filter in each dimension for a 256x256 image. The illumination field is smoother.	Halftone of "grating" with uneven illumination using the lower pass filtered image as the threshold image. The errors in the heavy shadows got worse.
How well does it work for real-world images?
Original "King Cake" 8-bit grayscale image.	Halftone of "King Cake".
Estimate of illumination using a 6-level integer wavelet transform, 8-bit grayscale image.	Halftone of "King Cake" using the lower pass filtered image as the threshold image. Both highlights and shadows appear to be distorted.
The standard way to calculate error is by subtracting the quantized pixel value from the original pixel value. Since we are adjusting the level of medium gray, we also adjust the range of the error as the intensity range is adapted to the estimated medium gray.
	Halftone of "King Cake" using the lower pass filtered image as the threshold image. By controlling the error, the distortions in the highlight are lessened.
It seems reasonable that the locally estimated medium gray should have a value in a "gray" range; i.e., it should be clamped to not take on the extreme values of 0.0 and 1.0.
Halftone of "King Cake" using the lower pass filtered image as the threshold image. Threshold image clamped to [0.1, 0.9].	Halftone of "King Cake" using the lower pass filtered image as the threshold image. Threshold image clamped to [0.25, 0.75].
Halftone of "King Cake" using the lower pass filtered image as the threshold image. Threshold image clamped to [0.35, 0.65].	Halftone of "King Cake" using the lower pass filtered image as the threshold image. Threshold image clamped to [0.45, 0.55].

Last update: 12.III.02