How are images larger than the display window rendered?

[Conner Phillips]

Greetings,

I am trying to compare a program written in LabVIEW with ImageJ. LabVIEW seems to do something odd when rendering images which are larger than their displaying window. I found an unverified mention that the largest value in a neighborhood is taken when downsizing to fit the image into the window (intensity chart). ImageJ seems to render the same ImageJ just fine though. How does ImageJ decided on the pixel values when resampling a large image to fit into the display window?

Thanks!

--
Fraunhofer-Institut für Keramische Technologien und Systeme IKTS
Conner Phillips
Charakterisierungsverfahren | Characterization Technologies
Maria-Reiche-Straße 2, 01109 Dresden, Germany
<https://s.fhg.de/ikts-klotzsche>Telefon [phone]
Fax [phone]
[hidden email]
<mailto:[hidden email]>www.ikts.fraunhofer.de
<http://www.ikts.fraunhofer.de/>
Folgen Sie uns | Follow us:
Twitter<https://twitter.com/FraunhoferIKTS/> | Facebook<http://www.facebook.com/fraunhofer.ikts/> | Instagram<https://www.instagram.com/fraunhoferikts/> | YouTube<https://s.fhg.de/ikts-youtube> | XING<https://www.xing.com/companies/fraunhoferikts> | LinkedIn<https://www.linkedin.com/company/fraunhofer-ikts-dresden> | IKTS-Blog<https://www.ikts.fraunhofer.de/de/blog.html>


--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Conner,

for displaying large images at zoom levels below 100%, ImageJ uses a
mixture of subsampling (i.e., pixels are omitted) and averaging.
This is actually done by Java, so it may depend on the version of the
Java virtual machine (JVM) that you use. ImageJ just specifies
'RenderingHints.VALUE_RENDER_QUALITY', i.e., Java should optimize for
quality, not speed.

You can check the method by drawing a 1-pixel wide line under a small
angle from horizontal or vertical on a blank image. When the image is
zoomed out, in case of subsampling the line will become a dashed line.

On my machine (Oracle Java 1.8.0_112 on Linux), for zoom levels from 50
to 99%, the displayed image uses averaging, i.e. a black line becomes
gray. At lower zoom levels, a mixture of averaging and subsampling is
used: Averaging of 2x2 pixel blocks and then subsampling. Thus, at 25%
zoom level, the dashes of a black line on white background have a mid
gray level and a "fill factor" of 1/2, i.e., the dashes and spaces have
the same length. At 12.5% the fill factor decreases to 1/4.
(Strictly speaking, since the display is nonlinear, one should not use a
mid gray level like 128 for the average of two pixels with 255 and two
with 0 but take the Gamma value of the display into account. At least my
Java version does not do this.)

---
By the way, for zooming in, there is an option whether to interpolate
(smooth the pixels) in Edit>Options>Appearance.


Michael
________________________________________________________________
On 31.05.21 15:17, Phillips, Conner wrote:

> Greetings,
>
> I am trying to compare a program written in LabVIEW with ImageJ. LabVIEW seems to do something odd when rendering images which are larger than their displaying window. I found an unverified mention that the largest value in a neighborhood is taken when downsizing to fit the image into the window (intensity chart). ImageJ seems to render the same ImageJ just fine though. How does ImageJ decided on the pixel values when resampling a large image to fit into the display window?
>
> Thanks!
>
> --
> Fraunhofer-Institut für Keramische Technologien und Systeme IKTS
> Conner Phillips
> Charakterisierungsverfahren | Characterization Technologies
> Maria-Reiche-Straße 2, 01109 Dresden, Germany
> <https://s.fhg.de/ikts-klotzsche>Telefon [phone]
> Fax [phone]
> [hidden email]

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Kenneth Sloan-2]

Well..."strictly speaking" the display software should gamma correct the original (non-zoomed) version of the image, too.  I don't think that is done, so I wouldn't worry about how averaging is done, either.

Now...suppose I have applied a gamma-correcting LUT.  Does the averaging process deal with the original data (in which case, the LUT will continue to do the right thing) or does it deal with the pixel values AFTER transformation by the LUT?

Same question for the case when the LUT colorizes a scalar image - are the *colors* averaged, or the scalar pixel values?

--
Kenneth Sloan
[hidden email]
Vision is the art of seeing what is invisible to others.





> On May 31, 2021, at 09:17, Michael Schmid <[hidden email]> wrote:
>  
> (Strictly speaking, since the display is nonlinear, one should not use a mid gray level like 128 for the average of two pixels with 255 and two with 0 but take the Gamma value of the display into account. At least my Java version does not do this.)
 
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Kenneth,

you can easily try it yourself, by drawing a 1-pixel wide line at some
angle close to horizontal or vertical and zooming out.

At least my version of Java (Oracle Java 1.8) does 2x2 averaging of the
displayed colors, not the pixel values when zooming out.

When using a color LUT, this can also make some sense:
If I have, e.g. the 'Fire LUT' and the line appears purple at pixel
value 64, on a background that is while (pixel value 255), one may
expect that the zoomed-out line does not appear orange (corresponding to
pixel value ~160) but a lighter shade of purple.

Similarly, if the image shows noise (with about equal occurrence of all
valeus between 0 and 255), and a spectrum LUT is applied, one may expect
less saturated colors of all different hue values, not everything in
blue-turquioise shades corresponding to mid-values, which would be the
result of averaging pixel values.

Of course, with an image using indexed colors (such as graphics saved as
GIF), where the colors do not represent a gradual variations of some
numeric value, the only way to go is averaging colors, not index values.

Concerning the details of the (2x2) averaging:

Looking more closely, it seems that this is not really averaging but
just a result of interpolation: Java seems to calculate the coordinate
that the pixel center of the scaled image would have in the original
image. If it falls onto a pixel center, the pixel coordinate is used. If
if falls in between two pixels, linear interpolation of the colors is used.
With a zoom level of 50%, 25%, etc., the coordinates of the pixel
centers in the zoomed image are always at pixel boundaries of the
original image (when taking the pixels as small squares). This is what
causes the averaging.

If the zoom level is different, this can result in interpolation or not.
E.g., at zoom levels of 100/3 ~ 33.33333% and 100/5 = 20% zoom, the
zoomed pixel centers would coincide with centers of the original pixels.
Indeed, in this case I see no averaging but only sub-sampling.

Michael
________________________________________________________________
On 01.06.21 00:02, Kenneth Sloan wrote:

> Well..."strictly speaking" the display software should gamma correct the original (non-zoomed) version of the image, too.  I don't think that is done, so I wouldn't worry about how averaging is done, either.
>
> Now...suppose I have applied a gamma-correcting LUT.  Does the averaging process deal with the original data (in which case, the LUT will continue to do the right thing) or does it deal with the pixel values AFTER transformation by the LUT?
>
> Same question for the case when the LUT colorizes a scalar image - are the *colors* averaged, or the scalar pixel values?
>
> --
> Kenneth Sloan
> [hidden email]
> Vision is the art of seeing what is invisible to others.
>
>
>
>
>
>> On May 31, 2021, at 09:17, Michael Schmid <[hidden email]> wrote:
>>  
>> (Strictly speaking, since the display is nonlinear, one should not use a mid gray level like 128 for the average of two pixels with 255 and two with 0 but take the Gamma value of the display into account. At least my Java version does not do this.)
>  

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Kenneth Sloan-2]

I'm not sure that *either* option is  "natural" or "expected" by everyone.

My first instinct is that I would prefer averaging the original pixel values, at least as a default.

For example, averaging gamma-corrected values is not correct - whether the image values themselves are gamma corrected or if the gamma-correction is done by a LUT.

For gray-scale images mapped with a LUT to colors - I would prefer to see the average grayscale value, which is THEN mapped through the LUT.  But, this depends on where the averaging is in the pipeline.

Finally, "averaging" is the first thing that occurs to someone used to processing linear, gray-scale images.  For anything more complex, it might be preferable to select the MEDIAN display value (or, better yet, the MODE - which does not depend on the values having any sort of natural ordering).

For example, I often use color LUT's with a small number of colors to sharpen boundaries.  Averaging defeats this and introduces displayed pixel colors which do not appear anywhere in the LUT.  I  consider this  *almost* as bad as averaging index values (which, I agree, is completely wrong).

Question: when I move the cursor over a down-sized image - does ImageJ report the original underlying pixel values?  If so, which value does it select from the region averaged into the screen pixel?

--
Kenneth Sloan
[hidden email]
Vision is the art of seeing what is invisible to others.

> On Jun 1, 2021, at 06:21, Michael Schmid <[hidden email]> wrote:
>
> Hi Kenneth,
>
> you can easily try it yourself, by drawing a 1-pixel wide line at some angle close to horizontal or vertical and zooming out.
>
> At least my version of Java (Oracle Java 1.8) does 2x2 averaging of the displayed colors, not the pixel values when zooming out.
>
> When using a color LUT, this can also make some sense:
> If I have, e.g. the 'Fire LUT' and the line appears purple at pixel value 64, on a background that is while (pixel value 255), one may expect that the zoomed-out line does not appear orange (corresponding to pixel value ~160) but a lighter shade of purple.
>
> Similarly, if the image shows noise (with about equal occurrence of all valeus between 0 and 255), and a spectrum LUT is applied, one may expect less saturated colors of all different hue values, not everything in blue-turquioise shades corresponding to mid-values, which would be the result of averaging pixel values.
>
> Of course, with an image using indexed colors (such as graphics saved as GIF), where the colors do not represent a gradual variations of some numeric value, the only way to go is averaging colors, not index values.
>
> Concerning the details of the (2x2) averaging:
>
> Looking more closely, it seems that this is not really averaging but just a result of interpolation: Java seems to calculate the coordinate that the pixel center of the scaled image would have in the original image. If it falls onto a pixel center, the pixel coordinate is used. If if falls in between two pixels, linear interpolation of the colors is used.
> With a zoom level of 50%, 25%, etc., the coordinates of the pixel centers in the zoomed image are always at pixel boundaries of the original image (when taking the pixels as small squares). This is what causes the averaging.
>
> If the zoom level is different, this can result in interpolation or not. E.g., at zoom levels of 100/3 ~ 33.33333% and 100/5 = 20% zoom, the zoomed pixel centers would coincide with centers of the original pixels. Indeed, in this case I see no averaging but only sub-sampling.
>
> Michael
> ________________________________________________________________
> On 01.06.21 00:02, Kenneth Sloan wrote:
>> Well..."strictly speaking" the display software should gamma correct the original (non-zoomed) version of the image, too.  I don't think that is done, so I wouldn't worry about how averaging is done, either.
>> Now...suppose I have applied a gamma-correcting LUT.  Does the averaging process deal with the original data (in which case, the LUT will continue to do the right thing) or does it deal with the pixel values AFTER transformation by the LUT?
>> Same question for the case when the LUT colorizes a scalar image - are the *colors* averaged, or the scalar pixel values?
>> --
>> Kenneth Sloan
>> [hidden email]
>> Vision is the art of seeing what is invisible to others.
>>> On May 31, 2021, at 09:17, Michael Schmid <[hidden email]> wrote:
>>>  (Strictly speaking, since the display is nonlinear, one should not use a mid gray level like 128 for the average of two pixels with 255 and two with 0 but take the Gamma value of the display into account. At least my Java version does not do this.)
>>  
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html