extreme edge effects with Gaussian filters

[Jeremy Adler-2]

I was playing with Gaussian filters, using images with a few dots, and noticed an extreme edge effect.
Run the code.
The dot on the edge of the image and even more markedly the dot in the corner produce disproportionate Intensities (x5 and x30 for the edge and corner).
I was expecting an edge and corner effect but not of this magnitude.
The effect increases further with larger radii.
Note the edge effect vanishes for the dot that is one pixel from the edge.

As a work around I make the images two pixels wider before running a Gaussian, but I was expecting to reduce the problem by simply halving the intensities of edge pixels and reducing corner pixels by 4.

 // Gaussian filter edge problem
sz=64;
newImage("Untitled", "32-bit black", sz, sz, 1); setPixel(0,0,1);// corner setPixel(sz-1,sz/2,1);// edge setPixel(1,sz/2,1);// 1 pixel from edge setPixel(sz/2,sz/2,1);// centre run("Gaussian Blur...", "sigma=4"); run("16_colors"); getRawStatistics(n,mea,min,max); setMinAndMax(0,max/4);

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Jeremy,

the phenomenon you observe is due to the handling of the image borders
in ImageJ:

If you have an operation that takes the pixels in the neighborhood into
account (such as Gaussian Blur), you need a convention how to
extrapolate what the pixels outside the image borders would be.

ImageJ uses the convention that the border pixels are repeated.  In
other words, it assumes that out-of-image pixels have a value equal to
the nearest edge pixel.  This is the same as in most other image
processing programs that I am aware of, also Adobe Photoshop does the
same (some programs give you a choice which extrapolation method to use).

So, if you have an outlier pixel at the left or right edge, it is taken
as a whole row with the same pixel value reaching from the border to
infinity.  It is even worse for corner pixels, they determine the value
of a square-like area from the corner to infinity.  This gives them a
large weight in averaging operations.  As a result, if the blur radius
(sigma) of the Gaussian Blur is orders of magnitude larger than the
image size, all pixels will be essentially set to the average of the
four corner pixels.

There are several other conventions for extrapolation like mirroring,
periodic boundary conditions, or assuming a fixed value.  Each of them
has advantages and disadvantages; there is no 'universal' method for all
purposes.

Implementing these methods was discussed in this mailing list already
many years ago (I don't find the thread any more).  It would be rather
easy to do this in the RankFilters (Mean, Minimum, Maximum, Median,
variance), but doing so for the GaussianBlur would be a bit more tricky
(except for the fixed value).  Unfortunately, no one found time to do so
yet (whereby I would probably be the person who could do it most easily,
but sorry, I have too much work in my main occupation to do it within
reasonable time).

So, if you want the 'constant value' extrapolation, the easiest option
is what you are doing already, enlarging the image canvas by one pixel
on each side, doing the operation, and cropping the image to the
original size.

Another option to alleviate the issue would be running 'Fast Filters'
plugin [1] with 'border-limited mean' and a radius of roughly 1/4 of the
sigma as a preprocessing step, and then apply a Gaussian Blur.  It won't
affect the radius (sigma) of the Gaussian Blur very much.

Another option might be a Gaussian filter in Fourier domain.  If I
remember it correctly, the FFT 'Custom filter' uses a combination of
mirroring and periodic boundary conditions.


Michael


[1] http://imagejdocu.tudor.lu/doku.php?id=plugin:filter:fast_filters:start
________________________________________________________________
On 17/08/2017 11:15, Jeremy Adler wrote:
 > I was playing with Gaussian filters, using images with a few dots,
and noticed an extreme edge effect.
 > Run the code.
 > The dot on the edge of the image and even more markedly the dot in
the corner produce disproportionate Intensities (x5 and x30 for the edge
and corner).
 > I was expecting an edge and corner effect but not of this magnitude.
 > The effect increases further with larger radii.
 > Note the edge effect vanishes for the dot that is one pixel from the
edge.
 >
 > As a work around I make the images two pixels wider before running a
Gaussian, but I was expecting to reduce the problem by simply halving
the intensities of edge pixels and reducing corner pixels by 4.
 >
 >   // Gaussian filter edge problem
sz=64;
newImage("Untitled", "32-bit black", sz, sz, 1);
setPixel(0,0,1);// cornersetPixel(sz-1,sz/2,1);// edge
setPixel(1,sz/2,1);// 1 pixel from edge
setPixel(sz/2,sz/2,1);// centre
run("Gaussian Blur...", "sigma=4");
run("16_colors");
getRawStatistics(n,mea,min,max);
setMinAndMax(0,max/4);

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ctrueden]

Hi Michael & everyone,

> There are several other conventions for extrapolation like mirroring,
> periodic boundary conditions, or assuming a fixed value.  Each of
> them has advantages and disadvantages; there is no 'universal'
> method for all purposes.

Indeed. ImgLib2 (and therefore ImageJ2) supports several different
out-of-bounds projections, as well as the ability to define your own
extensibly:


The code for generating these can be found in the "Introduction to ImageJ
Ops" tutorial notebook in the "Padding an image" section [1].

> Implementing these methods was discussed in this mailing list already
> many years ago (I don't find the thread any more).  It would be rather
> easy to do this in the RankFilters (Mean, Minimum, Maximum, Median,
> variance), but doing so for the GaussianBlur would be a bit more
> tricky (except for the fixed value).  Unfortunately, no one found time
> to do so yet (whereby I would probably be the person who could do it
> most easily, but sorry, I have too much work in my main occupation
> to do it within reasonable time).

I do not think that changing all the ImageJ1 algorithms is warranted here.
It is better to have general-purpose out-of-bounds padding support, and
then algorithms can be fed a padded image directly to use as needed. All of
ImageJ Ops works this way. Note that the image bounds do not change—it just
becomes possible to query sample values outside of the bounds once it has
been padded. So algorithms no longer need to explicitly code around
boundary conditions.

Regards,
Curtis

[1] ​https://imagej.github.io/tutorials/


--
Curtis Rueden
LOCI software architect - https://loci.wisc.edu/software
ImageJ2 lead, Fiji maintainer - https://imagej.net/User:Rueden
Did you know ImageJ has a forum? http://forum.imagej.net/


On Fri, Aug 18, 2017 at 3:45 AM, Michael Schmid <[hidden email]>
wrote:

> Hi Jeremy,
>
> the phenomenon you observe is due to the handling of the image borders in
> ImageJ:
>
> If you have an operation that takes the pixels in the neighborhood into
> account (such as Gaussian Blur), you need a convention how to extrapolate
> what the pixels outside the image borders would be.
>
> ImageJ uses the convention that the border pixels are repeated.  In other
> words, it assumes that out-of-image pixels have a value equal to the
> nearest edge pixel.  This is the same as in most other image processing
> programs that I am aware of, also Adobe Photoshop does the same (some
> programs give you a choice which extrapolation method to use).
>
> So, if you have an outlier pixel at the left or right edge, it is taken as
> a whole row with the same pixel value reaching from the border to
> infinity.  It is even worse for corner pixels, they determine the value of
> a square-like area from the corner to infinity.  This gives them a large
> weight in averaging operations.  As a result, if the blur radius (sigma) of
> the Gaussian Blur is orders of magnitude larger than the image size, all
> pixels will be essentially set to the average of the four corner pixels.
>
> There are several other conventions for extrapolation like mirroring,
> periodic boundary conditions, or assuming a fixed value.  Each of them has
> advantages and disadvantages; there is no 'universal' method for all
> purposes.
>
> Implementing these methods was discussed in this mailing list already many
> years ago (I don't find the thread any more).  It would be rather easy to
> do this in the RankFilters (Mean, Minimum, Maximum, Median, variance), but
> doing so for the GaussianBlur would be a bit more tricky (except for the
> fixed value).  Unfortunately, no one found time to do so yet (whereby I
> would probably be the person who could do it most easily, but sorry, I have
> too much work in my main occupation to do it within reasonable time).
>
> So, if you want the 'constant value' extrapolation, the easiest option is
> what you are doing already, enlarging the image canvas by one pixel on each
> side, doing the operation, and cropping the image to the original size.
>
> Another option to alleviate the issue would be running 'Fast Filters'
> plugin [1] with 'border-limited mean' and a radius of roughly 1/4 of the
> sigma as a preprocessing step, and then apply a Gaussian Blur.  It won't
> affect the radius (sigma) of the Gaussian Blur very much.
>
> Another option might be a Gaussian filter in Fourier domain.  If I
> remember it correctly, the FFT 'Custom filter' uses a combination of
> mirroring and periodic boundary conditions.
>
>
> Michael
>
>
> [1] http://imagejdocu.tudor.lu/doku.php?id=plugin:filter:fast_
> filters:start
> ________________________________________________________________
> On 17/08/2017 11:15, Jeremy Adler wrote:
> > I was playing with Gaussian filters, using images with a few dots, and
> noticed an extreme edge effect.
> > Run the code.
> > The dot on the edge of the image and even more markedly the dot in the
> corner produce disproportionate Intensities (x5 and x30 for the edge and
> corner).
> > I was expecting an edge and corner effect but not of this magnitude.
> > The effect increases further with larger radii.
> > Note the edge effect vanishes for the dot that is one pixel from the
> edge.
> >
> > As a work around I make the images two pixels wider before running a
> Gaussian, but I was expecting to reduce the problem by simply halving the
> intensities of edge pixels and reducing corner pixels by 4.
> >
> >   // Gaussian filter edge problem
> sz=64;
> newImage("Untitled", "32-bit black", sz, sz, 1);
> setPixel(0,0,1);// cornersetPixel(sz-1,sz/2,1);// edge
>
> setPixel(1,sz/2,1);// 1 pixel from edge
> setPixel(sz/2,sz/2,1);// centre
> run("Gaussian Blur...", "sigma=4");
> run("16_colors");
> getRawStatistics(n,mea,min,max);
> setMinAndMax(0,max/4);
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html