Autothresholding results; (was FW: Threshold is giving an incomplete range ?)

[robert atwood]

Hi, Gabriel, Wayne , all,


I'm sorry if this goes a bit beyond ImageJ .. for now..  I'll take it off list if necessary .. ? But it is all related to the auto thresholding results.



Once starting to look more closely at the exact values of the threshold I find some things I don't quite understand.

There is not a particular requirement for Huang' method, I used it as an example because it immediately showed the issue of the overlay.

 But following on from your observation that the reported value from the 8bit implementation is naturally different from the reported value of the 16 bit implementation, I then tried using other software to check what results occur.  But, I started using Otsu because it's faster and I believe less complicated. (and doesn't crash the plugin .. see below)

Using my generated image from slice 1 of the macro stack, and also the same image after Gaussian Blur 2 or even 10  to get rid of the unrealistic small number of full histogram bins, , I used SimpleITK in which the implementation is supposed to be derived from Gabriel's

 "All classes are largely derived from the AutoThresh [4] package for ImageJ. ... [4]  G. Landini et al, . Auto threshold. "
( from https://blog.kitware.com/histogram-based-thresholding/ )

However it also allows selection of number of bins to form the histogram, so seems good for trying to understand the practical effect of reducing the histogram.

But..
Using 'Otsu' I find  ---  not a little difference ---  but a big one. A little difference caused by the binning, a big one between implementations. Which is the 'real' Otsu threshold? or which different assumptions  can make this difference? The Otsu method seems well defined and established for many years so I wouldn't expect some ambiguity about what the result should be for a specific image and number of bins?

The other implementations (sItk, Applet) approach a similar value as the image gets more continuously varying, it seems, except the AutoThresh one gives the same value for each level of blurring?

Also, for some reason Autothresh -> Huang seems to consistently give a segfault in processing the one with Gaussian blur 10 ?  blur10blob16.tif ? I even rolled back from the daily build to the release 1.51n and find this  (on a Windows workstation -- haven't tried the  Linux yet)

I'd follow up to both the lists ImageJ and ITK  if I can understand a bit of why it should (or shouldn't ! )  be different?

 
'Otsu' method from each package/plugin:

Unblurred: nonblurbob16.tif

ImageJ applet 64037
AuthoThresh  64249

SimpleITK with 256 bins 15424
SimpleITK with 65536 bins 15419

Blurred radius 2:  blur2blob16.tif

ImageJ applet 40645
AuthoThresh  64249

SimpleITK with 256 bins 38938
SimpleITK with 65536 bins 38939

Blurred radius 10: blur10blob16.tif

ImageJ applet 42319
AutoThresh 64249

SimpleITK with 256 bins 42301
SimpleITK with 65536 bins 42452


>>>>>>>>simpleITk python script
import SimpleITK as sitk
import numpy as np

fname="nonblurblob.tif" #alter this line of code to try the blurred input file or other file
cropim=sitk.ReadImage(fname)

nbins=65536 #altered this line of code  to try 256 bins

othresher=sitk.OtsuThresholdImageFilter() #need to use the Object interface to expose GetThreshold() method

#as far as I understand, the additional parameters 0,255 define the output FG and BG colours, #and 'False' means don't apply any input masking; final 0 is the placeholder for the mask value but it's not used if masking isFalse.
#I don't see equivalent of 'ignore black/white' options, but this test image does not have 0 or 65535  values in it.

autothim=othresher.Execute(cropim,0,255,nbins,False,0)  


print "... finished!"
sitk.WriteImage(autothim,"otsuautoth.tif")
oth=othresher.GetThreshold()
print "otsu th %i "%nbins,oth

<<<<<<<<<<<<<<<<<<<<<<<<<<<<

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Gabriel Landini
Sent: Thursday, May 18, 2017 17:42
To: [hidden email]
Subject: Re: Threshold is giving an incomplete range ?

On Thursday, 18 May 2017 15:08:22 BST you wrote:
> But it is sure incredibly slow

Yep, it is a 2^16 bin histogram vs a 2^8 one... and most methods are iterative. That is why the binning is quite convenient for speeding it up, but as you found out it has its side effects.

> and doesn't help with  the observation that the red overlay does not
> match the reported value from the IJ builtin Huang method -- because
> there is no preview overlay!

16bit overlays are not supported in IJ. I do not think there is a  way of currently generating such a LUT table and display it.
The set threshold in that plugin currently works on 8 bit images only. When I get some time I will look into this.

> But, it doesn't work suitable for my intended use , which contains the
> following stanza, allowing the user to 'improve' upon the
> autothreshold before applying binarization, because it doesn't even
> provide a preview overlay at all.

Yes, I see, that is not currently possible. 32bit images are not supported either.

> There's a few other problems with it also...
I would be happy to know which ones, it would be useful if it can be improved.

> I don't fundamentally have a problem with the result being altered
> from the precise value  by 8bit reduction ... if it is consistent, but
> the problem now is the overlay doesn't match the actual result.

Going back to the built-in threshold applet, I did not quite understand  why does the 'stack histogram' option is not a suitable solution. It avoids the problem you found last too (or maybe I missed something?).

Regards
Gabriel

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Gabriel Landini]

On Friday, 19 May 2017 11:19:21 BST Robert Atwood wrote:
> Using 'Otsu' I find  ---  not a little difference ---  but a big one. A
> little difference caused by the binning, a big one between implementations.
> Which is the 'real' Otsu threshold? or which different assumptions  can
> make this difference? The Otsu method seems well defined and established
> for many years so I wouldn't expect some ambiguity about what the result
> should be for a specific image and number of bins?

> The other implementations (sItk, Applet) approach a similar value as the
> image gets more continuously varying, it seems, except the AutoThresh one
> gives the same value for each level of blurring?

Hi Robert,
The same Otsu's routine in the Auto_threshold plugin is used for both the 8
and the 16bit images.
So wouldn't the differences in the granularity of the histograms account for
the inter-class variances ending up being not-directly equivalent  in 256 and
in a 2^16 greylevel versions of an image? I suppose that something similar
happens with other methods. For example a minimum valley search does not
guarantee that a coarser histogram has a minimum in that same position.

> Also, for some reason Autothresh -> Huang seems to consistently give a
> segfault in processing the one with Gaussian blur 10 ?

Yes, I can reproduce it in that image. Had a look at the source, seems to be
an array dimension issues. The original code ported from Emre Celebi's
implementation was re-written by Johannes Schindelin in 2011 to make it more
efficient in handling 16bit data.  I will compare the old and new code results.
Thanks for reporting these issues. The IJ applet still uses E Celebi's code.

I see that the itk plugin appears similar to Johannes's code but it uses a bin
multiplier to re-scale the threshold value between the min and max of the
image. That might be the reason for the differences across implementations.

Regards

Gabriel

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[robert atwood]

THere are small differences depending on binning eg 15254 vs 15419 , that should be explained by the histogram
granularity, sure,

but the difference between 40645 and 64249 for blur2blob16.tif ? and the complete unchanging result of autoThresh->Otsu regardless of blurring the image? I'm not feelign competent to re-program the method myself to see which is right but they shouldnt really be that drastically different in behavour shoudl they?



________________________________________
From: Gabriel Landini [[hidden email]]
Sent: 19 May 2017 17:14
To: Atwood, Robert (DLSLtd,RAL,SCI)
Cc: [hidden email]
Subject: Re: Autothresholding results;  (was FW: Threshold is giving an incomplete range ?)

On Friday, 19 May 2017 11:19:21 BST Robert Atwood wrote:
> Using 'Otsu' I find  ---  not a little difference ---  but a big one. A
> little difference caused by the binning, a big one between implementations.
> Which is the 'real' Otsu threshold? or which different assumptions  can
> make this difference? The Otsu method seems well defined and established
> for many years so I wouldn't expect some ambiguity about what the result
> should be for a specific image and number of bins?

> The other implementations (sItk, Applet) approach a similar value as the
> image gets more continuously varying, it seems, except the AutoThresh one
> gives the same value for each level of blurring?

Hi Robert,
The same Otsu's routine in the Auto_threshold plugin is used for both the 8
and the 16bit images.
So wouldn't the differences in the granularity of the histograms account for
the inter-class variances ending up being not-directly equivalent  in 256 and
in a 2^16 greylevel versions of an image? I suppose that something similar
happens with other methods. For example a minimum valley search does not
guarantee that a coarser histogram has a minimum in that same position.

> Also, for some reason Autothresh -> Huang seems to consistently give a
> segfault in processing the one with Gaussian blur 10 ?

Yes, I can reproduce it in that image. Had a look at the source, seems to be
an array dimension issues. The original code ported from Emre Celebi's
implementation was re-written by Johannes Schindelin in 2011 to make it more
efficient in handling 16bit data.  I will compare the old and new code results.
Thanks for reporting these issues. The IJ applet still uses E Celebi's code.

I see that the itk plugin appears similar to Johannes's code but it uses a bin
multiplier to re-scale the threshold value between the min and max of the
image. That might be the reason for the differences across implementations.

Regards

Gabriel





--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[robert atwood]

I tried with some of the real data , also getting very different result from the plugin as from the built-in. Alternate programs (sITK, Matlab) are closer to the builtin result value.


Using IJ builtin, or simpleITK with either 256 or 65536 bins, Otsu's method finds the dark ball.

But AutoThresh plugin with Otsu does not find the dark ball.



Example files at anonymous ftp:  This won't persist for more than several weeks however.

Real data is X-ray of a metal sphere in a hole in a plastic cylinder. Fuzzy edges are because some X-rays get through,  as the ball is thinner at the edge (it is not out of focus) Some of the plastic features (another hole, edges of the hole, glue that holds the sphere ..) appear, faintly, due to diffraction effects.

ftp://ftpanon.diamond.ac.uk/I12/ForImageJ/autoThreshOtsu.zip


Archive:  autoThreshOtsu.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
   262292  05-19-2017 11:49   appletOtsu.tif     Result of blur10blob16.tif using builtin ImageJ threshold Otsu
   262292  05-19-2017 11:49   autoThreshOtsu.tif Result of blur10blob16.tif using AutoThresh Otsu
   524456  05-19-2017 11:37   blur10blob16.tif  macro generated image, blur radius 10
   524460  05-18-2017 17:41   blur2blob16.tif macro generated image, blur radius 2
   524456  05-18-2017 16:44   nonblurblob16.tif macro generated image
   262292  05-19-2017 11:49   sitkOtsu256bins.tif tif Result of blur10blob16.tif using simpleITK with 256 bins
   262292  05-19-2017 11:49   sitkOtsu65535bins.tif Result of blur10blob16.tif using simpleITK with 65536 bins

  2006466  05-22-2017 15:05   realball16.tif  A real image -- find the dark ball !

  1003360  05-22-2017 15:05   realball16_IJ_Otsu.tif
  1003360  05-22-2017 15:05   realball16_autoThresh_Otsu.tif
  1003311  05-22-2017 15:05   realball16_sitk_otsu_256.tif
  1003311  05-22-2017 15:05   realball16_sitk_otsu_65536.tif


-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Robert Atwood
Sent: Friday, May 19, 2017 17:33
To: [hidden email]
Subject: Re: Autothresholding results; (was FW: Threshold is giving an incomplete range ?)

THere are small differences depending on binning eg 15254 vs 15419 , that should be explained by the histogram granularity, sure,

but the difference between 40645 and 64249 for blur2blob16.tif ? and the complete unchanging result of autoThresh->Otsu regardless of blurring the image? I'm not feelign competent to re-program the method myself to see which is right but they shouldnt really be that drastically different in behavour shoudl they?



________________________________________
From: Gabriel Landini [[hidden email]]
Sent: 19 May 2017 17:14
To: Atwood, Robert (DLSLtd,RAL,SCI)
Cc: [hidden email]
Subject: Re: Autothresholding results;  (was FW: Threshold is giving an incomplete range ?)

On Friday, 19 May 2017 11:19:21 BST Robert Atwood wrote:
> Using 'Otsu' I find  ---  not a little difference ---  but a big one.
> A little difference caused by the binning, a big one between implementations.
> Which is the 'real' Otsu threshold? or which different assumptions  
> can make this difference? The Otsu method seems well defined and
> established for many years so I wouldn't expect some ambiguity about
> what the result should be for a specific image and number of bins?

> The other implementations (sItk, Applet) approach a similar value as
> the image gets more continuously varying, it seems, except the
> AutoThresh one gives the same value for each level of blurring?

Hi Robert,
The same Otsu's routine in the Auto_threshold plugin is used for both the 8 and the 16bit images.
So wouldn't the differences in the granularity of the histograms account for the inter-class variances ending up being not-directly equivalent  in 256 and in a 2^16 greylevel versions of an image? I suppose that something similar happens with other methods. For example a minimum valley search does not guarantee that a coarser histogram has a minimum in that same position.

> Also, for some reason Autothresh -> Huang seems to consistently give a
> segfault in processing the one with Gaussian blur 10 ?

Yes, I can reproduce it in that image. Had a look at the source, seems to be an array dimension issues. The original code ported from Emre Celebi's implementation was re-written by Johannes Schindelin in 2011 to make it more efficient in handling 16bit data.  I will compare the old and new code results.
Thanks for reporting these issues. The IJ applet still uses E Celebi's code.

I see that the itk plugin appears similar to Johannes's code but it uses a bin multiplier to re-scale the threshold value between the min and max of the image. That might be the reason for the differences across implementations.

Regards

Gabriel





--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Gabriel Landini]

Hello Robert,
I had a look at the Auto Threshold plugin. The Otsu algorithm had indeed an
issue only with certain 16bit histograms where the computation of the variance
could cause an overflow that carried on without triggering an error.
I checked Emre Celebi's library (Fourier 0.8) and his C++ version does not
have the problem so I replaced the previous version with Emre's.
8 bit images were not affected.  I just checked and it finds the ball in your
sample image now.

The Huang method is a different issue. The original code by Emre was
rewritten in 2011 by Johannes Schindelin to use more efficiently the 16bit data
and has a massive execution speed advantage. However I just noted that in some
cases it does return exactly the same values as the original even in some 8bit
images. I therefore added back the original code and kept Johannes' method as
"Huang2" for the time being until I have time to look at it in more depth.

I also found that the Mean method (again in 16bits only) had a potential for
overflow that could go without triggering errors, so I fixed that too.
Your example image was very useful to track that down.

Here is the link to the updated file.
http://www.mecourse.com/landinig/software/autothreshold/autothreshold.html

I will update the Fiji source too, but this won't happen for some days as I
will be away. Perhaps it is best consider the 16bit support in the other
methods as experimental in the mean time.

Cheers
Gabriel

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html