1D autocorrelation function

[LMAnovitz]

Hi
  I am looking for a plug-in for ImageJ that will calculate the 1-D
autocorrelation function from a binary image.

As per

Journal of Petroleum Science and Engineering 16 (1996) 251-261
Statistical analysis of the porous microstructure as a method for
estimating reservoir permeability
M.A. Ioannidis, M.J. Kwiecien, I. Chatzis

A fourier transform of this curve will then be used to extend my neutron
scattering data to
Lower Q values as per

Radlinski et al. (2004) Angstron-to-millimeter characterization of
sedimentary rock microstructure.
J Colloid and Interface Sci. 247, 607-612.


Can anyone suggest what the correct  plug-in might be ?
Thanks.

--Larry


--
Dr. Lawrence M. Anovitz
MS 6110 PO Box 2008
Aqueous and Geochemistry Group
Oak Ridge National Laboratory
Oak Ridge, Tennessee 37831-6110

865-574-5034 : phone
865-574-4961 : fax

[hidden email]

[Gluender-2]

Larry,

without reading the papers (not my job)...

Could you please explain what happens with the second coordinate of
an image during the 1-D autocorrelation? Is the image projected in
one direction or is the desired result a series of 1-D
autocorrelation functions?

As far as I know imagej does'nt provide neither 1-D
Fourier-transformation nor 1-D autocorrelation which in general is
computed by Fourier-retransformation of the power spectrum.

However there are numerous applications for data/signal analyses that
allow the computation of both, autocorrelation and power spectrum of
1-D signals. Hence you may export the 1-D data from imagej to one of
these applications and then perform the desired computations.

>Hi
>   I am looking for a plug-in for ImageJ that will calculate the 1-D
>autocorrelation function from a binary image.
>
>As per
>
>Journal of Petroleum Science and Engineering 16 (1996) 251-261
>Statistical analysis of the porous microstructure as a method for
>estimating reservoir permeability
>M.A. Ioannidis, M.J. Kwiecien, I. Chatzis
>
>A fourier transform of this curve will then be used to extend my neutron
>scattering data to
>Lower Q values as per

Because the Fourier transform of the autocorrelation function is the
power spectrum, I don't quite grasp why the latter isn't directly
computed, but I think I'm missing some essential information that you
may perhaps kindly provide.

>Radlinski et al. (2004) Angstron-to-millimeter characterization of
>sedimentary rock microstructure.
>J Colloid and Interface Sci. 247, 607-612.
>
>Can anyone suggest what the correct  plug-in might be ?
>Thanks.
>
>--Larry
>--
>Dr. Lawrence M. Anovitz
>MS 6110 PO Box 2008
>Aqueous and Geochemistry Group
>Oak Ridge National Laboratory
>Oak Ridge, Tennessee 37831-6110
>
>865-574-5034 : phone
>865-574-4961 : fax
>
>[hidden email]

HTH
--

                   Herbie

          ------------------------
          <http://www.gluender.de>

[LMAnovitz]

HI
    The correlation function is not a projection of the image.
Rather, it is a measure of the spatial distribution of, and within
particles in it.  It is sometimes called a pair distribution function
Or a radial distribution function. You may have seen these in various
types of computation chemistry or X-ray measurements where one plots
concentration as a function of distance, and there a peaks in the curves
That correspond to, say, in water, the average H-H, O-H and O-O distances.
In this case of a sample with pore space, this gives the correlations of
distances within a given pore, and between pores.

Summarizing from Ioannidis et al (I tried to attach it, just fyi, in case my
description is not clear enough, but the system won't let me. Please contact
Me directly at the address below, and I'll forward the paper if anyone needs
it)

1) let's describe the binary image by a phase function (Z(r)), which takes
the values zero or one, depending on whether what pixel is white (a grain)
or black (a pore)

2)The structure of a statistically homogeneous medium can be described by
the first two statistical moments of this function, the porosity (f) -
actually phi, but I've no idea how to put such symbols in an e-mail, and the
Autocorrelation function R(u) these are given by the averages <>

F = <Z(r)>

R(u) = ( <Z(r) - f>€<Z(r+u)-f> )/ abs(f - f^2)

So what does all this mean for a plugin programmer and a two-dimensional
image

One first calculates f for the image
Then one starts at the first pixel (0,0) and finds its Z(r) value and,
Or each of the other pixels relative to that one, finds the distance (u),
The Z(r) value for the second pixel (called Z(r+u) above, and calculates
The value of R.  One does this for all pairs of pixels in the image, and
sums the results for each value of (u) to get R(u).  (I'm not 100% sure, but
I think Ioannidis et al. only actually do this along x-and y-directions from
each pixel, not radially)

Does this make any sense ? Can anyone help ?

--larry

--
Dr. Lawrence M. Anovitz
MS 6110 PO Box 2008
Aqueous and Geochemistry Group
Oak Ridge National Laboratory
Oak Ridge, Tennessee 37831-6110

865-574-5034 : phone
865-574-4961 : fax

[hidden email]





On 4/4/09 7:58 AM, "Gluender" <[hidden email]> wrote:

> Larry,
>
> without reading the papers (not my job)...
>
> Could you please explain what happens with the second coordinate of
> an image during the 1-D autocorrelation? Is the image projected in
> one direction or is the desired result a series of 1-D
> autocorrelation functions?
>
> As far as I know imagej does'nt provide neither 1-D
> Fourier-transformation nor 1-D autocorrelation which in general is
> computed by Fourier-retransformation of the power spectrum.
>
> However there are numerous applications for data/signal analyses that
> allow the computation of both, autocorrelation and power spectrum of
> 1-D signals. Hence you may export the 1-D data from imagej to one of
> these applications and then perform the desired computations.
>
>
>> Hi
>>   I am looking for a plug-in for ImageJ that will calculate the 1-D
>> autocorrelation function from a binary image.
>>
>> As per
>>
>> Journal of Petroleum Science and Engineering 16 (1996) 251-261
>> Statistical analysis of the porous microstructure as a method for
>> estimating reservoir permeability
>> M.A. Ioannidis, M.J. Kwiecien, I. Chatzis
>>
>> A fourier transform of this curve will then be used to extend my neutron
>> scattering data to
>> Lower Q values as per
>
> Because the Fourier transform of the autocorrelation function is the
> power spectrum, I don't quite grasp why the latter isn't directly
> computed, but I think I'm missing some essential information that you
> may perhaps kindly provide.
>
>> Radlinski et al. (2004) Angstron-to-millimeter characterization of
>> sedimentary rock microstructure.
>> J Colloid and Interface Sci. 247, 607-612.
>>
>> Can anyone suggest what the correct  plug-in might be ?
>> Thanks.
>>
>> --Larry
>> --
>> Dr. Lawrence M. Anovitz
>> MS 6110 PO Box 2008
>> Aqueous and Geochemistry Group
>> Oak Ridge National Laboratory
>> Oak Ridge, Tennessee 37831-6110
>>
>> 865-574-5034 : phone
>> 865-574-4961 : fax
>>
>> [hidden email]
>
> HTH

[Gluender-2]

>HI
>     The correlation function is not a projection of the image.

I didn't state this. (Please read carefully!)

>[...]
>
>1) let's describe the binary image by a phase function (Z(r)), which takes
>the values zero or one, depending on whether what pixel is white (a grain)
>or black (a pore)
>
>2)The structure of a statistically homogeneous medium can be described by
>the first two statistical moments of this function, the porosity (f) -
>actually phi, but I've no idea how to put such symbols in an e-mail, and the
>Autocorrelation function R(u) these are given by the averages <>
>
>F = <Z(r)>
>
>R(u) = ( <Z(r) - f>*<Z(r+u)-f> )/ abs(f - f^2)
>
>So what does all this mean for a plugin programmer and a two-dimensional
>image

????

>One first calculates f for the image
>Then one starts at the first pixel (0,0) and finds its Z(r) value and,
>Or each of the other pixels relative to that one, finds the distance (u),
>The Z(r) value for the second pixel (called Z(r+u) above, and calculates
>The value of R.  One does this for all pairs of pixels in the image, and
>sums the results for each value of (u) to get R(u).  (I'm not 100% sure, but
>I think Ioannidis et al. only actually do this along x-and y-directions from
>each pixel, not radially)
>
>Does this make any sense ? Can anyone help ?

Sorry no, not for me.
Before you are going to tackle the ij-macro problem, you should have
understood the theory.

Best Herbie

>--larry
>
>--
>Dr. Lawrence M. Anovitz
>MS 6110 PO Box 2008
>Aqueous and Geochemistry Group
>Oak Ridge National Laboratory
>Oak Ridge, Tennessee 37831-6110
>
>865-574-5034 : phone
>865-574-4961 : fax
>
>[hidden email]
>
>
>
>
>
>On 4/4/09 7:58 AM, "Gluender" <[hidden email]> wrote:
>
>>  Larry,
>>
>>  without reading the papers (not my job)...
>>
>>  Could you please explain what happens with the second coordinate of
>>  an image during the 1-D autocorrelation? Is the image projected in
>>  one direction or is the desired result a series of 1-D
>>  autocorrelation functions?
>>
>>  As far as I know imagej does'nt provide neither 1-D
>>  Fourier-transformation nor 1-D autocorrelation which in general is
>>  computed by Fourier-retransformation of the power spectrum.
>>
>>  However there are numerous applications for data/signal analyses that
>>  allow the computation of both, autocorrelation and power spectrum of
>>  1-D signals. Hence you may export the 1-D data from imagej to one of
>>  these applications and then perform the desired computations.
>>
>>
>>>  Hi
>>>    I am looking for a plug-in for ImageJ that will calculate the 1-D
>>>  autocorrelation function from a binary image.
>>>
>>>  As per
>>>
>>>  Journal of Petroleum Science and Engineering 16 (1996) 251-261
>>>  Statistical analysis of the porous microstructure as a method for
>>>  estimating reservoir permeability
>>>  M.A. Ioannidis, M.J. Kwiecien, I. Chatzis
>>>
>>>  A fourier transform of this curve will then be used to extend my neutron
>>>  scattering data to
>>>  Lower Q values as per
>>
>>  Because the Fourier transform of the autocorrelation function is the
>>  power spectrum, I don't quite grasp why the latter isn't directly
>>  computed, but I think I'm missing some essential information that you
>>  may perhaps kindly provide.
>>
>>>  Radlinski et al. (2004) Angstron-to-millimeter characterization of
>>>  sedimentary rock microstructure.
>>>  J Colloid and Interface Sci. 247, 607-612.
>>>
>>>  Can anyone suggest what the correct  plug-in might be ?
>>>  Thanks.
>>>
>>>  --Larry
>>>  --
>>>  Dr. Lawrence M. Anovitz
>>>  MS 6110 PO Box 2008
>>>  Aqueous and Geochemistry Group
>>>  Oak Ridge National Laboratory
>>>  Oak Ridge, Tennessee 37831-6110
>>>
>>>  865-574-5034 : phone
>>>  865-574-4961 : fax
>>>
>>>  [hidden email]
>>
>>  HTH

--

                   Herbie

          ------------------------
          <http://www.gluender.de>

[Gabriel Landini]

Larry,
Are you sure you do not mean the "two-point correlation function"? That is a
commonly used technique in condensed matter physics (I have seen it several
times used to characterise fractal agglormerates and porous materials).

Dsho wrote a plugin some time ago. I have not used it, though:

http://wbgn013.biozentrum.uni-wuerzburg.de/ImageJ/two-point-correlation.html

Cheers,

G.

[Michael Schmid]

Hi Larry,

ok, so what you need is related to the Radial Distribution Function  
(RDF).

For the correlation of the pair distribution of particle *centers* (H-
H or O-O in the water example) you can use my RDF plugin - it was  
somewhere in this mailing list, but I have uploaded it now to the  
documentation wiki:
   http://imagejdocu.tudor.lu/doku.php?
id=macro:radial_distribution_function.

For the correlation, simply scale your binary image (it must be  
square) to a size of 2^n*2^n and use Process>FFT>FD Math for  
correlation (both inputs are the same, and check "do inverse  
transform"). You need holes/particles that are much smaller than the  
image size, otherwise the periodic boundary conditions inherent to  
any FFT function will limit the accuracy.

Then normalize the result: divide the image by the pixel value at  
size/2, size/2, and you will get the correlation between equal  
pixels. Use the Radial_Profile plugin from the ImageJ web site if  
only the distance is important (i.e., if you have an isotropic sample).

The correlation between unequal pairs is 1 minus the correlation  
between equal pairs.

Michael
________________________________________________________________

On 6 Apr 2009, at 15:35, Larry Anovitz wrote:

> HI
>     The correlation function is not a projection of the image.
> Rather, it is a measure of the spatial distribution of, and within
> particles in it.  It is sometimes called a pair distribution function
> Or a radial distribution function. You may have seen these in various
> types of computation chemistry or X-ray measurements where one plots
> concentration as a function of distance, and there a peaks in the  
> curves
> That correspond to, say, in water, the average H-H, O-H and O-O  
> distances.
> In this case of a sample with pore space, this gives the  
> correlations of
> distances within a given pore, and between pores.
>
> Summarizing from Ioannidis et al (I tried to attach it, just fyi,  
> in case my
> description is not clear enough, but the system won't let me.  
> Please contact
> Me directly at the address below, and I'll forward the paper if  
> anyone needs
> it)
>
> 1) let's describe the binary image by a phase function (Z(r)),  
> which takes
> the values zero or one, depending on whether what pixel is white (a  
> grain)
> or black (a pore)
>
> 2)The structure of a statistically homogeneous medium can be  
> described by
> the first two statistical moments of this function, the porosity (f) -
> actually phi, but I've no idea how to put such symbols in an e-
> mail, and the
> Autocorrelation function R(u) these are given by the averages <>
>
> F = <Z(r)>
>
> R(u) = ( <Z(r) - f>•<Z(r+u)-f> )/ abs(f - f^2)
>
> So what does all this mean for a plugin programmer and a two-
> dimensional
> image
>
> One first calculates f for the image
> Then one starts at the first pixel (0,0) and finds its Z(r) value and,
> Or each of the other pixels relative to that one, finds the  
> distance (u),
> The Z(r) value for the second pixel (called Z(r+u) above, and  
> calculates
> The value of R.  One does this for all pairs of pixels in the  
> image, and
> sums the results for each value of (u) to get R(u).  (I'm not 100%  
> sure, but
> I think Ioannidis et al. only actually do this along x-and y-
> directions from
> each pixel, not radially)
>
> Does this make any sense ? Can anyone help ?
>
> --larry
>
> --
> Dr. Lawrence M. Anovitz
> MS 6110 PO Box 2008
> Aqueous and Geochemistry Group
> Oak Ridge National Laboratory
> Oak Ridge, Tennessee 37831-6110
>
> 865-574-5034 : phone
> 865-574-4961 : fax
>
> [hidden email]
>
>
>
>
>
> On 4/4/09 7:58 AM, "Gluender" <[hidden email]> wrote:
>
>> Larry,
>>
>> without reading the papers (not my job)...
>>
>> Could you please explain what happens with the second coordinate of
>> an image during the 1-D autocorrelation? Is the image projected in
>> one direction or is the desired result a series of 1-D
>> autocorrelation functions?
>>
>> As far as I know imagej does'nt provide neither 1-D
>> Fourier-transformation nor 1-D autocorrelation which in general is
>> computed by Fourier-retransformation of the power spectrum.
>>
>> However there are numerous applications for data/signal analyses that
>> allow the computation of both, autocorrelation and power spectrum of
>> 1-D signals. Hence you may export the 1-D data from imagej to one of
>> these applications and then perform the desired computations.
>>
>>
>>> Hi
>>>   I am looking for a plug-in for ImageJ that will calculate the 1-D
>>> autocorrelation function from a binary image.
>>>
>>> As per
>>>
>>> Journal of Petroleum Science and Engineering 16 (1996) 251-261
>>> Statistical analysis of the porous microstructure as a method for
>>> estimating reservoir permeability
>>> M.A. Ioannidis, M.J. Kwiecien, I. Chatzis
>>>
>>> A fourier transform of this curve will then be used to extend my  
>>> neutron
>>> scattering data to
>>> Lower Q values as per
>>
>> Because the Fourier transform of the autocorrelation function is the
>> power spectrum, I don't quite grasp why the latter isn't directly
>> computed, but I think I'm missing some essential information that you
>> may perhaps kindly provide.
>>
>>> Radlinski et al. (2004) Angstron-to-millimeter characterization of
>>> sedimentary rock microstructure.
>>> J Colloid and Interface Sci. 247, 607-612.
>>>
>>> Can anyone suggest what the correct  plug-in might be ?
>>> Thanks.
>>>
>>> --Larry
>>> --
>>> Dr. Lawrence M. Anovitz
>>> MS 6110 PO Box 2008
>>> Aqueous and Geochemistry Group
>>> Oak Ridge National Laboratory
>>> Oak Ridge, Tennessee 37831-6110
>>>
>>> 865-574-5034 : phone
>>> 865-574-4961 : fax
>>>
>>> [hidden email]
>>
>> HTH

[LMAnovitz]

Gabriel and Michael
   thanks for the suggestions/links
Herbie
   thanks for looking

So far I have tried the plugin Gabriel sent, and it seems to work fine.
(the Berryman and Blair article referenced therein is one I am familiar
with) It is, as the author suggested, a bit slow, and I had to up the memory
quite A bit, but my first test yielded a successful curve.
I will also try the macro Michael sent in a bit and compare.

Next, the FFT !

--Larry


On 4/6/09 10:46 AM, "Gabriel Landini" <[hidden email]> wrote:

> Larry,
> Are you sure you do not mean the "two-point correlation function"? That is a
> commonly used technique in condensed matter physics (I have seen it several
> times used to characterise fractal agglormerates and porous materials).
>
> Dsho wrote a plugin some time ago. I have not used it, though:
>
> http://wbgn013.biozentrum.uni-wuerzburg.de/ImageJ/two-point-correlation.html
>
> Cheers,
>
> G.

[Prodanov Dimiter]

Dear Larry,

You can also use the Gray Morphology or Float Morphology packages with 2 points Structure Element.
The squared volume of the difference between the Original and the Eroded image divided by the area of the image A  -- V^2[I-E]/A[I] is the (semi)variogram for this displacement.
 
Best regards,

Dimiter

[LMAnovitz]

Hi,
    A few weeks back, Gabriel was kind enough to send me the suggestion
below in response to my question about a plug-in to calculate 2-point
correlation functions.
    Having been working with this plug-in for a few weeks now, I have run
into a couple of problems, and one query, which I hope someone can help me
solve.
    The biggest problem is memory.  Running this plug-in takes a lot.  If I
have a 1024 x 1024 pixel image I can get it to solve if I set the memory to
2 GB,  If I have a 2048 x 2048 image I have not yet gotten the program to
work.  Instead, it comes back with a window saying that it has used up the
available memory (I've used values as high as 4 GB, and downloaded the
newest version of ImageJ for the Mac this afternoon). While I have tried
down-sampling the image, and this works, it does not give me the resolution
I would like, and I need to keep the overall image size large to avoid edge
effects (I'd like to go to a 4096 x 4096 image at least).
    The second problem is that the "naïve" computation version of the plug
in does not seem to work at all.  The pop-up window does say that this is
slow, but I've left if for hours without any results.
    Finally, a question.  The plug-in has a default radius step (changeable)
of 0.3 pixels. Since this is (obviously) less than 1 pixel, and the
correlation has to do with the relationship between pixels, this seems a bit
odd.  I suspect it may be due to the FFT method of obtaining the
calculation, but am not sure, and was wondering if Dsho (who wrote the
plug-in) could enlighten me.

    Many thanks in advance for help with this.

--Larry

I am running a Mac, with system version 10.5.6, a 2 x 3.2 GHz Quad-Core
Intel Xeon processor, and 6 GB of 800 MHz DDR2 FB-DIMM memory.

--Larry Anovitz


--
Dr. Lawrence M. Anovitz
MS 6110 PO Box 2008
Aqueous and Geochemistry Group
Oak Ridge National Laboratory
Oak Ridge, Tennessee 37831-6110

865-574-5034 : phone
865-574-4961 : fax

[hidden email]





On 4/6/09 10:46 AM, "Gabriel Landini" <[hidden email]> wrote:

> Larry,
> Are you sure you do not mean the "two-point correlation function"? That is a
> commonly used technique in condensed matter physics (I have seen it several
> times used to characterise fractal agglormerates and porous materials).
>
> Dsho wrote a plugin some time ago. I have not used it, though:
>
> http://wbgn013.biozentrum.uni-wuerzburg.de/ImageJ/two-point-correlation.html
>
> Cheers,
>
> G.

[Michael Schmid]

Hi Larry,

if it is simply the radial distribution function that you are  
interested in you can try my macro
   http://imagejdocu.tudor.lu/doku.php?
id=macro:radial_distribution_function

It needs about half a minute (2.4 GHz Core2Duo) and 500 MB RAM for a  
2048*2048 binary image.
It uses the FFT, but does not assume periodic boundary conditions.  
Instead, it rather expands the image (in this case to 4096*4096) and  
corrects for the finite size (edge effects) of the image. So you need  
not (and should not) expand your image when using it.
---
Concerning the r resulution: For radii well above 1, there should be  
enough pixels in each zone between r and r+0.3 pixels. My macro  
simply uses the Radial Profile Plugin, which uses increments of 1 for  
the radius.
http://rsb.info.nih.gov/ij/plugins/radial-profile.html

It would be interesting to see how the results of my macro compare  
with the plugin by Dscho; if you find any significant differences or  
a bug in my macro, let me know, please.

Michael
________________________________________________________________

On 13 May 2009, at 22:47, Larry Anovitz wrote:

> Hi,
>     A few weeks back, Gabriel was kind enough to send me the  
> suggestion
> below in response to my question about a plug-in to calculate 2-point
> correlation functions.
>     Having been working with this plug-in for a few weeks now, I  
> have run
> into a couple of problems, and one query, which I hope someone can  
> help me
> solve.
>     The biggest problem is memory.  Running this plug-in takes a  
> lot.  If I
> have a 1024 x 1024 pixel image I can get it to solve if I set the  
> memory to
> 2 GB,  If I have a 2048 x 2048 image I have not yet gotten the  
> program to
> work.  Instead, it comes back with a window saying that it has used  
> up the
> available memory (I've used values as high as 4 GB, and downloaded the
> newest version of ImageJ for the Mac this afternoon). While I have  
> tried
> down-sampling the image, and this works, it does not give me the  
> resolution
> I would like, and I need to keep the overall image size large to  
> avoid edge
> effects (I'd like to go to a 4096 x 4096 image at least).
>     The second problem is that the "naïve" computation version of  
> the plug
> in does not seem to work at all.  The pop-up window does say that  
> this is
> slow, but I've left if for hours without any results.
>     Finally, a question.  The plug-in has a default radius step  
> (changeable)
> of 0.3 pixels. Since this is (obviously) less than 1 pixel, and the
> correlation has to do with the relationship between pixels, this  
> seems a bit
> odd.  I suspect it may be due to the FFT method of obtaining the
> calculation, but am not sure, and was wondering if Dsho (who wrote the
> plug-in) could enlighten me.
>
>     Many thanks in advance for help with this.
>
> --Larry
>
> I am running a Mac, with system version 10.5.6, a 2 x 3.2 GHz Quad-
> Core
> Intel Xeon processor, and 6 GB of 800 MHz DDR2 FB-DIMM memory.
>
> --Larry Anovitz
>
>
> --
> Dr. Lawrence M. Anovitz
> MS 6110 PO Box 2008
> Aqueous and Geochemistry Group
> Oak Ridge National Laboratory
> Oak Ridge, Tennessee 37831-6110
>
> 865-574-5034 : phone
> 865-574-4961 : fax
>
> [hidden email]
>
>
>
>
>
> On 4/6/09 10:46 AM, "Gabriel Landini" <[hidden email]> wrote:
>
>> Larry,
>> Are you sure you do not mean the "two-point correlation function"?  
>> That is a
>> commonly used technique in condensed matter physics (I have seen  
>> it several
>> times used to characterise fractal agglormerates and porous  
>> materials).
>>
>> Dsho wrote a plugin some time ago. I have not used it, though:
>>
>> http://wbgn013.biozentrum.uni-wuerzburg.de/ImageJ/two-point- 
>> correlation.html
>>
>> Cheers,
>>
>> G.

[LMAnovitz]

Michael
  I gave it a try, and you are right, this is much faster.  The results do
not, however seem to be identical.
Since you were curious about the comparison of the two techniques, here are
the results.  Unfortunately, the server won¹t let me sent
Figures, but if you want to see them send me a direct e-mail, mine is listed
below.

The  RDF macro/plug-in was done on the full 2048 x 2048 image (the original
image is actually a 1024 x 1024 image
Copied 4 times in order to try to reduce the edge effects.  This may not be
necessary for the RDF approach).
The 2-point correlation plug-in.  It was done on the same 4x Image,
downsampled to 1024 x 1024 using the Averaging Reducer plugin

There are several differences between
1. the 2PC curve is smoother, and composed of about 10x more points
2. the Y scale is different, with a maximum on the RDF af about 4.5 and the
2PC or about 4000.  This is probably immaterial, however, as this wil be
normalized
3. there is a long tail in the 2PC curve that is probably an artifact of the
image multiplication and edges. It can be fitted as a stretched exponential
and subtracted
4. importantly, the sharp drop-off in the curve appears to be at about 5-10x
larger r in the RDF plot (between about 8 and 25 microns) than the 2PC plot
(between 1.5 and 3 microns)

I can also compare these two results (normalized, and with the tail
subtracted from the 2PC curve)
 to a plot of the correlation function calculated from a measured neutron
scattering curve and a pore Volume obtained from the image.
This has a half-fall distance of about 4 microns. Thus the measured values
lie between the other 2.

The direct neutron measurements are for scales up to approx 24 microns, so
most of this curve is directly constrained. However, while it may not seem
very important in the correlation plot, a lot of the calculated volume comes
from that high-r part of the curve


What do you think ?

--larry


On 5/14/09 4:19 AM, "Michael Schmid" <[hidden email]> wrote:

> Hi Larry,
>
> if it is simply the radial distribution function that you are
> interested in you can try my macro
>    http://imagejdocu.tudor.lu/doku.php?
> id=macro:radial_distribution_function
>
> It needs about half a minute (2.4 GHz Core2Duo) and 500 MB RAM for a
> 2048*2048 binary image.
> It uses the FFT, but does not assume periodic boundary conditions.
> Instead, it rather expands the image (in this case to 4096*4096) and
> corrects for the finite size (edge effects) of the image. So you need
> not (and should not) expand your image when using it.
> ---
> Concerning the r resulution: For radii well above 1, there should be
> enough pixels in each zone between r and r+0.3 pixels. My macro
> simply uses the Radial Profile Plugin, which uses increments of 1 for
> the radius.
> http://rsb.info.nih.gov/ij/plugins/radial-profile.html
>
> It would be interesting to see how the results of my macro compare
> with the plugin by Dscho; if you find any significant differences or
> a bug in my macro, let me know, please.
>
> Michael
> ________________________________________________________________
>
> On 13 May 2009, at 22:47, Larry Anovitz wrote:
>
>> Hi,
>>     A few weeks back, Gabriel was kind enough to send me the
>> suggestion
>> below in response to my question about a plug-in to calculate 2-point
>> correlation functions.
>>     Having been working with this plug-in for a few weeks now, I
>> have run
>> into a couple of problems, and one query, which I hope someone can
>> help me
>> solve.
>>     The biggest problem is memory.  Running this plug-in takes a
>> lot.  If I
>> have a 1024 x 1024 pixel image I can get it to solve if I set the
>> memory to
>> 2 GB,  If I have a 2048 x 2048 image I have not yet gotten the
>> program to
>> work.  Instead, it comes back with a window saying that it has used
>> up the
>> available memory (I've used values as high as 4 GB, and downloaded the
>> newest version of ImageJ for the Mac this afternoon). While I have
>> tried
>> down-sampling the image, and this works, it does not give me the
>> resolution
>> I would like, and I need to keep the overall image size large to
>> avoid edge
>> effects (I'd like to go to a 4096 x 4096 image at least).
>>     The second problem is that the "naïve" computation version of
>> the plug
>> in does not seem to work at all.  The pop-up window does say that
>> this is
>> slow, but I've left if for hours without any results.
>>     Finally, a question.  The plug-in has a default radius step
>> (changeable)
>> of 0.3 pixels. Since this is (obviously) less than 1 pixel, and the
>> correlation has to do with the relationship between pixels, this
>> seems a bit
>> odd.  I suspect it may be due to the FFT method of obtaining the
>> calculation, but am not sure, and was wondering if Dsho (who wrote the
>> plug-in) could enlighten me.
>>
>>     Many thanks in advance for help with this.
>>
>> --Larry
>>
>> I am running a Mac, with system version 10.5.6, a 2 x 3.2 GHz Quad-
>> Core
>> Intel Xeon processor, and 6 GB of 800 MHz DDR2 FB-DIMM memory.
>>
>> --Larry Anovitz
>>
>>
>> --
>> Dr. Lawrence M. Anovitz
>> MS 6110 PO Box 2008
>> Aqueous and Geochemistry Group
>> Oak Ridge National Laboratory
>> Oak Ridge, Tennessee 37831-6110
>>
>> 865-574-5034 : phone
>> 865-574-4961 : fax
>>
>> [hidden email]
>>
>>
>>
>>
>>
>> On 4/6/09 10:46 AM, "Gabriel Landini" <[hidden email]> wrote:
>>
>>> Larry,
>>> Are you sure you do not mean the "two-point correlation function"?
>>> That is a
>>> commonly used technique in condensed matter physics (I have seen
>>> it several
>>> times used to characterise fractal agglormerates and porous
>>> materials).
>>>
>>> Dsho wrote a plugin some time ago. I have not used it, though:
>>>
>>> http://wbgn013.biozentrum.uni-wuerzburg.de/ImageJ/two-point-
>>> correlation.html
>>>
>>> Cheers,
>>>
>>> G.

[Michael Schmid]

Hi Larry,

maybe there is a misunderstanding:

My RDF plugin calculates the radial distribution function of particle  
CENTERS. I have updated the documentation to make this more clear.

If you are interested in the radially averaged autocorrelation of the  
binary image (thus including information on particle sizes and  
shapes), here is a macro for this task.

Note that the mailer may add line breaks in long lines such as the  
ones starting with 'run("FD Math'; in that case you have to join  
these lines.

// ImageJ macro to calculate the Radially Averaged Autocorrelation  
Function,
// Corrected for finite size effects
// The output is normalized to a value of 1 at zero radius
// and corrected for effects of the finite image size.
// (No need to extend the image size for avoiding edge effects)
//
// Use with a binary input image.
// Needs the "Radial Profile" plugin, http://rsb.info.nih.gov/ij/ 
plugins/radial-profile.html
//
// Version: 2008-May-14 Michael Schmid
//
doStack=false;
if (nSlices()>1) doStack = getBoolean("Get RDF from full stack?");
setBatchMode(true);
firstSlice=getSliceNumber();
lastSlice=getSliceNumber();
if (doStack) {
   firstSlice=1;
   lastSlice=nSlices();
}
width=getWidth;
height=getHeight;
getPixelSize(unit, pixelWidth, pixelHeight);
run("Select None");
//maxRadius may be modified, should not be larger than 0.3*minOf
(width, height);
maxRadius=0.3*minOf(width, height);
minFFTsize=1.3*maxOf(width, height);
title=getTitle();
size=4;
while(size<minFFTsize) size*=2;
for (slice=firstSlice; slice<=lastSlice; slice++) {
   //make autocorrelation of particle image
   tempTitle="temp-"+random();
   run("Duplicate...", "title="+tempTitle);
   tempID=getImageID();
   getRawStatistics(nPixels, mean);
   run("Canvas Size...", "width="+ size+" height="+ size+"  
position=Center zero");
   makeRectangle(floor((size-width)/2), floor((size-height)/2),  
width, height);
   run("Make Inverse");
   run("Set...", "value="+mean);
   run("Select None");
   getRawStatistics(nPixels, mean);
   run("FD Math...", "image1=["+tempTitle+"] operation=Correlate  
image2=["+tempTitle+"] result=AutoCorrelation do");
   psID=getImageID();
   run("Subtract...", "value="+(nPixels*mean*mean));
   selectImage(tempID);
   close();

   //make autocorrelation reference to correct finite image size effects
   newImage("frame", "8-bit White", width, height, 1);
   run("Set...", "value=255");
   tempID=getImageID();
   rename(tempTitle);
   run("Canvas Size...", "width="+ size+" height="+ size+"  
position=Center zero");
   run("FD Math...", "image1=["+tempTitle+"] operation=Correlate  
image2=["+tempTitle+"] result=AutoCorrReference do");
   refID=getImageID();
   imageCalculator("Divide", psID,refID);
   selectImage(refID);
   close();
   selectImage(tempID);
   close();

   //prepare normalized power spectrum for radial averaging
   selectImage(psID);
   circleSize=2*floor(maxRadius)+1;
   norm = getPixel(size/2, size/2);
   run("Divide...", "value="+norm);
   run("Specify...", "width="+circleSize+" height="+circleSize+" x="+
(size/2+0.5)+" y="+(size/2+0.5)+" oval centered");
   run("Radial Profile", "x="+(size/2+0.5)+" y="+(size/2+0.5)+"  
radius="+floor(maxRadius)-1);
   plotID=getImageID();
   selectImage(psID);
   close();
   Plot.getValues(x, y);
   if (slice==firstSlice) ySum = newArray(y.length);
   for (i=0; i<y.length; i++)
     ySum[i]+=y[i]/nSlices;
   selectImage(plotID);
   close();
}
if (pixelWidth==pixelHeight) {
   for (i=0; i<x.length; i++)
     x[i] *= pixelWidth;
} else
   unit = "pixels";
if (doStack) title = title + " (stack)";
Plot.create("Autocorrelation of "+title, "Distance ("+unit+")",  
"Normalized Autocorrelation", x, ySum);

setBatchMode(false);

Best wishes,

Michael
________________________________________________________________

On 14 May 2009, at 19:38, Larry Anovitz wrote:

> Michael
>   I gave it a try, and you are right, this is much faster.  The  
> results do
> not, however seem to be identical.
> Since you were curious about the comparison of the two techniques,  
> here are
> the results.  Unfortunately, the server won’t let me sent
> Figures, but if you want to see them send me a direct e-mail, mine  
> is listed
> below.
>
> The  RDF macro/plug-in was done on the full 2048 x 2048 image (the  
> original
> image is actually a 1024 x 1024 image
> Copied 4 times in order to try to reduce the edge effects.  This  
> may not be
> necessary for the RDF approach).
> The 2-point correlation plug-in.  It was done on the same 4x Image,
> downsampled to 1024 x 1024 using the Averaging Reducer plugin
>
> There are several differences between
> 1. the 2PC curve is smoother, and composed of about 10x more points
> 2. the Y scale is different, with a maximum on the RDF af about 4.5  
> and the
> 2PC or about 4000.  This is probably immaterial, however, as this  
> wil be
> normalized
> 3. there is a long tail in the 2PC curve that is probably an  
> artifact of the
> image multiplication and edges. It can be fitted as a stretched  
> exponential
> and subtracted
> 4. importantly, the sharp drop-off in the curve appears to be at  
> about 5-10x
> larger r in the RDF plot (between about 8 and 25 microns) than the  
> 2PC plot
> (between 1.5 and 3 microns)
>
> I can also compare these two results (normalized, and with the tail
> subtracted from the 2PC curve)
>  to a plot of the correlation function calculated from a measured  
> neutron
> scattering curve and a pore Volume obtained from the image.
> This has a half-fall distance of about 4 microns. Thus the measured  
> values
> lie between the other 2.
>
> The direct neutron measurements are for scales up to approx 24  
> microns, so
> most of this curve is directly constrained. However, while it may  
> not seem
> very important in the correlation plot, a lot of the calculated  
> volume comes
> from that high-r part of the curve
>
>
> What do you think ?
>
> --larry
>
>
> On 5/14/09 4:19 AM, "Michael Schmid" <[hidden email]> wrote:
>
>> Hi Larry,
>>
>> if it is simply the radial distribution function that you are
>> interested in you can try my macro
>>    http://imagejdocu.tudor.lu/doku.php?
>> id=macro:radial_distribution_function
>>
>> It needs about half a minute (2.4 GHz Core2Duo) and 500 MB RAM for a
>> 2048*2048 binary image.
>> It uses the FFT, but does not assume periodic boundary conditions.
>> Instead, it rather expands the image (in this case to 4096*4096) and
>> corrects for the finite size (edge effects) of the image. So you need
>> not (and should not) expand your image when using it.
>> ---
>> Concerning the r resulution: For radii well above 1, there should be
>> enough pixels in each zone between r and r+0.3 pixels. My macro
>> simply uses the Radial Profile Plugin, which uses increments of 1 for
>> the radius.
>> http://rsb.info.nih.gov/ij/plugins/radial-profile.html
>>
>> It would be interesting to see how the results of my macro compare
>> with the plugin by Dscho; if you find any significant differences or
>> a bug in my macro, let me know, please.
>>
>> Michael
>> ________________________________________________________________
>>
>> On 13 May 2009, at 22:47, Larry Anovitz wrote:
>>
>>> Hi,
>>>     A few weeks back, Gabriel was kind enough to send me the
>>> suggestion
>>> below in response to my question about a plug-in to calculate 2-
>>> point
>>> correlation functions.
>>>     Having been working with this plug-in for a few weeks now, I
>>> have run
>>> into a couple of problems, and one query, which I hope someone can
>>> help me
>>> solve.
>>>     The biggest problem is memory.  Running this plug-in takes a
>>> lot.  If I
>>> have a 1024 x 1024 pixel image I can get it to solve if I set the
>>> memory to
>>> 2 GB,  If I have a 2048 x 2048 image I have not yet gotten the
>>> program to
>>> work.  Instead, it comes back with a window saying that it has used
>>> up the
>>> available memory (I've used values as high as 4 GB, and  
>>> downloaded the
>>> newest version of ImageJ for the Mac this afternoon). While I have
>>> tried
>>> down-sampling the image, and this works, it does not give me the
>>> resolution
>>> I would like, and I need to keep the overall image size large to
>>> avoid edge
>>> effects (I'd like to go to a 4096 x 4096 image at least).
>>>     The second problem is that the "naïve" computation version of
>>> the plug
>>> in does not seem to work at all.  The pop-up window does say that
>>> this is
>>> slow, but I've left if for hours without any results.
>>>     Finally, a question.  The plug-in has a default radius step
>>> (changeable)
>>> of 0.3 pixels. Since this is (obviously) less than 1 pixel, and the
>>> correlation has to do with the relationship between pixels, this
>>> seems a bit
>>> odd.  I suspect it may be due to the FFT method of obtaining the
>>> calculation, but am not sure, and was wondering if Dsho (who  
>>> wrote the
>>> plug-in) could enlighten me.
>>>
>>>     Many thanks in advance for help with this.
>>>
>>> --Larry
>>>
>>> I am running a Mac, with system version 10.5.6, a 2 x 3.2 GHz Quad-
>>> Core
>>> Intel Xeon processor, and 6 GB of 800 MHz DDR2 FB-DIMM memory.
>>>
>>> --Larry Anovitz
>>>
>>>
>>> --
>>> Dr. Lawrence M. Anovitz
>>> MS 6110 PO Box 2008
>>> Aqueous and Geochemistry Group
>>> Oak Ridge National Laboratory
>>> Oak Ridge, Tennessee 37831-6110
>>>
>>> 865-574-5034 : phone
>>> 865-574-4961 : fax
>>>
>>> [hidden email]
>>>
>>>
>>>
>>>
>>>
>>> On 4/6/09 10:46 AM, "Gabriel Landini" <[hidden email]> wrote:
>>>
>>>> Larry,
>>>> Are you sure you do not mean the "two-point correlation function"?
>>>> That is a
>>>> commonly used technique in condensed matter physics (I have seen
>>>> it several
>>>> times used to characterise fractal agglormerates and porous
>>>> materials).
>>>>
>>>> Dsho wrote a plugin some time ago. I have not used it, though:
>>>>
>>>> http://wbgn013.biozentrum.uni-wuerzburg.de/ImageJ/two-point-
>>>> correlation.html
>>>>
>>>> Cheers,
>>>>
>>>> G.

[Gabriel Landini]

Hi Larry,

I think that the two procedures you are comparing are different.

What I suggested was this:

http://books.google.co.uk/books?id=dSrgqo8POYIC&pg=PA22&lpg=PA22&dq=:
+TWO+POINT+CORRELATION+FUNCTION+fractal&source=bl&ots=TMbKJT__fQ&sig=gghkBZ4eE51FW_Ufgoie1KtMpos&hl=en&ei=khgNSqmgGMKrjAf8s6i0Cg&sa=X&oi=book_result&ct=result&resnum=7#PPA22,M1

In case you cannot get the link above, this is from Fractal Growth Phenomena
(page 22) by T. Vicsek: "The expectation value that two points separated by r
belong to the structure. [...] ...the probability of finding a particle at the
position r + r' if there is one at r'. "

For fractal objects, this function is a power law.
Here is some recipe (look for "Correlation Dimension")
http://sprott.physics.wisc.edu/phys505/lect12.htm

http://sprott.physics.wisc.edu/phys505/images/corrdim.gif

I hope this clarifies it.
Regards

Gabriel

[Michael Schmid]

Hi everyone,

after a few checks to see whether it works correctly, I have uploaded  
my "Radially Averaged Autocorrelation" to the documentation wiki:
   http://imagejdocu.tudor.lu/doku.php?
id=macro:radially_averaged_autocorrelation

I have also updated the documentation of the "Radial Distribution  
Function" macro, to make the difference between the two more clear:
   http://imagejdocu.tudor.lu/doku.php?
id=macro:radial_distribution_function

Michael