Normalising XY data by Z-direction in stacks

[jamescleverley]

Hi All,

I am using ImageJ to process spatial XANES maps where we map an area in X-Y and each image is taken at a different energy over a specific element K-edge.  By combining into a stack the Z direction represents energy step and the pixel records intensity.  Because the data is slightly noisy and I am only interested in peak positions I would like to normalise the data in the stack  by z-direction so that the intensities at each X-Y position are scaled throughout the Z-direction to between 0 and 1. In other words if I plot a Z-profile for a single pixel in the stack it will show a profile scaled between 0 and 1.

I have searched high and low but can't seem to find the method to do this exact problem.  Does anyone have a suggestion?

Thanks

James

[Thomson Neil (EAST KENT HOSPITALS UNIVERSITY NHS FOUNDATION TRUST)]

Hi James

I don't know your background, so forgive me if I'm way off the mark...

This should be a pretty trivial problem with some coding. If you haven't looked at the developer<http://rsbweb.nih.gov/ij/developer/index.html> section of he ImageJ website, have a look now. If you've never done any programming then writing a Java plugin might be a bit scary, but the macro language <http://rsbweb.nih.gov/ij/developer/macro/macros.html> should so the trick.

This macro-code snippet might give you an idea on normalisation:

// Make sure image accepts floating point values
run("32-bit");
// get image width x
// get image height y
// get image depth z
for (i=0; i < x; i++){
  for (j=0; i < y; j++){
    // Get max value in the x-y coord
    maxValue = 0;
    for (k=0; i < z; k++){
      setSlice(k);
      currValue = getPixel(x, y);
      if (currValue > maxValue){
        maxValue = currValue;
      }
    }
    // Set new value normalised to max value
    for (k=0; i < z; k++){
      setSlice(k);
      setValue(x, y, getPixel(x, y)/maxValue);
    }
  }
}

Neil

Neil Thomson,
Nuclear Medicine Physics Section,
Medical Physics,
Kent and Canterbury Hospital,
UK. CT1 3NG
[phone]

********************************************************************************************************************

This message may contain confidential information. If you are not the intended recipient please inform the
sender that you have received the message in error before deleting it.
Please do not disclose, copy or distribute information in this e-mail or take any action in reliance on its contents:
to do so is strictly prohibited and may be unlawful.

Thank you for your co-operation.

NHSmail is the secure email and directory service available for all NHS staff in England and Scotland
NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and GSi recipients
NHSmail provides an email address for your career in the NHS and can be accessed anywhere
For more information and to find out how you can switch, visit www.connectingforhealth.nhs.uk/nhsmail

********************************************************************************************************************

[jamescleverley]

Thanks Neil, That was all the pointer I needed to get a modified version of your code snippet to do what I needed.  Next will be to think about effective processing order to maximise speed .... Code below if anyone is interested:

// Make sure image accepts floating point values

run("32-bit");

// get image width x
x = getWidth;
// get image height y
y = getHeight;
// get image depth z
z = nSlices;

for (i=0; i < x; i++){
  for (j=0; j < y; j++){
    // Get max and min value in the x-y coord
    maxValue = 0;
    minValue = 10000;
    for (k=1; k <= z; k++){
     
      setSlice(k);
      currValue = getPixel(i, j);
      if (currValue > maxValue){
        maxValue = currValue;
      }
      if (currValue < minValue){
        minValue = currValue;
      }
    }
    // Set new value normalised to max value
    for (k=1; k <= z; k++){
      setSlice(k);
      currPixelValue = getPixel(i,j);
      newPixelValue = (currPixelValue-(minValue))/(maxValue-minValue);
      putPixel(i, j, (newPixelValue));
    }
  }
}

Much appreciated,

Cheers

James

Dr James Cleverley 
Senior Geochemist
Stream Leader | Mineral Systems & Targeting
Minerals Down Under Research Flagship
CSIRO Earth Science & Resource Engineering
Phone: [phone] | Mobile: [phone]
[hidden email] | www.csiro.au | www.csiro.au/
Address: Australian Resources Research Centre, PO Box 1130, Bentley, WA, 6102

PLEASE NOTE
The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please delete it immediately and notify the sender by return email. Thank you. To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus, interception or interference.
Please consider the environment before printing this email.

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Thomson Neil (EAST KENT HOSPITALS UNIVERSITY NHS FOUNDATION TRUST)
Sent: Thursday, 7 April 2011 20:51
To: [hidden email]
Subject: Normalising XY data by Z-direction in stacks

Hi James

I don't know your background, so forgive me if I'm way off the mark...

This should be a pretty trivial problem with some coding. If you haven't looked at the developer<http://rsbweb.nih.gov/ij/developer/index.html> section of he ImageJ website, have a look now. If you've never done any programming then writing a Java plugin might be a bit scary, but the macro language <http://rsbweb.nih.gov/ij/developer/macro/macros.html> should so the trick.

This macro-code snippet might give you an idea on normalisation:

// Make sure image accepts floating point values
run("32-bit");
// get image width x
// get image height y
// get image depth z
for (i=0; i < x; i++){
  for (j=0; i < y; j++){
    // Get max value in the x-y coord
    maxValue = 0;
    for (k=0; i < z; k++){
      setSlice(k);
      currValue = getPixel(x, y);
      if (currValue > maxValue){
        maxValue = currValue;
      }
    }
    // Set new value normalised to max value
    for (k=0; i < z; k++){
      setSlice(k);
      setValue(x, y, getPixel(x, y)/maxValue);
    }
  }
}

Neil

Neil Thomson,
Nuclear Medicine Physics Section,
Medical Physics,
Kent and Canterbury Hospital,
UK. CT1 3NG
[phone]

********************************************************************************************************************

This message may contain confidential information. If you are not the intended recipient please inform the
sender that you have received the message in error before deleting it.
Please do not disclose, copy or distribute information in this e-mail or take any action in reliance on its contents:
to do so is strictly prohibited and may be unlawful.

Thank you for your co-operation.

NHSmail is the secure email and directory service available for all NHS staff in England and Scotland
NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and GSi recipients
NHSmail provides an email address for your career in the NHS and can be accessed anywhere
For more information and to find out how you can switch, visit www.connectingforhealth.nhs.uk/nhsmail

********************************************************************************************************************

[Michael Schmid]

Hi James,

stepping through the pixels one by one in a macro is slower than the  
corresponding ImageJ functions written in java.
This macro avoids the loops:

   run("32-bit");
   original = getImageID();  //remember this image
   run("Z Project...", "start=1 stop=36 projection=[Min Intensity]");
   minimum = getImageID();   //reference to the output image created
   selectImage(original);
   run("Z Project...", "start=1 stop=36 projection=[Max Intensity]");
   maximum = getImageID();
   imageCalculator("subtract create", maximum, minimum); //maxValue-
minValue
   difference = getImageID();
   imageCalculator("subtract stack", original, minimum); //pixel-
minValue
   imageCalculator("divide stack", original, difference);
   // now we have (pixel-minValue)/(maxValue-minValue)
   selectImage(maximum);
   close();
   selectImage(minimum);
   close();
   selectImage(difference);
   close();


Michael
________________________________________________________________

On 7 Apr 2011, at 16:33, James Cleverley wrote:

> Thanks Neil, That was all the pointer I needed to get a modified  
> version of your code snippet to do what I needed.  Next will be to  
> think about effective processing order to maximise speed .... Code  
> below if anyone is interested:
>
> // Make sure image accepts floating point values
>
> run("32-bit");
>
> // get image width x
> x = getWidth;
> // get image height y
> y = getHeight;
> // get image depth z
> z = nSlices;
>
> for (i=0; i < x; i++){
>   for (j=0; j < y; j++){
>     // Get max and min value in the x-y coord
>     maxValue = 0;
>     minValue = 10000;
>     for (k=1; k <= z; k++){
>
>       setSlice(k);
>       currValue = getPixel(i, j);
>       if (currValue > maxValue){
>         maxValue = currValue;
>       }
>       if (currValue < minValue){
>         minValue = currValue;
>       }
>     }
>     // Set new value normalised to max value
>     for (k=1; k <= z; k++){
>       setSlice(k);
>       currPixelValue = getPixel(i,j);
>       newPixelValue = (currPixelValue-(minValue))/(maxValue-minValue);
>       putPixel(i, j, (newPixelValue));
>     }
>   }
> }
>
> Much appreciated,
>
> Cheers
>
> James
>
> Dr James Cleverley
> Senior Geochemist
> Stream Leader | Mineral Systems & Targeting
> Minerals Down Under Research Flagship
> CSIRO Earth Science & Resource Engineering
> Phone: [phone] | Mobile: [phone]
> [hidden email] | www.csiro.au | www.csiro.au/
> Address: Australian Resources Research Centre, PO Box 1130,  
> Bentley, WA, 6102
>
> PLEASE NOTE
> The information contained in this email may be confidential or  
> privileged. Any unauthorised use or disclosure is prohibited. If  
> you have received this email in error, please delete it immediately  
> and notify the sender by return email. Thank you. To the extent  
> permitted by law, CSIRO does not represent, warrant and/or  
> guarantee that the integrity of this communication has been  
> maintained or that the communication is free of errors, virus,  
> interception or interference.
> Please consider the environment before printing this email.
>
> -----Original Message-----
> From: ImageJ Interest Group [mailto:[hidden email]] On Behalf  
> Of Thomson Neil (EAST KENT HOSPITALS UNIVERSITY NHS FOUNDATION TRUST)
> Sent: Thursday, 7 April 2011 20:51
> To: [hidden email]
> Subject: Normalising XY data by Z-direction in stacks
>
> Hi James
>
> I don't know your background, so forgive me if I'm way off the mark...
>
> This should be a pretty trivial problem with some coding. If you  
> haven't looked at the developer<http://rsbweb.nih.gov/ij/developer/ 
> index.html> section of he ImageJ website, have a look now. If  
> you've never done any programming then writing a Java plugin might  
> be a bit scary, but the macro language <http://rsbweb.nih.gov/ij/ 
> developer/macro/macros.html> should so the trick.
>
> This macro-code snippet might give you an idea on normalisation:
>
> // Make sure image accepts floating point values
> run("32-bit");
> // get image width x
> // get image height y
> // get image depth z
> for (i=0; i < x; i++){
>   for (j=0; i < y; j++){
>     // Get max value in the x-y coord
>     maxValue = 0;
>     for (k=0; i < z; k++){
>       setSlice(k);
>       currValue = getPixel(x, y);
>       if (currValue > maxValue){
>         maxValue = currValue;
>       }
>     }
>     // Set new value normalised to max value
>     for (k=0; i < z; k++){
>       setSlice(k);
>       setValue(x, y, getPixel(x, y)/maxValue);
>     }
>   }
> }
>
> Neil
>
> Neil Thomson,
> Nuclear Medicine Physics Section,
> Medical Physics,
> Kent and Canterbury Hospital,
> UK. CT1 3NG
> [phone]
>
> **********************************************************************
> **********************************************
>
> This message may contain confidential information. If you are not  
> the intended recipient please inform the
> sender that you have received the message in error before deleting it.
> Please do not disclose, copy or distribute information in this e-
> mail or take any action in reliance on its contents:
> to do so is strictly prohibited and may be unlawful.
>
> Thank you for your co-operation.
>
> NHSmail is the secure email and directory service available for all  
> NHS staff in England and Scotland
> NHSmail is approved for exchanging patient data and other sensitive  
> information with NHSmail and GSi recipients
> NHSmail provides an email address for your career in the NHS and  
> can be accessed anywhere
> For more information and to find out how you can switch, visit  
> www.connectingforhealth.nhs.uk/nhsmail
>
> **********************************************************************
> **********************************************

[Daniel James White]

Hi James,

On Apr 8, 2011, at 2:13 AM, IMAGEJ automatic digest system wrote:

> Date:    Thu, 7 Apr 2011 16:14:56 +0800
> From:    James Cleverley <[hidden email]>
> Subject: Normalising XY data by Z-direction in stacks
>
> Hi All,
>
> I am using ImageJ to process spatial XANES maps where we map an area in X-Y and each image is taken at a different energy over a specific element K-edge.

Interesting.... in a former life I did EXAFS on molybdenum complexes.... Mop, and ModG molybdate binding proteins....

>  By combining into a stack the Z direction represents energy step and the pixel records intensity.  Because the data is slightly noisy and I am only interested in peak positions I would like to normalise the data in the stack  by z-direction so that the intensities at each X-Y position are scaled throughout the Z-direction to between 0 and 1. In other words if I plot a Z-profile for a single pixel in the stack it will show a profile scaled between 0 and 1.

Ok, so mathematically speaking its easy....
convert the whole image to 32 bit if its not alrady (Image - type- 32 bit)

Measure the mean intensity of each z slice,
and divide each slice by that mean

Now there is the same mean intensity in each z slice,
you can make the plot profile,
save the plot x, y numbers from that

then normalize those from 0 - 1 in a spread sheet, mat lab, R etc.

if you wanted you could make a little macro or script, that
iterates through the z stack , finds the mean of each one,
and divides that slice by that mean.

see
http://pacific.mpi-cbg.de/wiki/index.php/Introduction_into_Macro_Programming
http://pacific.mpi-cbg.de/wiki/index.php/Jython_Scripting 

>
> I have searched high and low but can't seem to find the method to do this exact problem.  Does anyone have a suggestion?
>

you could also pretend the stack is a time sereis, and do a "bleaching  correction"
which attempts to correct images for gradula lodd of intensity down the stack...
but it might not work in your case, if expects a high start, and a gradual loss of signal...
not sure how it works, but you can look at the code and see...


http://www.embl.de/eamnet/html/bleach_correction.html

cheers

Dan



> Thanks
>
> James

Dr. Daniel James White BSc. (Hons.) PhD
Senior Microscopist / Image Visualisation, Processing and Analysis
Light Microscopy and Image Processing Facilities
Max Planck Institute of Molecular Cell Biology and Genetics
Pfotenhauerstrasse 108
01307 DRESDEN
Germany

[phone] (German Mobile)
[phone] (Work phone at MPI-CBG)
[phone] (Fax MPI-CBG LMF)

http://www.bioimagexd.net  BioImageXD
http://pacific.mpi-cbg.de                Fiji -  is just ImageJ (Batteries Included)
http://www.chalkie.org.uk                Dan's Homepages
https://ifn.mpi-cbg.de  Dresden Imaging Facility Network
dan (at) chalkie.org.uk
( white (at) mpi-cbg.de )