Memory consumption

[Mathias Beller]

Hi!

I am a new ImageJ user running into memory problems performing batch
image processing. I am going to analyze about 75 000 pictures with an
image segmentation algorithm using the macro script language. The macro
contains: reading the pictures from a source folder (one at a time),
adjusting brightness and contrast, performing a background subtraction,
OTSU thresholding and the particle analysis command. The generated
outline picture as well as the contents of the summary and results text
Windows are subsequently saved into separate destination folders, and
everything is closed again.

And here comes the problem: watching the memory monitor shows me an
increasing filling of the memory heap (memory is set to 1 GByte). I
included several Garbage Collection commands using the GarbageCollect
Plugin (found here on the Listserv - many thanks for that), however the
memory is not emptied. I cannot see any traces of open windows or
something similar that could explain the memory consumption. It would be
really great if someone would have an idea to get around the annoying
re-starting of the script about every 1200 pictures.

Thank you very much for your help in advance,
Best,
Mathias

[Mark R. Besonen]

At 10:24 PM 2/6/2006, you wrote:

>Hi!
>
>I am a new ImageJ user running into memory problems performing batch
>image processing. I am going to analyze about 75 000 pictures with
>an image segmentation algorithm using the macro script language. The
>macro contains: reading the pictures from a source folder (one at a
>time), adjusting brightness and contrast, performing a background
>subtraction, OTSU thresholding and the particle analysis command.
>The generated outline picture as well as the contents of the summary
>and results text Windows are subsequently saved into separate
>destination folders, and everything is closed again.
>
>And here comes the problem: watching the memory monitor shows me an
>increasing filling of the memory heap (memory is set to 1 GByte). I
>included several Garbage Collection commands using the
>GarbageCollect Plugin (found here on the Listserv - many thanks for
>that), however the memory is not emptied. I cannot see any traces of
>open windows or something similar that could explain the memory
>consumption. It would be really great if someone would have an idea
>to get around the annoying re-starting of the script about every 1200 pictures.
>
>Thank you very much for your help in advance,
>Best,
>Mathias

Hello Mathias,

         Often a single call to the GarbageCollect plugin does not
free up all the memory--sometimes you have to call it 2 or 3 times in
a row.  My solution is to make a simple loop, and run the command 5 times.

         If you are seeing the memory growing progressively via the
Memory Monitor, perhaps you are calling the GarbageCollect plugin
from a location within your script that is not so effective.  If you
send your script, perhaps the many eyes here could make some
suggestions to help.

         When I build a complicated script with multiple loops, it's
easy for my brain to lose track of exactly what is happening.  So as
I am debugging it I use lots of print statements to the ImageJ log
that help me track the flow of the macro.  To make sure the
GarbageCollect plugin is being called effectively, perhaps you could
add a print statement before or after it, or even within the
GarbageCollect loop itself.  Yes, it might fill up the log window
quickly, but it might also provide you with some information about
how effective your call to the GarbageCollect plugin is.

Mark Besonen
UMass Geosciences

[Jo C. Bruusgaard]

hello!

I have run into a problem when I use ImageJ to extract 3D coordinates  
from a stack. The x,y coordinates are written nicely to the  
"measurements"-window. However, I am not able to get info about which  
slice in the stack the measurement is taken from. Does anyone know  
how to do this?


Dr. Jo C. Bruusgaard
Department of Molecular Biosciences
University of Oslo
PO Box 1041, Blindern
N-0316 Oslo
NORWAY
Phone: [phone]
Fax: [phone]
Mobile: [phone]

[Per Christian Henden]

On Tuesday 07 February 2006 14:26, Jo C. Bruusgaard wrote:
> hello!
>
> I have run into a problem when I use ImageJ to extract 3D coordinates
> from a stack. The x,y coordinates are written nicely to the
> "measurements"-window. However, I am not able to get info about which
> slice in the stack the measurement is taken from. Does anyone know
> how to do this?

Hi,

Are you talking about the Analyze->Measure tool? Indeed, the mass center and
centroid reported by that tool seem to take only the current stack slice into
account (ImageJ v. 1.34s).
 
This missing z-coordinate is easy to "fix", but question is if supposed to
work like that, and if we fix it, will code/macros depending on it working
the way it does now break?

Wayne, what do you think?

--
Per Christian Henden

[Greg Joss]

If you check 'Display Label' in Analyze>Set Measurements...
eg in macrozzzz:
run("Set Measurements...", "area centroid center bounding integrated
display redirect=None decimal=0");
IJ measure or analyzeParticles will record the slice in the Label
field appended with a preceding colon ":".

Unfortunately, this is not directly accessible as there is macro
function equivalent to
getResult("Label", row);
even though there is a
setResult("Label", row, string);

Although it is a bit of a sledgehammer to crack a nut, the following
macro code will read the ResultsTable
and add a Column "Slice" with the value extracted from the Label
field. (It is quite quick even for thousands of particle result rows).

Your macros can then read X,Y and Z "Slice" columns via
getResult("Slice",row);.
__________________________________________
getSliceFromLabelCol();

function getSliceFromLabelCol(){
        selectWindow("Results");
        rr=split(getInfo(),"\n");
        for(i=1;i<lengthOf(rr);i++){
                rrr=split(rr[i],"\t");
                ss=split(rrr[1],":");
                ls=lengthOf(ss);
                if(ls>1){
                        slice=0+ss[ls-1];
                        setResult("Slice",i-1,slice);
                }
        }
        updateResults();
}
_________________________________________

>On Tuesday 07 February 2006 14:26, Jo C. Bruusgaard wrote:
>>  hello!
>>
>>  I have run into a problem when I use ImageJ to extract 3D coordinates
>>  from a stack. The x,y coordinates are written nicely to the
>>  "measurements"-window. However, I am not able to get info about which
>>  slice in the stack the measurement is taken from. Does anyone know
>>  how to do this?
>
>Hi,
>
>Are you talking about the Analyze->Measure tool? Indeed, the mass center and
>centroid reported by that tool seem to take only the current stack slice into
>account (ImageJ v. 1.34s).
>
>This missing z-coordinate is easy to "fix", but question is if supposed to
>work like that, and if we fix it, will code/macros depending on it working
>the way it does now break?
>
>Wayne, what do you think?
>
>--
>Per Christian Henden

--
__
Greg Joss,
Department of Biological Sciences, Phone: [phone]
Macquarie University,              Email: [hidden email]
North Ryde, (Sydney,) NSW 2109, Australia

[Wayne Rasband]

In ImageJ 1.35p, available at <http://rsb.info.nih.gov/ij/upgrade/>,  
you can record the slice number by checking "Slice Number" in  
Analyze>Set Measurements.

-wayne

On Feb 13, 2006, at 9:53 PM, Greg Joss wrote:

> If you check 'Display Label' in Analyze>Set Measurements...
> eg in macrozzzz:
> run("Set Measurements...", "area centroid center bounding  
> integrated display redirect=None decimal=0");
> IJ measure or analyzeParticles will record the slice in the Label  
> field appended with a preceding colon ":".
>
> Unfortunately, this is not directly accessible as there is macro  
> function equivalent to
> getResult("Label", row);
> even though there is a
> setResult("Label", row, string);
>
> Although it is a bit of a sledgehammer to crack a nut, the  
> following macro code will read the ResultsTable
> and add a Column "Slice" with the value extracted from the Label  
> field. (It is quite quick even for thousands of particle result rows).
>
> Your macros can then read X,Y and Z "Slice" columns via getResult
> ("Slice",row);.
> __________________________________________
> getSliceFromLabelCol();
>
> function getSliceFromLabelCol(){
> selectWindow("Results");
> rr=split(getInfo(),"\n");
> for(i=1;i<lengthOf(rr);i++){
> rrr=split(rr[i],"\t");
> ss=split(rrr[1],":");
> ls=lengthOf(ss);
> if(ls>1){
> slice=0+ss[ls-1];
> setResult("Slice",i-1,slice);
> }
> }
> updateResults();
> }
> _________________________________________
>
>
>> On Tuesday 07 February 2006 14:26, Jo C. Bruusgaard wrote:
>>>  hello!
>>>
>>>  I have run into a problem when I use ImageJ to extract 3D  
>>> coordinates
>>>  from a stack. The x,y coordinates are written nicely to the
>>>  "measurements"-window. However, I am not able to get info about  
>>> which
>>>  slice in the stack the measurement is taken from. Does anyone know
>>>  how to do this?
>>
>> Hi,
>>
>> Are you talking about the Analyze->Measure tool? Indeed, the mass  
>> center and
>> centroid reported by that tool seem to take only the current stack  
>> slice into
>> account (ImageJ v. 1.34s).
>>
>> This missing z-coordinate is easy to "fix", but question is if  
>> supposed to
>> work like that, and if we fix it, will code/macros depending on it  
>> working
>> the way it does now break?
>>
>> Wayne, what do you think?
>>
>> --
>> Per Christian Henden
>
>
> --
> __
> Greg Joss,
> Department of Biological Sciences, Phone: [phone]
> Macquarie University,              Email: [hidden email]
> North Ryde, (Sydney,) NSW 2109, Australia