Memory leak

[Gib Bogle]

I have written a plugin that causes a memory leak.  Each time I run the
plugin on a single image (3600 kB TIF) the memory monitor shows about 14
MB of memory is not released.

The plugin does a variety of operations:
Polynomial Fit (by Bob Dougherty)
Image Calculator
Multiply
Bandpass Filter
Threshold
Analyze Particles

At this stage I can't tell where the memory is being retained.  I'm
wondering if there is a general rule for ImageJ plugin development to
avoid this problem.  If I was just looking at single images it wouldn't
be a concern, but I want to process sets of images (e.g. 100s) and I can
run out of available memory.

Thanks
Gib

[Wayne Rasband]

> I have written a plugin that causes a memory leak.  Each time I run
> the plugin on a single image (3600 kB TIF) the memory monitor shows
> about 14 MB of memory is not released.
>
> The plugin does a variety of operations:
> Polynomial Fit (by Bob Dougherty)
> Image Calculator
> Multiply
> Bandpass Filter
> Threshold
> Analyze Particles
>
> At this stage I can't tell where the memory is being retained.  I'm
> wondering if there is a general rule for ImageJ plugin development to
> avoid this problem.  If I was just looking at single images it
> wouldn't be a concern, but I want to process sets of images (e.g.
> 100s) and I can run out of available memory.

You can monitor memory usage by starting the Memory Monitor
(Plugins>Utilities>Monitor Memory) and then running the plugin. While
it is running, periodically click in the ImageJ status bar to force the
Java garbage collector to run. Try adding two or three second delays
[IJ.wait(2000)] between operations to help determine which one is
retaining the memory.

-wayne

[Gib Bogle]

Wayne Rasband wrote:

>> I have written a plugin that causes a memory leak.  Each time I run
>> the plugin on a single image (3600 kB TIF) the memory monitor shows
>> about 14 MB of memory is not released.
>>
>> The plugin does a variety of operations:
>> Polynomial Fit (by Bob Dougherty)
>> Image Calculator
>> Multiply
>> Bandpass Filter
>> Threshold
>> Analyze Particles
>>
>> At this stage I can't tell where the memory is being retained.  I'm
>> wondering if there is a general rule for ImageJ plugin development to
>> avoid this problem.  If I was just looking at single images it
>> wouldn't be a concern, but I want to process sets of images (e.g.
>> 100s) and I can run out of available memory.
>
>
> You can monitor memory usage by starting the Memory Monitor
> (Plugins>Utilities>Monitor Memory) and then running the plugin. While
> it is running, periodically click in the ImageJ status bar to force
> the Java garbage collector to run. Try adding two or three second
> delays [IJ.wait(2000)] between operations to help determine which one
> is retaining the memory.
>
> -wayne

I think I've located the (or one) source of the memory leak, and
surprisingly it doesn't seem to be related to any of the plugins.  When
I open an 4.7 MB RGB tif file the memory usage goes up to about 10.5
MB.  I do Image > Color > RGB Split and the memory used is 18 MB.  When
I close the three image windows the memory used drops to 10.5 MB.  Is
this an ImageJ problem, or is there a way for me to force this memory to
be freed?

Thanks
Gib

[Wayne Rasband]

I don't see this problem on my Mac running ImageJ 1.38h and Java 1.6.  
When I open a 4.7MB RGB image memory usage goes to 10MB. Memory usage  
drop to 5.5MB when I use the Image>Color>RGB Split command. It drops  
to less than 1MB when I close the three images. I monitor memory  
usage by clicking in the status bar. Sometimes it takes several  
clicks to reclaim all the memory.

With ImageJ 1.38i, due by the end of the week, the 4.7MB image uses  
5.5MB when opened and memory usage drops to 4.3MB after using the RGB  
Split command. ImageJ 1.38i opens RGB images using half the memory  
required by earlier versions of ImageJ because the AWT Image needed  
to display the image shares the pixel array with the ColorProcessor.  
ImageJ 1.38h did something similar with 8-bit images. Pixel array  
sharing only works with Java 1.6 due to due to bugs in earlier  
versions of Java.

-wayne

On Jan 29, 2007, at 5:22 PM, Gib Bogle wrote:

> Wayne Rasband wrote:
>
>>> I have written a plugin that causes a memory leak.  Each time I  
>>> run the plugin on a single image (3600 kB TIF) the memory monitor  
>>> shows about 14 MB of memory is not released.
>>>
>>> The plugin does a variety of operations:
>>> Polynomial Fit (by Bob Dougherty)
>>> Image Calculator
>>> Multiply
>>> Bandpass Filter
>>> Threshold
>>> Analyze Particles
>>>
>>> At this stage I can't tell where the memory is being retained.  
>>> I'm wondering if there is a general rule for ImageJ plugin  
>>> development to avoid this problem.  If I was just looking at  
>>> single images it wouldn't be a concern, but I want to process  
>>> sets of images (e.g. 100s) and I can run out of available memory.
>>
>>
>> You can monitor memory usage by starting the Memory Monitor  
>> (Plugins>Utilities>Monitor Memory) and then running the plugin.  
>> While it is running, periodically click in the ImageJ status bar  
>> to force the Java garbage collector to run. Try adding two or  
>> three second delays [IJ.wait(2000)] between operations to help  
>> determine which one is retaining the memory.
>>
>> -wayne
>
> I think I've located the (or one) source of the memory leak, and  
> surprisingly it doesn't seem to be related to any of the plugins.  
> When I open an 4.7 MB RGB tif file the memory usage goes up to  
> about 10.5 MB.  I do Image > Color > RGB Split and the memory used  
> is 18 MB.  When I close the three image windows the memory used  
> drops to 10.5 MB.  Is this an ImageJ problem, or is there a way for  
> me to force this memory to be freed?
>
> Thanks
> Gib

[Gib Bogle]

Hi Wayne,
I'm not at my work machine at the moment, but it is a Windows 2000  
box.  I installed the latest ImageJ version a couple of weeks ago, and  
today I installed the latest revision of Java 1.5 (something like  
1.5.11).  The memory is definitely taken - waiting and clicking the  
status bar many times has no effect.

Gib

Quoting Rasband Wayne <[hidden email]>:

> I don't see this problem on my Mac running ImageJ 1.38h and Java 1.6.
> When I open a 4.7MB RGB image memory usage goes to 10MB. Memory usage
> drop to 5.5MB when I use the Image>Color>RGB Split command. It drops to
> less than 1MB when I close the three images. I monitor memory usage by
> clicking in the status bar. Sometimes it takes several clicks to
> reclaim all the memory.
>
> With ImageJ 1.38i, due by the end of the week, the 4.7MB image uses
> 5.5MB when opened and memory usage drops to 4.3MB after using the RGB
> Split command. ImageJ 1.38i opens RGB images using half the memory
> required by earlier versions of ImageJ because the AWT Image needed to
> display the image shares the pixel array with the ColorProcessor.
> ImageJ 1.38h did something similar with 8-bit images. Pixel array
> sharing only works with Java 1.6 due to due to bugs in earlier versions
> of Java.
>
> -wayne
>
> On Jan 29, 2007, at 5:22 PM, Gib Bogle wrote:
>
>> Wayne Rasband wrote:
>>
>>>> I have written a plugin that causes a memory leak.  Each time I  
>>>> run the plugin on a single image (3600 kB TIF) the memory monitor  
>>>>  shows about 14 MB of memory is not released.
>>>>
>>>> The plugin does a variety of operations:
>>>> Polynomial Fit (by Bob Dougherty)
>>>> Image Calculator
>>>> Multiply
>>>> Bandpass Filter
>>>> Threshold
>>>> Analyze Particles
>>>>
>>>> At this stage I can't tell where the memory is being retained.    
>>>> I'm wondering if there is a general rule for ImageJ plugin  
>>>> development to avoid this problem.  If I was just looking at  
>>>> single images it wouldn't be a concern, but I want to process  
>>>> sets of images (e.g. 100s) and I can run out of available memory.
>>>
>>>
>>> You can monitor memory usage by starting the Memory Monitor  
>>> (Plugins>Utilities>Monitor Memory) and then running the plugin.  
>>> While it is running, periodically click in the ImageJ status bar  
>>> to force the Java garbage collector to run. Try adding two or  
>>> three second delays [IJ.wait(2000)] between operations to help  
>>> determine which one is retaining the memory.
>>>
>>> -wayne
>>
>> I think I've located the (or one) source of the memory leak, and  
>> surprisingly it doesn't seem to be related to any of the plugins.    
>> When I open an 4.7 MB RGB tif file the memory usage goes up to  
>> about 10.5 MB.  I do Image > Color > RGB Split and the memory used  
>> is 18 MB. When I close the three image windows the memory used  
>> drops to 10.5 MB.  Is this an ImageJ problem, or is there a way for  
>>  me to force this memory to be freed?
>>
>> Thanks
>> Gib

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

[Albert Cardona]

Hi Gib,

I have found only three instances in all my ImageJ-related java
programming where memory leaks occurred:

- you create an ij.ImagePlus on which getImage() has been called, and
flush() is never called. Note that if you go through the standard
ImagePlus.show() to create an ImageWindow, on closing that ImageWindow
will call flush() and then there's no leak.

- you create a java.awt.Image of any sort, and then never call flush()
on it.

- you register a listener somewhere that holds onto any of the above
image types, direclty or indirectly through a pointer to your plugin,
and you don't remove the listener when your plugin exits.

- you are using static fields that hold directly or indirectly images,
and manage not to get garbage collected (for instance, by calling
IJ.register() to prevent so, or by closed loop pointer references).


Hope it helps.

Albert

[Gib Bogle]

Albert Cardona wrote:

> Hi Gib,
>
> I have found only three instances in all my ImageJ-related java
> programming where memory leaks occurred:
>
> - you create an ij.ImagePlus on which getImage() has been called, and
> flush() is never called. Note that if you go through the standard
> ImagePlus.show() to create an ImageWindow, on closing that ImageWindow
> will call flush() and then there's no leak.
>
> - you create a java.awt.Image of any sort, and then never call flush()
> on it.
>
> - you register a listener somewhere that holds onto any of the above
> image types, direclty or indirectly through a pointer to your plugin,
> and you don't remove the listener when your plugin exits.
>
> - you are using static fields that hold directly or indirectly images,
> and manage not to get garbage collected (for instance, by calling
> IJ.register() to prevent so, or by closed loop pointer references).

Thanks Albert, but I've tracked the problem to one of ImageJ's functions, Image>Color>RGB Split, so it isn't related to my plugin code.  
I have ImageJ version 1.37v, Java version 1.5.0_11, Windows 2000 Service Pack 4.  Every time I open a 4.7 MB tif file, do RGB Split, then close the 3 images, the memory usage increases by about 10 MB, and this memory is never freed.
I'd be grateful if someone else using Windows 2000 could check this on their system.  I need to perform this operation many times and I run out of memory before I'm finished.

[Albert Cardona]

Hi Gib,

I can confirm that "RGB Split" is causing a leak. I've been processing
stacks for about 2 hours using precisely the RGB Split a lot, and I can
see some unreleased 18 Mb of memory (compared to the about 2 Gb worth of
images I've opened, it's rather small, so it may be sneaky to find).
This looks to me like a bug introduced in the late memory-saving
routines involving images, but I may be wrong.

Have you tracked the exact code lines responsible?

I'm using java 1.6.0 on linux x86

Albert

>
> Thanks Albert, but I've tracked the problem to one of ImageJ's
> functions, Image>Color>RGB Split, so it isn't related to my plugin
> code.  I have ImageJ version 1.37v, Java version 1.5.0_11, Windows
> 2000 Service Pack 4.  Every time I open a 4.7 MB tif file, do RGB
> Split, then close the 3 images, the memory usage increases by about 10
> MB, and this memory is never freed.

[Gib Bogle]

Albert, the behaviour I'm seeing is creating much more lost memory than
this.  The amount of memory lost is about twice the size of the tif
file.  I should emphasize that this occurs when doing nothing else other
than File>Open and Image>Color>RGB Split.  That is, immediately after
starting ImageJ:

Plugins>Utilities>Monitor Memory
File>Open test.tif
Image>Color>RGB Split
close 3 image windows

Memory usage is now about 10 MB.  Repeat the above steps and 10 MB is
added to the memory usage each time (when test.tif is 4.7 MB).

Gib

Albert Cardona wrote:

> Hi Gib,
>
> I can confirm that "RGB Split" is causing a leak. I've been processing
> stacks for about 2 hours using precisely the RGB Split a lot, and I
> can see some unreleased 18 Mb of memory (compared to the about 2 Gb
> worth of images I've opened, it's rather small, so it may be sneaky to
> find). This looks to me like a bug introduced in the late
> memory-saving routines involving images, but I may be wrong.
>
> Have you tracked the exact code lines responsible?
>
> I'm using java 1.6.0 on linux x86
>
> Albert
>
>>
>> Thanks Albert, but I've tracked the problem to one of ImageJ's
>> functions, Image>Color>RGB Split, so it isn't related to my plugin
>> code.  I have ImageJ version 1.37v, Java version 1.5.0_11, Windows
>> 2000 Service Pack 4.  Every time I open a 4.7 MB tif file, do RGB
>> Split, then close the 3 images, the memory usage increases by about
>> 10 MB, and this memory is never freed.
>

[Albert Cardona]

Gib,

I can't reproduce the problem you are describing. Over here, with the
latest ij.jar 1.38h running on java 1.6.0, the problem is very much
another then (or reflect the difference between java 1.5.0 and 1.6.0).
Both are related to the RGB Split, so I'll look at the diff with 1.37v
for the ij/plugin/filter/RGBStackSplitter.java file ... latest known to
work just fine.

In any case Wayne may be looking at this as well.

Albert



>>> Every time I open a 4.7 MB tif file, do RGB Split, then close the 3
>>> images, the memory usage increases by about 10 MB, and this memory
>>> is never freed.

[Gabriel Landini]

On Tuesday 30 January 2007 21:37:47 Gib Bogle wrote:

> Albert, the behaviour I'm seeing is creating much more lost memory than
> this.  The amount of memory lost is about twice the size of the tif
> file.  I should emphasize that this occurs when doing nothing else other
> than File>Open and Image>Color>RGB Split.  That is, immediately after
> starting ImageJ:
>
> Plugins>Utilities>Monitor Memory
> File>Open test.tif
> Image>Color>RGB Split
> close 3 image windows
>
> Memory usage is now about 10 MB.  Repeat the above steps and 10 MB is
> added to the memory usage each time (when test.tif is 4.7 MB).

I am testing this in linux with java 1.6

for(i=0;i<100;i++){
 open("/home/gabriel/ImageJ/f00.tif");
 run("RGB Split");
 close();
 close();
 close();
}

The file is a 9MB rgb. One run of the macro (100 times open split and close)
does not increase the memory use as reported by the memory monitor.


Gib, although you said in a message that you installed the latest version of
IJ, then you said :

> I have ImageJ version 1.37v, Java version 1.5.0_11, Windows

Note that 1.37 is not the latest but 1.38h is.

Check:

http://rsbweb.nih.gov/ij/notes.html

Not sure if this applies to the behaviour you found but lots of people report
bugs on old versions. It is more useful -if one finds some strange behaviour-
to upgrade first to the ij.jar file using the link above and see if the
problem persists.

Cheers,

Gabriel

[Albert Cardona]

Hi Gabriel,

I run your macro, with java 1.6.0 ij138h, both with the baboon.tif image
and a 10-slice stack of baboon images, and I don't see any memory
increase at all either. It decreases if anything. Now I'm puzzled, can't
explain the leak of this morning after working on so many different RGB
stacks.

As for the code, 1.37v and 1.38h contain identical RGBStackSplitter.java
files, and nothing has changed in the
ColorProcessor.getRGB(int[],int[],int[]) method either.

Albert

[Gib Bogle]

Gabriel Landini wrote:

>On Tuesday 30 January 2007 21:37:47 Gib Bogle wrote:
>  
>
>>Albert, the behaviour I'm seeing is creating much more lost memory than
>>this.  The amount of memory lost is about twice the size of the tif
>>file.  I should emphasize that this occurs when doing nothing else other
>>than File>Open and Image>Color>RGB Split.  That is, immediately after
>>starting ImageJ:
>>
>>Plugins>Utilities>Monitor Memory
>>File>Open test.tif
>>Image>Color>RGB Split
>>close 3 image windows
>>
>>Memory usage is now about 10 MB.  Repeat the above steps and 10 MB is
>>added to the memory usage each time (when test.tif is 4.7 MB).
>>    
>>
>
>I am testing this in linux with java 1.6
>
>for(i=0;i<100;i++){
> open("/home/gabriel/ImageJ/f00.tif");
> run("RGB Split");
> close();
> close();
> close();
>}
>
>The file is a 9MB rgb. One run of the macro (100 times open split and close)
>does not increase the memory use as reported by the memory monitor.
>
>
>Gib, although you said in a message that you installed the latest version of
>IJ, then you said :
>
>  
>
>>I have ImageJ version 1.37v, Java version 1.5.0_11, Windows
>>    
>>
>
>Note that 1.37 is not the latest but 1.38h is.
>
>Check:
>
>http://rsbweb.nih.gov/ij/notes.html
>
>Not sure if this applies to the behaviour you found but lots of people report
>bugs on old versions. It is more useful -if one finds some strange behaviour-
>to upgrade first to the ij.jar file using the link above and see if the
>problem persists.
>
>Cheers,
>
>Gabriel
>  
>

Hi Gabriel,

I can't recall the site where I installed 1.37v from, but it claimed to
be the latest version.  I have now done as you suggest (thanks) and
installed 1.38h.  The problem persists.  BTW, for this testing I'm doing
the file open and RGB split manually, not using a macro as you are.  
Anyway, I suspect that the problem is with W2K, since Mac and Linux
don't seem to show it.

Gib

[Wayne Rasband]

ImageJ 1.38i fixes a bug in the Image>Color>RGB Split command that
could cause it to leak memory. It also opens RGB images using half as
much memory if you are using Java 1.6.

-wayne

On Jan 29, 2007, at 5:22 PM, Gib Bogle wrote:

> Wayne Rasband wrote:
>
>>> I have written a plugin that causes a memory leak.  Each time I run
>>> the plugin on a single image (3600 kB TIF) the memory monitor shows
>>> about 14 MB of memory is not released.
>>>
>>> The plugin does a variety of operations:
>>> Polynomial Fit (by Bob Dougherty)
>>> Image Calculator
>>> Multiply
>>> Bandpass Filter
>>> Threshold
>>> Analyze Particles
>>>
>>> At this stage I can't tell where the memory is being retained.  I'm
>>> wondering if there is a general rule for ImageJ plugin development
>>> to avoid this problem.  If I was just looking at single images it
>>> wouldn't be a concern, but I want to process sets of images (e.g.
>>> 100s) and I can run out of available memory.
>>
>>
>> You can monitor memory usage by starting the Memory Monitor
>> (Plugins>Utilities>Monitor Memory) and then running the plugin. While
>> it is running, periodically click in the ImageJ status bar to force
>> the Java garbage collector to run. Try adding two or three second
>> delays [IJ.wait(2000)] between operations to help determine which one
>> is retaining the memory.
>>
>> -wayne
>
> I think I've located the (or one) source of the memory leak, and
> surprisingly it doesn't seem to be related to any of the plugins.  
> When I open an 4.7 MB RGB tif file the memory usage goes up to about
> 10.5 MB.  I do Image > Color > RGB Split and the memory used is 18 MB.
>  When I close the three image windows the memory used drops to 10.5
> MB.  Is this an ImageJ problem, or is there a way for me to force this
> memory to be freed?
>
> Thanks
> Gib
>

[Jon Harman]

Hi,

Will it open Jpegs faster?  How do I get it?  On the downloads page I
can find only 1.37 or 1.36.

Jon


Wayne Rasband wrote:

> ImageJ 1.38i fixes a bug in the Image>Color>RGB Split command that
> could cause it to leak memory. It also opens RGB images using half as
> much memory if you are using Java 1.6.
>
> -wayne
>
> On Jan 29, 2007, at 5:22 PM, Gib Bogle wrote:
>
>> Wayne Rasband wrote:
>>
>>>> I have written a plugin that causes a memory leak.  Each time I run
>>>> the plugin on a single image (3600 kB TIF) the memory monitor shows
>>>> about 14 MB of memory is not released.
>>>>
>>>> The plugin does a variety of operations:
>>>> Polynomial Fit (by Bob Dougherty)
>>>> Image Calculator
>>>> Multiply
>>>> Bandpass Filter
>>>> Threshold
>>>> Analyze Particles
>>>>
>>>> At this stage I can't tell where the memory is being retained.  I'm
>>>> wondering if there is a general rule for ImageJ plugin development
>>>> to avoid this problem.  If I was just looking at single images it
>>>> wouldn't be a concern, but I want to process sets of images (e.g.
>>>> 100s) and I can run out of available memory.
>>>
>>>
>>> You can monitor memory usage by starting the Memory Monitor
>>> (Plugins>Utilities>Monitor Memory) and then running the plugin.
>>> While it is running, periodically click in the ImageJ status bar to
>>> force the Java garbage collector to run. Try adding two or three
>>> second delays [IJ.wait(2000)] between operations to help determine
>>> which one is retaining the memory.
>>>
>>> -wayne
>>
>> I think I've located the (or one) source of the memory leak, and
>> surprisingly it doesn't seem to be related to any of the plugins.  
>> When I open an 4.7 MB RGB tif file the memory usage goes up to about
>> 10.5 MB.  I do Image > Color > RGB Split and the memory used is 18
>> MB.  When I close the three image windows the memory used drops to
>> 10.5 MB.  Is this an ImageJ problem, or is there a way for me to
>> force this memory to be freed?
>>
>> Thanks
>> Gib
>>
>

[jmutterer]

Jon,

The 'Downloads' page only lets you download the latest 'stable full
release'.
To obtain the very latest version of ij.jar to upgrade your installation,
follow the 'upgrade' link at the top of the 'News' page, and read the
instructions.
The 'News' page also lists all the new features and bug fixes of all
intermediate versions.

Jerome

On 2/1/07, Jon Harman <[hidden email]> wrote:

>
> Hi,
>
> Will it open Jpegs faster?  How do I get it?  On the downloads page I
> can find only 1.37 or 1.36.
>
> Jon
>
>
> Wayne Rasband wrote:
> > ImageJ 1.38i fixes a bug in the Image>Color>RGB Split command that
> > could cause it to leak memory. It also opens RGB images using half as
> > much memory if you are using Java 1.6.
> >
> > -wayne
> >
> > On Jan 29, 2007, at 5:22 PM, Gib Bogle wrote:
> >
> >> Wayne Rasband wrote:
> >>
> >>>> I have written a plugin that causes a memory leak.  Each time I run
> >>>> the plugin on a single image (3600 kB TIF) the memory monitor shows
> >>>> about 14 MB of memory is not released.
> >>>>
> >>>> The plugin does a variety of operations:
> >>>> Polynomial Fit (by Bob Dougherty)
> >>>> Image Calculator
> >>>> Multiply
> >>>> Bandpass Filter
> >>>> Threshold
> >>>> Analyze Particles
> >>>>
> >>>> At this stage I can't tell where the memory is being retained.  I'm
> >>>> wondering if there is a general rule for ImageJ plugin development
> >>>> to avoid this problem.  If I was just looking at single images it
> >>>> wouldn't be a concern, but I want to process sets of images (e.g.
> >>>> 100s) and I can run out of available memory.
> >>>
> >>>
> >>> You can monitor memory usage by starting the Memory Monitor
> >>> (Plugins>Utilities>Monitor Memory) and then running the plugin.
> >>> While it is running, periodically click in the ImageJ status bar to
> >>> force the Java garbage collector to run. Try adding two or three
> >>> second delays [IJ.wait(2000)] between operations to help determine
> >>> which one is retaining the memory.
> >>>
> >>> -wayne
> >>
> >> I think I've located the (or one) source of the memory leak, and
> >> surprisingly it doesn't seem to be related to any of the plugins.
> >> When I open an 4.7 MB RGB tif file the memory usage goes up to about
> >> 10.5 MB.  I do Image > Color > RGB Split and the memory used is 18
> >> MB.  When I close the three image windows the memory used drops to
> >> 10.5 MB.  Is this an ImageJ problem, or is there a way for me to
> >> force this memory to be freed?
> >>
> >> Thanks
> >> Gib
> >>
> >
>