memory leaks

[Girault France]

Dear all,

 

I'm having problem using a plugin to segment images :

It causes leak of memory when I use it in a loop on hundreds of images.

I need to detect the problem in the source code, but as I m completely
ignorant is Java, I don't know how to process and which program do I
need to use that allows detecting memory leaks in the code of the
plugin...

Does any body have experience in this already? any advice for a very
very very very easy (for non programmer) software that could be used
??...

 

 

Many thanks

 

france

 

[Mark R. Besonen]

At 07:40 AM 10/27/2006, you wrote:

>Dear all,
>
>I'm having problem using a plugin to segment images :
>
>It causes leak of memory when I use it in a loop on hundreds of images.
>
>I need to detect the problem in the source code, but as I m completely
>ignorant is Java, I don't know how to process and which program do I
>need to use that allows detecting memory leaks in the code of the
>plugin...
>
>Does any body have experience in this already? any advice for a very
>very very very easy (for non programmer) software that could be used
>??...
>
>Many thanks
>
>france

Hello France,

         First of all, I am assuming your "memory leak" is manifested
by an out of memory error.  If not, simply skip this message.

         Anyway, I don't think the problem you've noticed is with
ImageJ or one of the plugins, but with the underlying Java Virtual
Machine.  I have run into similar out of memory conditions when using
a macro to automate processing of a whole series of directories full
of images.  If I just process a single directory at a time with the
macro, I never have a problem.  But when I automate a recursive trawl
through a whole series of directories to use the same macro, I'd run
into out-out-memory errors at random times.

         The JVM does "garbage collection" by itself, and recovers
unused memory on the fly as necessary, so theoretically you should
never run into an out of memory condition.  However, by automating a
routine, I think you can sometimes overpower the JVM's garbage
collection schedule, and actually force an out of memory
condition.  Fortunately, ImageJ has a some built-in tools to help
figure this out, and probably resolve your problem.

         First, I'd suggest you pop up the memory monitor window
(Plugins-->Utilities-->Monitor Memory...), and keep an eye on it as
you run your macro.  When I first start it up on my system, and
without doing anything, it fluctuates around 1-1.5 MB of usage.  As
you start to open/process images, you'll see the memory rise.  When
you shut down those images, theoretically, the memory usage should
fall down to the same original base level of 1-1.5 MB usage, but in
fact, it does not.  After opening up one small image, running a
couple of filters over it, and then shutting down the image, my new
base level of memory usage is in the 7-8 MB range.  And if you were
to do something similar hundreds of times rapidly in succession via a
macro, the problem gets compounded, and can eventually lead to the
out of memory errors you have seen.

         Fortunately, as I mentioned above, ImageJ has a couple of
mechanisms to fix this issue.  In particular, you can have ImageJ
manually force the JVM to do garbage collection.  With you new 7-8 MB
(or whatever) base memory usage, now click the status bar on the main
ImageJ window--you should see the memory usage drop back down to its
original footprint.  In my experience, it sometimes takes multiple
clicks on the ImageJ status bar to drive garbage collection to completion.

         Great, now you see the problem, and how to fix it manually,
but how can you do it programatically like in your macros to avoid
the problem when automating work with large numbers of
images?  Several revs. ago, Wayne added the ability to call Java
routines directly from the macro language using the call
command.  So, you can manually force the JVM to do garbage collection
from a macro using the following:

call("java.lang.System.gc");

         As I mentioned, sometimes it would take multiple clicks to
drive garbage collection to completion, so you could simply embed the
above call command in a small loop, and have it execute say 3-5 times.

         Now, you could include this logic in your macros, and have
it run every certain amount of time, for example, after every 30
images, or after one directory of images.  This solved my random, out
of memory errors, and I suspect it will fix the "memory leak" you
have noticed if I am not totally off base.

Best wishes,

Mark Besonen
UMass Geosciences

[Michael Cammer]

Is there a way to call the curve fitting routine from within a macro
where it is automatically passed the xData, yData and fittype?

Thanks.

____________________________________________________________________________
Michael Cammer   Analytical Imaging Facility   Albert Einstein Coll. of Med.
URL:  http://www.aecom.yu.edu/aif/ 

[Girault France]

Hello,
 
I've tried what you suggested (use call("java.lang.System.gc"); in a
loop), but the memory don't decrease and the problem stays  (stupped
after 510 images processed) and instead I get a message in the log
window saying that the memory runs out, I get 2 exceptions messages that
I can see on the window manager, but these messages don't show up and
the entire system bugs... Do you have another suggestion ?
 
image506;  14MB of 1000MB (1%)
image507;  14MB of 1000MB (1%)
image508;  14MB of 1000MB (1%)
image509;  14MB of 1000MB (1%)
image510;  14MB of 1000MB (1%)
 
 
france
 
At 07:40 AM 10/27/2006, you wrote:

>Dear all,

>

>I'm having problem using a plugin to segment images :

>

>It causes leak of memory when I use it in a loop on hundreds of images.

>

>I need to detect the problem in the source code, but as I m completely

>ignorant is Java, I don't know how to process and which program do I

>need to use that allows detecting memory leaks in the code of the

>plugin...

>

>Does any body have experience in this already? any advice for a very

>very very very easy (for non programmer) software that could be used

>??...

>

>Many thanks

>

>france

 

Hello France,

First of all, I am assuming your "memory leak" is manifested by an out
of memory error. If not, simply skip this message.

Anyway, I don't think the problem you've noticed is with ImageJ or one
of the plugins, but with the underlying Java Virtual Machine. I have run
into similar out of memory conditions when using a macro to automate
processing of a whole series of directories full of images. If I just
process a single directory at a time with the macro, I never have a
problem. But when I automate a recursive trawl through a whole series of
directories to use the same macro, I'd run into out-out-memory errors at
random times.

The JVM does "garbage collection" by itself, and recovers unused memory
on the fly as necessary, so theoretically you should never run into an
out of memory condition. However, by automating a routine, I think you
can sometimes overpower the JVM's garbage collection schedule, and
actually force an out of memory condition. Fortunately, ImageJ has a
some built-in tools to help figure this out, and probably resolve your
problem.

First, I'd suggest you pop up the memory monitor window
(Plugins-->Utilities-->Monitor Memory...), and keep an eye on it as you
run your macro. When I first start it up on my system, and without doing
anything, it fluctuates around 1-1.5 MB of usage. As you start to
open/process images, you'll see the memory rise. When you shut down
those images, theoretically, the memory usage should fall down to the
same original base level of 1-1.5 MB usage, but in fact, it does not.
After opening up one small image, running a couple of filters over it,
and then shutting down the image, my new base level of memory usage is
in the 7-8 MB range. And if you were to do something similar hundreds of
times rapidly in succession via a macro, the problem gets compounded,
and can eventually lead to the out of memory errors you have seen.

Fortunately, as I mentioned above, ImageJ has a couple of mechanisms to
fix this issue. In particular, you can have ImageJ manually force the
JVM to do garbage collection. With you new 7-8 MB (or whatever) base
memory usage, now click the status bar on the main ImageJ window--you
should see the memory usage drop back down to its original footprint. In
my experience, it sometimes takes multiple clicks on the ImageJ status
bar to drive garbage collection to completion.

Great, now you see the problem, and how to fix it manually, but how can
you do it programatically like in your macros to avoid the problem when
automating work with large numbers of images? Several revs. ago, Wayne
added the ability to call Java routines directly from the macro language
using the call command. So, you can manually force the JVM to do garbage
collection from a macro using the following:

call("java.lang.System.gc");

As I mentioned, sometimes it would take multiple clicks to drive garbage
collection to completion, so you could simply embed the above call
command in a small loop, and have it execute say 3-5 times.

Now, you could include this logic in your macros, and have it run every
certain amount of time, for example, after every 30 images, or after one
directory of images. This solved my random, out of memory errors, and I
suspect it will fix the "memory leak" you have noticed if I am not
totally off base.

Best wishes,

Mark Besonen

UMass Geosciences

 

 

 

[Mark R. Besonen]

Hello again France,

         Sorry to hear that you are still having a
problem.  According to the memory monitoring stats for your images
506-510, you obviously have plenty of memory available, so the out of
memory error seems confusing.  An important point is that you've run
into two exception messages and a full system hang--this should not
happen with a simple out of memory condition, so it points to
something else being wrong.

         It would probably be helpful if you provided some
information about your setup (OS, JVM version, IJ version, etc.), and
the exception messages so other more knowledgeable than me can help.

         Another thought is to post your macro code, and let others
run it to see if we can reproduce the problem.  You don't have to
send the actual images you are processing--you could just send specs
like resolution, format, color depth, etc. of the images, and we
could have ImageJ build just a bunch of generic images with the
newImage command.  If you are willing to do this, I'll have some time
later today when I could run it to see if I experience the same problem.

Best wishes,

Mark Besonen
UMass Geosciences




At 03:37 AM 10/30/2006, Girault France wrote:

>Hello,
>
>I've tried what you suggested (use call("java.lang.System.gc"); in a
>loop), but the memory don't decrease and the problem stays  (stupped
>after 510 images processed) and instead I get a message in the log
>window saying that the memory runs out, I get 2 exceptions messages that
>I can see on the window manager, but these messages don't show up and
>the entire system bugs... Do you have another suggestion ?
>
>image506;  14MB of 1000MB (1%)
>image507;  14MB of 1000MB (1%)
>image508;  14MB of 1000MB (1%)
>image509;  14MB of 1000MB (1%)
>image510;  14MB of 1000MB (1%)
>
>france
>
>At 07:40 AM 10/27/2006, you wrote:
> >Dear all,
> >
> >I'm having problem using a plugin to segment images :
> >
> >It causes leak of memory when I use it in a loop on hundreds of images.
> >
> >I need to detect the problem in the source code, but as I m completely
> >ignorant is Java, I don't know how to process and which program do I
> >need to use that allows detecting memory leaks in the code of the
> >plugin...
> >
> >Does any body have experience in this already? any advice for a very
> >very very very easy (for non programmer) software that could be used
> >??...
> >
> >Many thanks
> >
> >france
>
>
>
>Hello France,
>
>First of all, I am assuming your "memory leak" is manifested by an out
>of memory error. If not, simply skip this message.
>
>Anyway, I don't think the problem you've noticed is with ImageJ or one
>of the plugins, but with the underlying Java Virtual Machine. I have run
>into similar out of memory conditions when using a macro to automate
>processing of a whole series of directories full of images. If I just
>process a single directory at a time with the macro, I never have a
>problem. But when I automate a recursive trawl through a whole series of
>directories to use the same macro, I'd run into out-out-memory errors at
>random times.
>
>The JVM does "garbage collection" by itself, and recovers unused memory
>on the fly as necessary, so theoretically you should never run into an
>out of memory condition. However, by automating a routine, I think you
>can sometimes overpower the JVM's garbage collection schedule, and
>actually force an out of memory condition. Fortunately, ImageJ has a
>some built-in tools to help figure this out, and probably resolve your
>problem.
>
>First, I'd suggest you pop up the memory monitor window
>(Plugins-->Utilities-->Monitor Memory...), and keep an eye on it as you
>run your macro. When I first start it up on my system, and without doing
>anything, it fluctuates around 1-1.5 MB of usage. As you start to
>open/process images, you'll see the memory rise. When you shut down
>those images, theoretically, the memory usage should fall down to the
>same original base level of 1-1.5 MB usage, but in fact, it does not.
>After opening up one small image, running a couple of filters over it,
>and then shutting down the image, my new base level of memory usage is
>in the 7-8 MB range. And if you were to do something similar hundreds of
>times rapidly in succession via a macro, the problem gets compounded,
>and can eventually lead to the out of memory errors you have seen.
>
>Fortunately, as I mentioned above, ImageJ has a couple of mechanisms to
>fix this issue. In particular, you can have ImageJ manually force the
>JVM to do garbage collection. With you new 7-8 MB (or whatever) base
>memory usage, now click the status bar on the main ImageJ window--you
>should see the memory usage drop back down to its original footprint. In
>my experience, it sometimes takes multiple clicks on the ImageJ status
>bar to drive garbage collection to completion.
>
>Great, now you see the problem, and how to fix it manually, but how can
>you do it programatically like in your macros to avoid the problem when
>automating work with large numbers of images? Several revs. ago, Wayne
>added the ability to call Java routines directly from the macro language
>using the call command. So, you can manually force the JVM to do garbage
>collection from a macro using the following:
>
>call("java.lang.System.gc");
>
>As I mentioned, sometimes it would take multiple clicks to drive garbage
>collection to completion, so you could simply embed the above call
>command in a small loop, and have it execute say 3-5 times.
>
>Now, you could include this logic in your macros, and have it run every
>certain amount of time, for example, after every 30 images, or after one
>directory of images. This solved my random, out of memory errors, and I
>suspect it will fix the "memory leak" you have noticed if I am not
>totally off base.
>
>Best wishes,
>
>Mark Besonen
>
>UMass Geosciences
>
>
>
>
>
>

[Bulent Celasun]

Dear All,

Think of an image of an orange's cut surface (the one resembling a wheel).
Then, imagine thresholding the juicy lobes that are separated from each
other.
You now have objects distributed radially around a central point.
(Petals of a flower seen from above can also serve the same purpose).

Question:
Is there a measure of "radialness" of the thresholded objects similar to the
one called as "circularity factor" or "roundness".
To be useful, such a measure should be tolerant to changing the number and
the distance of the objects to the center
(i.e. adding "radial" objects of differing sizes and shapes here and there
should not alter it much),
but responsive to changes in their axes.

In other words; can I define and measure "radialness" as a variable in
ImageJ?
If yes, how?

Thanks,

Bülent Celasun, MD


Department of Pathology
Baskent Uni. School of Medicine,
Ankara, Turkey

[audrey karperien-2]

I've been measuring the convex hull, then finding the coefficient of variation for the radii on similar forms.  Would that help?
Audrey


----- Original Message ----
From: Bulent Celasun <[hidden email]>
To: [hidden email]
Sent: Wednesday, November 1, 2006 11:54:58 AM
Subject: Test of radialness


Dear All,

Think of an image of an orange's cut surface (the one resembling a wheel).
Then, imagine thresholding the juicy lobes that are separated from each
other.
You now have objects distributed radially around a central point.
(Petals of a flower seen from above can also serve the same purpose).

Question:
Is there a measure of "radialness" of the thresholded objects similar to the
one called as "circularity factor" or "roundness".
To be useful, such a measure should be tolerant to changing the number and
the distance of the objects to the center
(i.e. adding "radial" objects of differing sizes and shapes here and there
should not alter it much),
but responsive to changes in their axes.

In other words; can I define and measure "radialness" as a variable in
ImageJ?
If yes, how?

Thanks,

Bülent Celasun, MD


Department of Pathology
Baskent Uni. School of Medicine,
Ankara, Turkey

[Kenneth Sloan-2]

On Nov 1, 2006, at 12:54 PM, Bulent Celasun wrote:

> Dear All,
>
> Think of an image of an orange's cut surface (the one resembling a  
> wheel).
> Then, imagine thresholding the juicy lobes that are separated from  
> each
> other.
> You now have objects distributed radially around a central point.
> (Petals of a flower seen from above can also serve the same purpose).
>
> Question:
> Is there a measure of "radialness" of the thresholded objects  
> similar to the
> one called as "circularity factor" or "roundness".
> To be useful, such a measure should be tolerant to changing the  
> number and
> the distance of the objects to the center
> (i.e. adding "radial" objects of differing sizes and shapes here  
> and there
> should not alter it much),
> but responsive to changes in their axes.
>
> In other words; can I define and measure "radialness" as a variable in
> ImageJ?
> If yes, how?

I don't know if there's a standard measure - but here's what springs  
to my mind:

Given that you have isolated the individual lobes, I would express  
the shape and extent of the lobes in polar coordinates....AND  
THEN...consider the polar coordinates to be Cartesian
coordinates.  In other words, warp the lobes so that a perfect  
triangle would warp to a rectangle.  Be careful about points very  
close to the center....there's a singularity lurking there.  Having  
done this, use any standard measure of "rectangularity" (perhaps,  
perimeter of the transformed lobe divided by the perimeter of the  
bounding rectangle.

Another way to view this idea: compute the bounding "sector" for each  
lobe, where the sector is bounded by the circular arc and the two  
radii that just contain the lobe.  Call the Area of the lobe "L", and  
call the Area of the sector "S".  Then perhaps L/S measures what you  
mean
by "radialness"?  Or, just perimeter of the lobe / perimeter of the  
sector?

Finally, perhaps it's orientation you are interested in?  Compute the  
eigenvectors/eigenvalues for the covariance matrix associated with  
the lobe.  A "very radial" lobe should have the eigenvector  
associated with the largest eigenvalue pointing in the same direction  
as the
sector.




--
Kenneth Sloan                                          
[hidden email]
Computer and Information Sciences                        [phone]
University of Alabama at Birmingham              FAX [phone]
Birmingham, AL 35294-1170                http://www.cis.uab.edu/sloan/

[Batishko, Charles R]

I had a problem like this a number of years ago. I don't have a standard metric, but the way I did it was as follows.

I segmented the objects and measured the angle of the major axis for each. I defined straight lines at those angles and found the set of intersections between each pair of lines. I measured the tightness (density) of the cluster of intersection points. The tighter the cluster, the more radial. The more diffuse the cluster, the less radial. One would need to have some kind of distrubuted population of samples so as to define a distribution, and a threshold that might then discriminate between radial and non-radial. But that would need to be associated with the physical reality. For example, my work involved whether to not the segmented features emanated from a point source or an extended source. Based on reality, one would need to define a finite threshold differentiating between "point" and "extended" sources - probably based on measurement uncertainty that leads to there being no such thing as a point source where physical measurements are concerned.

Probably not the answer you wanted, but perhaps will trigger thoughts from others, or help you think of it in terms that lead to a solution.

Chuck

> -----Original Message-----
> From: ImageJ Interest Group [mailto:[hidden email]] On
> Behalf Of Bulent Celasun
> Sent: Wednesday, November 01, 2006 10:55 AM
> To: [hidden email]
> Subject: Test of radialness
>
> Dear All,
>
> Think of an image of an orange's cut surface (the one
> resembling a wheel).
> Then, imagine thresholding the juicy lobes that are separated
> from each other.
> You now have objects distributed radially around a central point.
> (Petals of a flower seen from above can also serve the same purpose).
>
> Question:
> Is there a measure of "radialness" of the thresholded objects
> similar to the one called as "circularity factor" or "roundness".
> To be useful, such a measure should be tolerant to changing
> the number and the distance of the objects to the center
> (i.e. adding "radial" objects of differing sizes and shapes
> here and there should not alter it much), but responsive to
> changes in their axes.
>
> In other words; can I define and measure "radialness" as a
> variable in ImageJ?
> If yes, how?
>
> Thanks,
>
> Bülent Celasun, MD
>
>
> Department of Pathology
> Baskent Uni. School of Medicine,
> Ankara, Turkey
>