Login  Register

Re: imagej and Java 1.8

Posted by Mark Hiner on Dec 08, 2015; 3:07pm
URL: http://imagej.273.s1.nabble.com/imagej-and-Java-1-8-tp5015143p5015175.html

Hi Aryeh!

>I hope that this is useful.

Yep this is exactly what I needed and your thoroughness is greatly
appreciated. :)

I started with heapdump-144955328636 and it was immediately clear that
something on the python side is keeping hard references to your images. It
looks like they are being stored in a ConcurrentHashMap which is not
cleaned up when your script ends.

So it looks like a bug in the Python script you're using. I put up some
screen captures[1] of the reference path that's keeping your objects alive,
along with a brief interpretation. The variable names are distinct so I'm
hoping it will be obvious where in your script they are being mapped.
Ideally I would suggest caching weak references on the python side if
that's an option, or possibly clearing the cache at the end of the script.

If it's not clear how to fix the script and you'd like more eyes on it,
share it here - happy to have a look! :)  Also happy to give more info on
the heap dump analysis if anything's unclear there.

If anyone else is interested in looking at the heap dumps I personally
recommend the Eclipse Memory Analyzer plugin[2].

Best,
Mark

[1] https://gist.github.com/hinerm/727453aab0837246367e
[2] https://eclipse.org/mat/downloads.php

On Tue, Dec 8, 2015 at 2:25 AM, Aryeh Weiss <[hidden email]> wrote:

> Hi Mark
>
> On 07/12/2015 11:04 PM, Mark Hiner wrote:
>
> Hi Aryeh,
>
> On Mon, Dec 7, 2015 at 11:46 AM, Aryeh Weiss <[hidden email]> wrote:
>
>> Thanks for this information. I decided to run IJ, do a heap dump before
>> running a script that opens a few 1.3Gpixel images, then run it after, then
>> close the images and get a heap dump when the images have been closed but
>> the memory is not freed (which is reproducible).
>
>
> This sounds like the perfect set of information!
>
> > In order to send it to you, I will probably need to send it to an FTP
> server (like I have done with bio-formats. DO you have a server to which I
> can send the dumps?
>
> Actually you should be able to use the Upload Sample Image[1] command to
> send the zip. If that doesn't work for you let me know and I can send FTP
> credentails.
>
>
> I uploaded a set of heap dumps using  Upload Sample Image as you
> suggested.
> They are:
> heapdump-1449551945772 -- made immediately after launching Fiji. Small, as
> expected.
> heapdump-1449552198991 -- made immediately after running my python
> script. Three large image are open.
> heapdump-1449552667004 -- made after closing those three large images and
> double clicking in the Fiji window
>                                                 a few times to run the
> garbage collector
> heapdump-1449553044705 -- made after running the script a second time,
> during which Fiji ran out of memory.
>                                                Two large images are open.
> heapdump-1449553286368 -- made after closing those two windows and garbage
> collecting.
>                                                As expected, memory was not
> freed.
>
> The large images are a bit under 1Gpixel, 8-bit images.
>
> I hope that this is useful.
> Thanks for your efforts.
> --aryeh
>
> --
> Aryeh Weiss
> Faculty of Engineering
> Bar Ilan University
> Ramat Gan 52900 Israel
>
> Ph:  972-3-5317638
> FAX: 972-3-7384051
>
>
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html