Re: huge memory leaks in Vista 64 with LOCI Bioformats

Posted by ctrueden on Dec 08, 2008; 8:17pm
URL: http://imagej.273.s1.nabble.com/huge-memory-leaks-in-Vista-64-with-LOCI-Bioformats-tp3694295p3694296.html

Hi John,

I'm CCing the list as this stuff may be of interest to others.

I should have been more clear about the "malloc" function: it allocates 32
MB of memory, but only once. It won't stack up. There is a block of data the
program has, that is either null, or 32 MB in size. If you click malloc a
second time, it will re-allocate the 32MB, and the Java garbage collector
will eat the old 32 MB piece of data since it has been overwritten.
Nonetheless, you will still see a temporary spike in memory usage before the
garbage collector kicks in -- and if Java is set to allow only a 64 MB
maximum heap, which it is by default on many systems, it will actually throw
an OutOfMemoryException because it allocates the new block before releasing
the old block.

Clicking "free" does two things: 1) nulls out the data block; and 2) calls
the garbage collector. Sometimes calling the garbage collector once is not
enough to fully free everything in Java, but you can get maximal results by
mashing the free button a few times.

So, in summary, the reason you are seeing the behavior you describe is that
when you mash "malloc" really fast, it allocates a bunch of 32 MB blocks
before the garbage collector gets around to releasing the old ones. But if
you mash "free" a few times it should drop (nearly) all the way back down.
And when Java starts getting close to its max, it forces a garbage
collection, since it needs the memory at that point -- which is why you get
the "flip over" you noticed.

The question is, what does the Windows Task Manager say about all this? Does
the Mem Usage go back down after clicking "free" a few times? Or does it
stay at its peak?

Lastly, here is an interesting article about the differences between Vista's
approach to memory, versus XP and other older Windows versions:

http://www.codinghorror.com/blog/archives/000688.html

I am not sure how accurate or up to date it is, but it does suggest that
perhaps Vista not releasing memory is not as bad as it may seem -- perhaps
Vista *would* release the memory once you have other applications actively
demanding it. Hard to say without doing some more testing.

-Curtis

On Mon, Dec 8, 2008 at 2:08 PM, John Alexander <[hidden email]>wrote:

> Thanks,
>
> Can you help me interpret some of this?
>
> Running your program blindly ... (so trusting, I know) ...
>
> At start up, I get:
> Total: 62848 KB
> Max: 932096 KB
> Free: 56721 KB
> Used: 6470 KB
>
> I then click malloc 10 times ...
> Total: 358336 KB
> Max: 932096 KB
> Free: 21730 KB
> Used: 336950 KB
>
> I click free once:
> Total: 358336 KB
> Max: 932096 KB
> Free: 355780 KB
> Used: 2560 KB
>
> I then click malloc 10 times, and get the same as above.
>
> Then 10 MORE times and I get
> Total: 686656 KB
> Max: 932096 KB
> Free: 27030 KB
> Used: 659924 KB
>
> Followed by free:
> Total: 686656 KB
> Max: 932096 KB
> Free: 684424 KB
> Used: 2513 KB
>
> interesting ... when I keep clicking malloc ... it never
> reaches the Max ... when Total gets close to Max ... it flips
> over.
>
> For example, closing then starting the program all over ...
> and clicking malloc 21 times gets me:
> Total: 718208 KB
> Max: 932096 KB
> Free: 25734 KB
> Used: 692167 KB
>
> and on the 22nd click of malloc, I get:
> Total: 106112 KB
> Max: 932096 KB
> Free: 38170 KB
> Used: 68280 KB
>
> My machine specs are vista 64-bit running Java 1.6 64-bit with
> 4 GB system RAM.
>
> John
>
>
> John Alexander, Ph.D.
> Post-doctoral Fellow
> William Green Lab
> University of Chicago
> 947 E. 58th St.
> Abott Hall 402
> Chicago, IL 60637
> off: [phone]
> fax: [phone]
> [hidden email]
>