Virtual Memory in Headless Mode

[Greg]

Hello,

I am using Fiji on Mac OS 10.10. We have to handle huge image stacks, which can be easily around 20Gb. When I running Fiji in GUI mode, the adjusted memory value (Memory & Threads option) extends far beyond the physical amount of RAM. I know that it gets slower, but at least we can process these big files at once then.

Now when I try to run in headless mode via

ImageJ-macosx --headless open.py

where open.py basically just opens one of the files I get the following output on the console:

>>>>>>>>>>>>>>>>>>>>>>>>>>>
<Out of memory>
<All available memory (16033MB) has been>
<used. To make more available, use the>
<Edit>Options>Memory & Threads command.>
>>>>>>>>>>>>>>>>>>>>>>>>>>>

I checked the Info.plist file in the Fiji.app folder, and there the adjusted value is correct (I set it to 40000), and I can open these big files in GUI mode.
So is it not possible at all to use virtual memory in headless mode or how can I pass that option to Fiji/ImageJ in headless mode ?

Thanks and greets,
Greg

[Greg]

Hi again,

I just want to say that by using virtual stacks, I can open the image stacks slice by slice which is very convenient !

Details can be found here in case anybody stumbles upon this thread with the same problem:

http://fiji.sc/Jython_Scripting#Open_the_slices_of_a_very_large_multi-image_stack_file_one_by_one.2C_and_save_each_as_a_new_image_file

So although the original problem concerning virtual memory in headless mode is not adressed, I think this solution is even better.

Greg

[ctrueden]

Hi Greg,

> So is it not possible at all to use virtual memory in headless mode or
> how can I pass that option to Fiji/ImageJ in headless mode ?

I cannot replicate this problem on my system.

I created a script mem.bsh with contents:
   print(Runtime.getRuntime().maxMemory() / 1024 / 1024);

Then I set the maximum memory to 20GB (my machine has 8GB RAM).

And then when I run:
   ImageJ-macosx --headless mem.bsh

I see 19933 as output (the small percentage difference is normal).

In any case, glad to hear virtual stacks are working for your use case. I
think that is likely to perform better for you (in both space and time)
than using OS disk/memory swap.

Regards,
Curtis

On Mon, Feb 23, 2015 at 9:57 AM, Greg <[hidden email]> wrote:

> Hello,
>
> I am using Fiji on Mac OS 10.10. We have to handle huge image stacks, which
> can be easily around 20Gb. When I running Fiji in GUI mode, the adjusted
> memory value (Memory & Threads option) extends far beyond the physical
> amount of RAM. I know that it gets slower, but at least we can process
> these
> big files at once then.
>
> Now when I try to run in headless mode via
>
> ImageJ-macosx --headless open.py
>
> where open.py basically just opens one of the files I get the following
> output on the console:
>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> <Out of memory>
> <All available memory (16033MB) has been>
> <used. To make more available, use the>
> <Edit>Options>Memory & Threads command.>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>
> I checked the Info.plist file in the Fiji.app folder, and there the
> adjusted
> value is correct (I set it to 40000), and I can open these big files in GUI
> mode.
> So is it not possible at all to use virtual memory in headless mode or how
> can I pass that option to Fiji/ImageJ in headless mode ?
>
> Thanks and greets,
> Greg
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Virtual-Memory-in-Headless-Mode-tp5011730.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ctrueden]

Hi Greg,

> I cannot replicate this problem on my system.

Looks like I spoke too soon: the bug is actually fixed with the new ImageJ
launcher (discussed on another recent thread). I had the new launcher
installed locally, and could not reproduce using it, but was able to
reproduce with the old version of the launcher.

So good news: this bug is fixed, and will be pushed to Fiji soon!

Cheers,
Curtis

On Mon, Feb 23, 2015 at 2:14 PM, Curtis Rueden <[hidden email]> wrote:

> Hi Greg,
>
> > So is it not possible at all to use virtual memory in headless mode or
> > how can I pass that option to Fiji/ImageJ in headless mode ?
>
> I cannot replicate this problem on my system.
>
> I created a script mem.bsh with contents:
>    print(Runtime.getRuntime().maxMemory() / 1024 / 1024);
>
> Then I set the maximum memory to 20GB (my machine has 8GB RAM).
>
> And then when I run:
>    ImageJ-macosx --headless mem.bsh
>
> I see 19933 as output (the small percentage difference is normal).
>
> In any case, glad to hear virtual stacks are working for your use case. I
> think that is likely to perform better for you (in both space and time)
> than using OS disk/memory swap.
>
> Regards,
> Curtis
>
> On Mon, Feb 23, 2015 at 9:57 AM, Greg <[hidden email]> wrote:
>
>> Hello,
>>
>> I am using Fiji on Mac OS 10.10. We have to handle huge image stacks,
>> which
>> can be easily around 20Gb. When I running Fiji in GUI mode, the adjusted
>> memory value (Memory & Threads option) extends far beyond the physical
>> amount of RAM. I know that it gets slower, but at least we can process
>> these
>> big files at once then.
>>
>> Now when I try to run in headless mode via
>>
>> ImageJ-macosx --headless open.py
>>
>> where open.py basically just opens one of the files I get the following
>> output on the console:
>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <Out of memory>
>> <All available memory (16033MB) has been>
>> <used. To make more available, use the>
>> <Edit>Options>Memory & Threads command.>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>
>> I checked the Info.plist file in the Fiji.app folder, and there the
>> adjusted
>> value is correct (I set it to 40000), and I can open these big files in
>> GUI
>> mode.
>> So is it not possible at all to use virtual memory in headless mode or how
>> can I pass that option to Fiji/ImageJ in headless mode ?
>>
>> Thanks and greets,
>> Greg
>>
>>
>>
>> --
>> View this message in context:
>> http://imagej.1557.x6.nabble.com/Virtual-Memory-in-Headless-Mode-tp5011730.html
>> Sent from the ImageJ mailing list archive at Nabble.com.
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html