Memory usage in WEKA segmentation

[ben_king]

While doing the training in WEKA segmentation, I keep running out of memory. Sometimes a dialog box comes up with "Out of Memory" exception. Other times, an out of memory message appears in the log window only.

I've tried increasing the maximum memory in Edit>Options>Memory & Threads from the default 505MB to 1, 2, 4GB and more (running Win 8.1 64bit with 16GB RAM), but ImageJ never uses more than 505MB - and always runs out of memory while running WEKA.

What else can I try to get ImageJ to use more memory?

[ben_king]

After more experimentation while monitoring the memory usage, nothing in FIJI will use more than 505MB RAM.

I've set higher memory limits in FIJI, and the Windows Java control panel (which I pointed FIJI to). Help?

[ctrueden]

Hi Ben,

> I've tried increasing the maximum memory in Edit>Options>Memory &
> Threads from the default 505MB to 1, 2, 4GB and more (running Win 8.1
> 64bit with 16GB RAM), but ImageJ never uses more than 505MB

I have seen other users describe this same problem a couple of times. But I
do not know why it happens. It would be great to troubleshoot and find the
answer, so that we can add it to the http://imagej.net/Troubleshooting page.

You can try launching ImageJ from the command line:
http://imagej.net/Troubleshooting#Launching_ImageJ_from_the_console

Try changing the memory allocation directly; e.g.:

C:\Users\me\Programs\Fiji.app\ImageJ-win64.exe -mx512m --
C:\Users\me\Programs\Fiji.app\ImageJ-win64.exe -mx1g --
C:\Users\me\Programs\Fiji.app\ImageJ-win64.exe -mx4g --

You can also use:
C:\Users\me\Programs\Fiji.app\ImageJ-win64.exe --dry-run --console

To get the full command line used by the ImageJ launcher. You can then
experiment with making changes to that command line as needed.

Lastly, you could give this Windows batch launcher a try:
* https://raw.githubusercontent.com/imagej/imagej/master/bin/ImageJ.bat

Let us know how what you find out! Again, I would love to have a solution
for those people who occasionally encounter this issue.

Regards,
Curtis

On Tue, Nov 17, 2015 at 11:08 AM, ben_king <[hidden email]> wrote:

> After more experimentation while monitoring the memory usage, nothing in
> FIJI
> will use more than 505MB RAM.
>
> I've set higher memory limits in FIJI, and the Windows Java control panel
> (which I pointed FIJI to). Help?
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Memory-usage-in-WEKA-segmentation-tp5014961p5014962.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ben_king]

Curtis,

Running from the console didn't fix it but it did something. Here's what I have tried:

1. Ran from console with -mx*anything* flag, ImageJ loads with 494MB available memory and gives out of memory exception when it uses all 494MB

2. Ran from console with -Xmx*anything* or no flags, ImageJ loads with 505MB and uses up to 505MB

3. Put -Xmx256m flag in ImageJ.cfg file then ran ImageJ (either from console with no flags or by clicking icon), ImageJ loads and says 256MB available under Help->About ImageJ, but still uses up to 505MB (200%)

4. Put -Xmx1g flag in ImageJ.cfg file then ran ImageJ (either from console with no flags or by clicking icon), ImageJ loads and says 1024MB available under Help->About ImageJ, but only uses up to 505MB (50%)

Tried 3 and 4 with -mx* instead of -Xmx* and the result was the same, except IJ used 495MB before crashing.

What's the difference between -mx and -Xmx?

[ctrueden]

Hi Ben,

Thanks for the report. All of that behavior is quite bizarre.

> What's the difference between -mx and -Xmx?

Until your post, I would have said "nothing," but it seems there are slight
differences. I am not certain, but I think "-mx" is for backwards
compatibility, whereas "-Xmx" is the currently recommended invocation.

Have you tested with different versions of Java?
*
http://imagej.net/FAQ#How_do_I_launch_ImageJ_with_a_different_version_of_Java.3F

64-bit? 32-bit? Java 8 instead of Java 6?

Maybe the problem does not occur when using Java 8 64-bit?

Regards,
Curtis

On Tue, Nov 17, 2015 at 4:25 PM, ben_king <[hidden email]> wrote:

> Curtis,
>
> Running from the console didn't fix it but it did something. Here's what I
> have tried:
>
> 1. Ran from console with -mx*anything* flag, ImageJ loads with 494MB
> available memory and gives out of memory exception when it uses all 494MB
>
> 2. Ran from console with -Xmx*anything* or no flags, ImageJ loads with
> 505MB
> and uses up to 505MB
>
> 3. Put -Xmx256m flag in ImageJ.cfg file then ran ImageJ (either from
> console
> with no flags or by clicking icon), ImageJ loads and says 256MB available
> under Help->About ImageJ, but still uses up to 505MB (200%)
>
> 4. Put -Xmx1g flag in ImageJ.cfg file then ran ImageJ (either from console
> with no flags or by clicking icon), ImageJ loads and says 1024MB available
> under Help->About ImageJ, but only uses up to 505MB (50%)
>
> Tried 3 and 4 with -mx* instead of -Xmx* and the result was the same,
> except
> IJ used 495MB before crashing.
>
> What's the difference between -mx and -Xmx?
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Memory-usage-in-WEKA-segmentation-tp5014961p5014965.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ben_king]

Problem solved - there was a system environment variable _JAVA_OPTIONS containing "-Xmx512m." I must have. I noticed it when I went to make an environment variable to try other javas.

I changed the  _JAVA_OPTIONS environment variable to -Xmx10g and then IJ loads with/uses upto 10g.

Curiously, I tried deleting the environment variable and setting -Xmx256m in ImageJ.cfg, but IJ still uses over 256MB so I don't know what is limiting IJ memory usage without the environment variable. Maybe the system needs to restart for it to take effect.

Thanks for all your help, Curtis!

[ctrueden]

Hi Ben,

Awesome, so glad you figured it out. And glad to have another known culprit
for anyone else who might report this issue in the future.

Regards,
Curtis

On Wed, Nov 18, 2015 at 3:16 PM, ben_king <[hidden email]> wrote:

> Problem solved - there was a system environment variable _JAVA_OPTIONS
> containing "-Xmx512m." I must have. I noticed it when I went to make an
> environment variable to try other javas.
>
> I changed the  _JAVA_OPTIONS environment variable to -Xmx10g and then IJ
> loads with/uses upto 10g.
>
> Curiously, I tried deleting the environment variable and setting -Xmx256m
> in
> ImageJ.cfg, but IJ still uses over 256MB so I don't know what is limiting
> IJ
> memory usage without the environment variable. Maybe the system needs to
> restart for it to take effect.
>
> Thanks for all your help, Curtis!
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Memory-usage-in-WEKA-segmentation-tp5014961p5014972.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html