ImageJ

Memory Problems in Fiji using macro setBatchMode(true)

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

LIM Soon Yew John (IMB)

Memory Problems in Fiji using macro setBatchMode(true)

Dear All,

I noticed that when I run macro setBatchMode(true) in Fiji, all the images created during setBatchMode(true) are trapped in memory and cannot be clear away [run("Close All") does not work]. This become a problem for 32bit Fiji as the program quickly run out of memory.

I do not encounter this problem in ImageJ. Does anyone else face this problem?

Best Regards,
John
________________________________________
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

LIM Soon Yew John (IMB)

Re: Memory Problems in Fiji using macro setBatchMode(true)

Dear List,

Sorry for the extra e-mail as the problem with the memory I wrote below had been solved by updating to ImageJ 1.48b daily build and using run("Collect Garbage") in the macro to free up memory.

Best Regards,
John
________________________________________
From: ImageJ Interest Group [[hidden email]] On Behalf Of LIM Soon Yew John (IMB) [[hidden email]]
Sent: Tuesday, August 06, 2013 2:52 PM
To: [hidden email]
Subject: Memory Problems in Fiji using macro setBatchMode(true)

Dear All,

I noticed that when I run macro setBatchMode(true) in Fiji, all the images created during setBatchMode(true) are trapped in memory and cannot be clear away [run("Close All") does not work]. This become a problem for 32bit Fiji as the program quickly run out of memory.

I do not encounter this problem in ImageJ. Does anyone else face this problem?

Best Regards,
John
________________________________________
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

> Dear List,
>
> I experience the same problem when using the doCommand in a macro:
Thanks to Johannes Schindelin, this bug is fixed in the ImageJ 1.48b daily build. Use the following macro to demonstrate both the problem and the fix. Run it using ImageJ 1.48b1 and the memory used to open the 100 images is reclaimed when you run the garbage collector by clicking in the "Memory" window. With ImageJ 1.48a and earlier, the memory is not reclaimed when you click in the "Memory" window.
-wayne
doCommand("Monitor Memory...");
setBatchMode(true);
for(i=0;i<100;i++) {
run("Open Image");
wait(20);
}
run("Close All");
exit;
macro "Open Image" {
newImage("Untitled", "8-bit ramp", 1024, 1024, 1);
}
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

dscho

Re: Memory Problems in Fiji using macro setBatchMode(true)

Hi John,

On Tue, 6 Aug 2013, LIM Soon Yew John (IMB) wrote:

> Sorry for the extra e-mail as the problem with the memory I wrote below
> had been solved by updating to ImageJ 1.48b daily build and using
> run("Collect Garbage") in the macro to free up memory.

Yep, there is even a Fiji bug for it:

http://fiji.sc/bugzilla/show_bug.cgi?id=641

Hopefully 1.48b gets released soon, as this critical bug affects not only
Fiji but barebones ImageJ, too.

Ciao,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

JimmiB

Re: Memory Problems in Fiji using macro setBatchMode(true)

In reply to this post by LIM Soon Yew John (IMB)

Hi all,

I believe I am having a similar issue to this. I am running a macro designed to open an image crop out 2 ROI's resize one of them and save the new images. The process works fine in both ImageJ and Fiji except that it rapidly builds up memory usage.

I have tried incorporating the suggested
run("Collect Garbage"); in Fiji

All this has done is slow down the processing, but the memory issue is still there.

I am running the latest nightly build of ImageJ.

Any help would be greatly appreciated.

The macro in question is below.

Cheers,
James

macro "seperate channels[s]"{

dir1 = getDirectory("Choose Source Directory ");
dest = getDirectory("Choose Destination Directory ");
list = getFileList(dir1);
greendir = dest + "greenTEMP"+File.separator;
reddir = dest + "redTEMP"+File.separator;
//print(greendir);
File.makeDirectory(greendir);
File.makeDirectory(reddir);
x=getString("set x offset", 3);
y=getString("set y offset", 1);
setBatchMode(true);
for (i=1; i<list.length; i++) {
print (dir1+list[i]);
if (endsWith(list[i],".tif")){
showProgress(i+1, list.length);
open (dir1+list[i]); //open image i
rename ("original" + i);
run("Rotate 90 Degrees Right");
//define Green Channel
w=getWidth()-16;
h=getHeight()/2-10;
makeRectangle(x, y, w, h);
run("Duplicate...", "title=Green");
saveAs("Tiff", greendir+"green_"+i+".tif");
close("Green");

//define red channel
selectWindow("original"+i);
w_red=w/1.975*2;
h_red=h/1.985*2;
makeRectangle(3, 10+h_red, w_red, h_red);
run("Duplicate...", "title=Red_Original");
run("Scale...", "x=1.975 y=1.985 interpolation=Bicubic create title=Red_Engorged");
close("Red_Original");
run("Scale...", "x=0.5 y=0.5 interpolation=Bilinear average create title=Red");
close("Red_Engorged");
selectWindow("Red");
run("Rotate... ", "angle=0.25 grid=2 interpolation=Bilinear stack");
saveAs("Tiff", reddir +"red_"+i+".tif");
close("Red");
close("original" + i);
run("Collect Garbage");
}
}

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

JimmiB

Re: Memory Problems in Fiji using macro setBatchMode(true)

I should also mention that the problem is also apparent when not using
batchmode.

*
*
*Dr James Burchfield*
The Garvan Institute of Medical Research
384 Victoria Street
Darlinghurst, NSW, 2010
Australia

Email: [hidden email]
Phone:+61 2 92958229
Web: *www.garvan.org.au*

On Mon, Aug 12, 2013 at 5:07 PM, James Burchfield <
[hidden email]> wrote:

> Hi all,
>
> I believe I am having a similar issue to this. I am running a macro
> designed to open an image crop out 2 ROI's resize one of them and save the
> new images. The process works fine in both ImageJ and Fiji except that it
> rapidly builds up memory usage.
>
> I have tried incorporating the suggested
> run("Collect Garbage"); in Fiji
>
> All this has done is slow down the processing, but the memory issue is
> still there.
>
> I am running the latest nightly build of ImageJ.
>
> Any help would be greatly appreciated.
>
> The macro in question is below.
>
> Cheers,
> James
>
> macro "seperate channels[s]"{
>
> dir1 = getDirectory("Choose Source Directory ");
> dest = getDirectory("Choose Destination Directory ");
> list = getFileList(dir1);
> greendir = dest + "greenTEMP"+File.separator;
> reddir = dest + "redTEMP"+File.separator;
> //print(greendir);
> File.makeDirectory(greendir);
> File.makeDirectory(reddir);
> x=getString("set x offset", 3);
> y=getString("set y offset", 1);
> setBatchMode(true);
> for (i=1; i<list.length; i++) {
> print (dir1+list[i]);
> if (endsWith(list[i],".tif")){
> showProgress(i+1, list.length);
> open (dir1+list[i]); //open image i
> rename ("original" + i);
> run("Rotate 90 Degrees Right");
> //define Green Channel
> w=getWidth()-16;
> h=getHeight()/2-10;
> makeRectangle(x, y, w, h);
> run("Duplicate...", "title=Green");
> saveAs("Tiff", greendir+"green_"+i+".tif");
> close("Green");
>
>
> //define red channel
> selectWindow("original"+i);
> w_red=w/1.975*2;
> h_red=h/1.985*2;
> makeRectangle(3, 10+h_red, w_red, h_red);
> run("Duplicate...", "title=Red_Original");
> run("Scale...", "x=1.975 y=1.985 interpolation=Bicubic create
> title=Red_Engorged");
> close("Red_Original");
> run("Scale...", "x=0.5 y=0.5 interpolation=Bilinear average create
> title=Red");
> close("Red_Engorged");
> selectWindow("Red");
> run("Rotate... ", "angle=0.25 grid=2 interpolation=Bilinear
> stack");
> saveAs("Tiff", reddir +"red_"+i+".tif");
> close("Red");
> close("original" + i);
> run("Collect Garbage");
> }
> }
>
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

Cameron Nowell-2

Re: Memory Problems in Fiji using macro setBatchMode(true)

Hi James,

Not sure if it matters but whenever i use the garbage collection option i use

call("java.lang.System.gc");

i ahve also found that putting a small delay, say 500 - 1000ms (using wait(500);) after the call helps it make sure it works ok

Cheers

Cam

Cameron J. Nowell
Research Facilities Manager

Monash Institute of Pharmaceutical Sciences
Monash University
399 Royal Parade
(Mail address: 381 Royal Parade)
Parkville, VIC, 3052
Australia

Email: [hidden email]
Mobile: +61 422882700
Office: +61 9903 9587

LinkedIn: Profile
Research Gate: Profile

________________________________________
From: ImageJ Interest Group [[hidden email]] on behalf of James Burchfield [[hidden email]]
Sent: Monday, August 12, 2013 5:17 PM
To: [hidden email]
Subject: Re: Memory Problems in Fiji using macro setBatchMode(true)

I should also mention that the problem is also apparent when not using
batchmode.

*
*
*Dr James Burchfield*
The Garvan Institute of Medical Research
384 Victoria Street
Darlinghurst, NSW, 2010
Australia

Email: [hidden email]
Phone:+61 2 92958229
Web: *www.garvan.org.au*

On Mon, Aug 12, 2013 at 5:07 PM, James Burchfield <
[hidden email]> wrote:

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

dscho

Re: Memory Problems in Fiji using macro setBatchMode(true)

In reply to this post by JimmiB

Hi James,

On Mon, 12 Aug 2013, James Burchfield wrote:

> I am running a macro designed to open an image crop out 2 ROI's resize
> one of them and save the new images. The process works fine in both
> ImageJ and Fiji except that it rapidly builds up memory usage.

Have you tried running it with the daily build? (Help>Update ImageJ...
-- not Help>Update Fiji)

Ciao,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

Rasband, Wayne (NIH/NIMH) [E]

Re: Memory Problems in Fiji using macro setBatchMode(true)

In reply to this post by JimmiB

On Aug 12, 2013, at 3:07 AM, James Burchfield wrote:

> Hi all,
>
> I believe I am having a similar issue to this. I am running a macro designed to open an image crop out 2 ROI's resize one of them and save the new images. The process works fine in both ImageJ and Fiji except that it rapidly builds up memory usage.

The macro is opening five images per loop iteration that are not closed. You can see this by changing the line

print (dir1+list[i]);

to

print (i, nImages, dir1+list[i]);

After making this change and running the macro I get output that looks likes this:

1 0 /Users/wayne/stack/Untitled0001.tif
2 5 /Users/wayne/stack/Untitled0002.tif
3 10 /Users/wayne/stack/Untitled0003.tif
4 15 /Users/wayne/stack/Untitled0004.tif
5 20 /Users/wayne/stack/Untitled0005.tif

I was able to fix the problem by changing

close("Red");
close("original" + i);

to

run("Close All");

Here is the modified macro:

dir1 = getDirectory("Choose Source Directory ");
dest = getDirectory("Choose Destination Directory ");
list = getFileList(dir1);
greendir = dest + "greenTEMP"+File.separator;
reddir = dest + "redTEMP"+File.separator;
//print(greendir);
File.makeDirectory(greendir);
File.makeDirectory(reddir);
x=getString("set x offset", 3);
y=getString("set y offset", 1);
setBatchMode(true);
for (i=1; i<list.length; i++) {
print (i, nImages, dir1+list[i]);
if (endsWith(list[i],".tif")) {
showProgress(i+1, list.length);
open (dir1+list[i]); //open image i
rename ("original" + i);
run("Rotate 90 Degrees Right");
//define Green Channel
w=getWidth()-16;
h=getHeight()/2-10;
makeRectangle(x, y, w, h);
run("Duplicate...", "title=Green");
saveAs("Tiff", greendir+"green_"+i+".tif");
close("Green");
//define red channel
selectWindow("original"+i);
w_red=w/1.975*2;
h_red=h/1.985*2;
makeRectangle(3, 10+h_red, w_red, h_red);
run("Duplicate...", "title=Red_Original");
run("Scale...", "x=1.975 y=1.985 interpolation=Bicubic create title=Red_Engorged");
close("Red_Original");
run("Scale...", "x=0.5 y=0.5 interpolation=Bilinear average create title=Red");
close("Red_Engorged");
selectWindow("Red");
run("Rotate... ", "angle=0.25 grid=2 interpolation=Bilinear stack");
saveAs("Tiff", reddir +"red_"+i+".tif");
run("Close All");
}
}

-wayne

> I have tried incorporating the suggested
> run("Collect Garbage"); in Fiji
>
> All this has done is slow down the processing, but the memory issue is still there.
>
> I am running the latest nightly build of ImageJ.
>
> Any help would be greatly appreciated.
>
> The macro in question is below.
>
> Cheers,
> James
>
> macro "seperate channels[s]"{
>
> dir1 = getDirectory("Choose Source Directory ");
> dest = getDirectory("Choose Destination Directory ");
> list = getFileList(dir1);
> greendir = dest + "greenTEMP"+File.separator;
> reddir = dest + "redTEMP"+File.separator;
> //print(greendir);
> File.makeDirectory(greendir);
> File.makeDirectory(reddir);
> x=getString("set x offset", 3);
> y=getString("set y offset", 1);
> setBatchMode(true);
> for (i=1; i<list.length; i++) {
> print (dir1+list[i]);
> if (endsWith(list[i],".tif")){
> showProgress(i+1, list.length);
> open (dir1+list[i]); //open image i
> rename ("original" + i);
> run("Rotate 90 Degrees Right");
> //define Green Channel
> w=getWidth()-16;
> h=getHeight()/2-10;
> makeRectangle(x, y, w, h);
> run("Duplicate...", "title=Green");
> saveAs("Tiff", greendir+"green_"+i+".tif");
> close("Green");
>
>
> //define red channel
> selectWindow("original"+i);
> w_red=w/1.975*2;
> h_red=h/1.985*2;
> makeRectangle(3, 10+h_red, w_red, h_red);
> run("Duplicate...", "title=Red_Original");
> run("Scale...", "x=1.975 y=1.985 interpolation=Bicubic create title=Red_Engorged");
> close("Red_Original");
> run("Scale...", "x=0.5 y=0.5 interpolation=Bilinear average create title=Red");
> close("Red_Engorged");
> selectWindow("Red");
> run("Rotate... ", "angle=0.25 grid=2 interpolation=Bilinear stack");
> saveAs("Tiff", reddir +"red_"+i+".tif");
> close("Red");
> close("original" + i);
> run("Collect Garbage");
> }
> }
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

JimmiB

Re: Memory Problems in Fiji using macro setBatchMode(true)

Thanks Wayne,
Much appreciated.
It seems to be working well.
Why are extra images being opened?
When run with batchmode off the images never appear.

There seems to be something strange going on here.
I just opened a 5000 frame stack

Then
ran
run("Scale...", "x=1.975 y=1.985 z=1.0 width=726 height=492 depth=5000
interpolation=Bicubic average process create title=Red_Engorged");
run("Scale...", "x=0.5 y=0.5 z=1.0 width=363 height=246 depth=5000
interpolation=Bilinear average process create title=Red_Engorged-1")

I then closed all the open images.

ImageJ shows <1% memory usage, however in the task manager javaw.exe is
using 8 486 740 K
Running the garbage collector does nothing.....
run("Close All"); does nothing

Does that imply some kind of memory leak?

Thanks again for your help.

Cheers

James

*
*
*Dr James Burchfield*
The Garvan Institute of Medical Research
384 Victoria Street
Darlinghurst, NSW, 2010
Australia

Email: [hidden email]
Phone:+61 2 92958229
Web: *www.garvan.org.au*

On Tue, Aug 13, 2013 at 2:31 AM, Rasband, Wayne (NIH/NIMH) [E] <
[hidden email]> wrote:

> On Aug 12, 2013, at 3:07 AM, James Burchfield wrote:
>
> > Hi all,
> >
> > I believe I am having a similar issue to this. I am running a macro
> designed to open an image crop out 2 ROI's resize one of them and save the
> new images. The process works fine in both ImageJ and Fiji except that it
> rapidly builds up memory usage.
>
> The macro is opening five images per loop iteration that are not closed.
> You can see this by changing the line
>
> print (dir1+list[i]);
>
> to
>
> print (i, nImages, dir1+list[i]);
>
> After making this change and running the macro I get output that looks
> likes this:
>
> 1 0 /Users/wayne/stack/Untitled0001.tif
> 2 5 /Users/wayne/stack/Untitled0002.tif
> 3 10 /Users/wayne/stack/Untitled0003.tif
> 4 15 /Users/wayne/stack/Untitled0004.tif
> 5 20 /Users/wayne/stack/Untitled0005.tif
>
> I was able to fix the problem by changing
>
> close("Red");
> close("original" + i);
>
> to
>
> run("Close All");
>
> Here is the modified macro:
>
> dir1 = getDirectory("Choose Source Directory ");
> dest = getDirectory("Choose Destination Directory ");
> list = getFileList(dir1);
> greendir = dest + "greenTEMP"+File.separator;
> reddir = dest + "redTEMP"+File.separator;
> //print(greendir);
> File.makeDirectory(greendir);
> File.makeDirectory(reddir);
> x=getString("set x offset", 3);
> y=getString("set y offset", 1);
> setBatchMode(true);
> for (i=1; i<list.length; i++) {
> print (i, nImages, dir1+list[i]);
> if (endsWith(list[i],".tif")) {
> showProgress(i+1, list.length);
> open (dir1+list[i]); //open image i
> rename ("original" + i);
> run("Rotate 90 Degrees Right");
> //define Green Channel
> w=getWidth()-16;
> h=getHeight()/2-10;
> makeRectangle(x, y, w, h);
> run("Duplicate...", "title=Green");
> saveAs("Tiff", greendir+"green_"+i+".tif");
> close("Green");
> //define red channel
> selectWindow("original"+i);
> w_red=w/1.975*2;
> h_red=h/1.985*2;
> makeRectangle(3, 10+h_red, w_red, h_red);
> run("Duplicate...", "title=Red_Original");
> run("Scale...", "x=1.975 y=1.985 interpolation=Bicubic create
> title=Red_Engorged");
> close("Red_Original");
> run("Scale...", "x=0.5 y=0.5 interpolation=Bilinear average
> create title=Red");
> close("Red_Engorged");
> selectWindow("Red");
> run("Rotate... ", "angle=0.25 grid=2 interpolation=Bilinear
> stack");
> saveAs("Tiff", reddir +"red_"+i+".tif");
> run("Close All");
> }
> }
>
>
> -wayne
>
> > I have tried incorporating the suggested
> > run("Collect Garbage"); in Fiji
> >
> > All this has done is slow down the processing, but the memory issue is
> still there.
> >
> > I am running the latest nightly build of ImageJ.
> >
> > Any help would be greatly appreciated.
> >
> > The macro in question is below.
> >
> > Cheers,
> > James
> >
> > macro "seperate channels[s]"{
> >
> > dir1 = getDirectory("Choose Source Directory ");
> > dest = getDirectory("Choose Destination Directory ");
> > list = getFileList(dir1);
> > greendir = dest + "greenTEMP"+File.separator;
> > reddir = dest + "redTEMP"+File.separator;
> > //print(greendir);
> > File.makeDirectory(greendir);
> > File.makeDirectory(reddir);
> > x=getString("set x offset", 3);
> > y=getString("set y offset", 1);
> > setBatchMode(true);
> > for (i=1; i<list.length; i++) {
> > print (dir1+list[i]);
> > if (endsWith(list[i],".tif")){
> > showProgress(i+1, list.length);
> > open (dir1+list[i]); //open image i
> > rename ("original" + i);
> > run("Rotate 90 Degrees Right");
> > //define Green Channel
> > w=getWidth()-16;
> > h=getHeight()/2-10;
> > makeRectangle(x, y, w, h);
> > run("Duplicate...", "title=Green");
> > saveAs("Tiff", greendir+"green_"+i+".tif");
> > close("Green");
> >
> >
> > //define red channel
> > selectWindow("original"+i);
> > w_red=w/1.975*2;
> > h_red=h/1.985*2;
> > makeRectangle(3, 10+h_red, w_red, h_red);
> > run("Duplicate...", "title=Red_Original");
> > run("Scale...", "x=1.975 y=1.985 interpolation=Bicubic create
> title=Red_Engorged");
> > close("Red_Original");
> > run("Scale...", "x=0.5 y=0.5 interpolation=Bilinear average create
> title=Red");
> > close("Red_Engorged");
> > selectWindow("Red");
> > run("Rotate... ", "angle=0.25 grid=2 interpolation=Bilinear
> stack");
> > saveAs("Tiff", reddir +"red_"+i+".tif");
> > close("Red");
> > close("original" + i);
> > run("Collect Garbage");
> > }
> > }
> >
> > --
> > ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html