batch process macro and memory

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

batch process macro and memory

Knecht, David
I am working on a macro to batch process a folder of large tif files.  I find when I run it that ImageJ runs out of memory after processing several stacks.  I know there is enough memory to process each stack individually, but monitoring memory, I can see it gradually accumulating memory as it proceeds.  I would guess that it is either not dumping memory as it finishes a stack or else it is running processes in parallel and so gradually becoming overwhelmed.  Is there a command I can insert into the macro to allow it to free up memory along the way? Other suggestions?  Thanks- Dave


macro "batch process folder scale and make movie"{
        requires("1.33s");
        dir = getDirectory("Choose a Directory ");
        dir2 = getDirectory("Choose Destination Directory ");
        setBatchMode(true);
  count = 0;
  countFiles(dir);
  n = 0;
  processFiles(dir, dir2);
  //print(count+" files processed");
}


  function countFiles(dir) {
        list = getFileList(dir);
        for (i=0; i<list.length; i++) {
                 if (endsWith(list[i], "/"))
                        countFiles(""+dir+list[i]);
        else
              count++;
      }
 }


  function processFiles(dir, dir2) {
        list = getFileList(dir);
        for (i=0; i<list.length; i++) {
                if (endsWith(list[i], "/"))
              processFiles(""+dir+list[i], dir2);
          else {
            showProgress(n++, count);
            path = dir+list[i];
            processFile(path, dir2);
          }
    }
 }

function processFile(path,dir2) {
        if (endsWith(path, ".tif")) {
          open(path);
          title1=getTitle();
                run("8-bit");
                run("Properties...", " unit=pixel pixel_width=.64 pixel_height=.64 voxel_depth=1.0000 frame=[5 min] origin=0,0");
                run("Scale...", "x=.7 y=.7 z=1   interpolation=Bilinear average process create");
                run("Label...", "format=00:00 starting=0 interval=4 x=800 y=20 font=18 text=hr:min ");
                run("QuickTime Movie...", "compression=MPEG-4 quality=Normal frame=20");
                save(dir2+title1);
                close();
        }
}


Visiting Professor David Knecht
Beatson Institute for Cancer Research
University of Glasgow
Switchback Road, Bearsden
Glasgow Scotland G61 1BD
UK
Reply | Threaded
Open this post in threaded view
|

Re: batch process macro and memory

Michael Schmid
Hi David,

your 'scale' command has a 'create' in it, thus it creates a new image (or
image stack, in your case).
You close only one of the image stacks (the new one), and leave the
original one open.

As a more general rule, if a BatchMode macro runs out of memory, quite
often one can find the problem by running it without BatchMode, and see
whether any images remain open.

Michael

________________________________________________________________________

On Sun, March 4, 2012 09:41, David Knecht wrote:

> I am working on a macro to batch process a folder of large tif files.  I
> find when I run it that ImageJ runs out of memory after processing several
> stacks.  I know there is enough memory to process each stack individually,
> but monitoring memory, I can see it gradually accumulating memory as it
> proceeds.  I would guess that it is either not dumping memory as it
> finishes a stack or else it is running processes in parallel and so
> gradually becoming overwhelmed.  Is there a command I can insert into the
> macro to allow it to free up memory along the way? Other suggestions?
> Thanks- Dave
>
>
> macro "batch process folder scale and make movie"{
> requires("1.33s");
> dir = getDirectory("Choose a Directory ");
> dir2 = getDirectory("Choose Destination Directory ");
> setBatchMode(true);
>   count = 0;
>   countFiles(dir);
>   n = 0;
>   processFiles(dir, dir2);
>   //print(count+" files processed");
> }
>
>
>   function countFiles(dir) {
> list = getFileList(dir);
> for (i=0; i<list.length; i++) {
> if (endsWith(list[i], "/"))
> countFiles(""+dir+list[i]);
>         else
>               count++;
>       }
>  }
>
>
>   function processFiles(dir, dir2) {
> list = getFileList(dir);
> for (i=0; i<list.length; i++) {
> if (endsWith(list[i], "/"))
>               processFiles(""+dir+list[i], dir2);
>           else {
>             showProgress(n++, count);
>             path = dir+list[i];
>             processFile(path, dir2);
>           }
>     }
>  }
>
> function processFile(path,dir2) {
> if (endsWith(path, ".tif")) {
>           open(path);
>           title1=getTitle();
> run("8-bit");
> run("Properties...", " unit=pixel pixel_width=.64 pixel_height=.64
> voxel_depth=1.0000 frame=[5 min] origin=0,0");
> run("Scale...", "x=.7 y=.7 z=1   interpolation=Bilinear average process
> create");
> run("Label...", "format=00:00 starting=0 interval=4 x=800 y=20 font=18
> text=hr:min ");
> run("QuickTime Movie...", "compression=MPEG-4 quality=Normal frame=20");
> save(dir2+title1);
> close();
> }
> }
>
>
> Visiting Professor David Knecht
> Beatson Institute for Cancer Research
> University of Glasgow
> Switchback Road, Bearsden
> Glasgow Scotland G61 1BD
> UK
>
Reply | Threaded
Open this post in threaded view
|

Re: batch process macro and memory

Alistair MacDougall
And if you still need to call for garbage collection in your macro the code
is:

call("java.lang.System.gc");

Alistair




-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
Michael Schmid
Sent: 04 March 2012 18:39
To: [hidden email]
Subject: Re: batch process macro and memory

Hi David,

your 'scale' command has a 'create' in it, thus it creates a new image (or
image stack, in your case).
You close only one of the image stacks (the new one), and leave the
original one open.

As a more general rule, if a BatchMode macro runs out of memory, quite
often one can find the problem by running it without BatchMode, and see
whether any images remain open.

Michael

________________________________________________________________________

On Sun, March 4, 2012 09:41, David Knecht wrote:

> I am working on a macro to batch process a folder of large tif files.  I
> find when I run it that ImageJ runs out of memory after processing several
> stacks.  I know there is enough memory to process each stack individually,
> but monitoring memory, I can see it gradually accumulating memory as it
> proceeds.  I would guess that it is either not dumping memory as it
> finishes a stack or else it is running processes in parallel and so
> gradually becoming overwhelmed.  Is there a command I can insert into the
> macro to allow it to free up memory along the way? Other suggestions?
> Thanks- Dave
>
>
> macro "batch process folder scale and make movie"{
> requires("1.33s");
> dir = getDirectory("Choose a Directory ");
> dir2 = getDirectory("Choose Destination Directory ");
> setBatchMode(true);
>   count = 0;
>   countFiles(dir);
>   n = 0;
>   processFiles(dir, dir2);
>   //print(count+" files processed");
> }
>
>
>   function countFiles(dir) {
> list = getFileList(dir);
> for (i=0; i<list.length; i++) {
> if (endsWith(list[i], "/"))
> countFiles(""+dir+list[i]);
>         else
>               count++;
>       }
>  }
>
>
>   function processFiles(dir, dir2) {
> list = getFileList(dir);
> for (i=0; i<list.length; i++) {
> if (endsWith(list[i], "/"))
>               processFiles(""+dir+list[i], dir2);
>           else {
>             showProgress(n++, count);
>             path = dir+list[i];
>             processFile(path, dir2);
>           }
>     }
>  }
>
> function processFile(path,dir2) {
> if (endsWith(path, ".tif")) {
>           open(path);
>           title1=getTitle();
> run("8-bit");
> run("Properties...", " unit=pixel pixel_width=.64
pixel_height=.64
> voxel_depth=1.0000 frame=[5 min] origin=0,0");
> run("Scale...", "x=.7 y=.7 z=1   interpolation=Bilinear
average process
> create");
> run("Label...", "format=00:00 starting=0 interval=4 x=800
y=20 font=18
> text=hr:min ");
> run("QuickTime Movie...", "compression=MPEG-4 quality=Normal
frame=20");

> save(dir2+title1);
> close();
> }
> }
>
>
> Visiting Professor David Knecht
> Beatson Institute for Cancer Research
> University of Glasgow
> Switchback Road, Bearsden
> Glasgow Scotland G61 1BD
> UK
>