Slowdown when calling a plugin from within a loop

[Eric Olson-3]

I've been using ImageJ for a few years now, but just wrote my first
plugin. I just came across a strange issue that I don't really
understand and was hoping someone might have some insight. I guess this
is more of a Java question than an ImageJ question.

My plugin just converts a batch of files from one format to another and
is based around the DM3_Reader plugin
<http://rsb.info.nih.gov/ij/plugins/DM3_Reader.html> by Gregory
Jefferis.  I found that my plugin got progressively slower and slower
when it was looping over a directory of images. The first image took
about 1-2 seconds to convert, by the time it got to the seventh image,
it was taking 30-40 seconds.

Here is a simplification of what I was originally trying to do:

public class myConverter1{
   // DM3_Reader extends ImagePlus
   private DM3_Reader myReader = new DM3_Reader();

   public void run(String args) {
      for(int i=0; i<filename.length; i++){
         myReader.run (filename(i));

         fileSaver = new FileSaver(myReader);
         fileSaver.saveAsTiff(outputFile);
      }
   }
}


This change fixes the problem, every image takes only 1-2 seconds to
convert, but I don't really understand why:

public class myConverter2{
   public void run(String args) {
      for(int i=0; i<filename.length; i++){
         // DM3_Reader extends ImagePlus
         DM3_Reader myReader = new DM3_Reader();
         myReader (filename)

         fileSaver = new FileSaver(myReader);
         fileSaver.saveAsTiff(outputFile);
      }
   }
}

The only significant change is that myReader now goes out of scope at
the end of the for loop. I don't seeI don't see why this would have such
a large effect on the processing time. Am I just being daft or can
someone share some insight?

--
Eric Olson
Postdoctoral Research Associate, University of Illinois at Urbana-Champaign
[hidden email], 217-244-2117 (voice), 217-244-2278 (fax)

[Jeffrey B. Woodward]

I would have to guess that the DM3_Reader class has a "memory leak" [of
sorts] (I am not familiar with that class, but have you verified that
there isn't a method to call to indicate that you are done with a file
prior to "run"ning the next file). I suspect that if you monitor the
memory usage of your system that you will find that you have exhausted a
lot of memory and the system is having to page/swap memory to and from disk.

This, of course, is all just a guess. Best luck,

-Woody


Eric Olson wrote:

> I've been using ImageJ for a few years now, but just wrote my first
> plugin. I just came across a strange issue that I don't really
> understand and was hoping someone might have some insight. I guess
> this is more of a Java question than an ImageJ question.
>
> My plugin just converts a batch of files from one format to another
> and is based around the DM3_Reader plugin
> <http://rsb.info.nih.gov/ij/plugins/DM3_Reader.html> by Gregory
> Jefferis.  I found that my plugin got progressively slower and slower
> when it was looping over a directory of images. The first image took
> about 1-2 seconds to convert, by the time it got to the seventh image,
> it was taking 30-40 seconds.
> Here is a simplification of what I was originally trying to do:
>
> public class myConverter1{
>   // DM3_Reader extends ImagePlus
>   private DM3_Reader myReader = new DM3_Reader();
>
>   public void run(String args) {
>      for(int i=0; i<filename.length; i++){
>         myReader.run (filename(i));
>
>         fileSaver = new FileSaver(myReader);
>         fileSaver.saveAsTiff(outputFile);
>      }
>   }
> }
>
>
> This change fixes the problem, every image takes only 1-2 seconds to
> convert, but I don't really understand why:
>
> public class myConverter2{
>   public void run(String args) {
>      for(int i=0; i<filename.length; i++){
>         // DM3_Reader extends ImagePlus
>         DM3_Reader myReader = new DM3_Reader();
>         myReader (filename)
>
>         fileSaver = new FileSaver(myReader);
>         fileSaver.saveAsTiff(outputFile);
>      }
>   }
> }
>
> The only significant change is that myReader now goes out of scope at
> the end of the for loop. I don't seeI don't see why this would have
> such a large effect on the processing time. Am I just being daft or
> can someone share some insight?
>
> --
> Eric Olson
> Postdoctoral Research Associate, University of Illinois at
> Urbana-Champaign
> [hidden email], 217-244-2117 (voice), 217-244-2278 (fax)
>
>

[Eric Olson-3]

Jeffrey B. Woodward said the following on 3/7/2008 6:58 PM:
> I would have to guess that the DM3_Reader class has a "memory leak"
> [of sorts] (I am not familiar with that class, but have you verified
> that there isn't a method to call to indicate that you are done with a
> file prior to "run"ning the next file). I suspect that if you monitor
> the memory usage of your system that you will find that you have
> exhausted a lot of memory and the system is having to page/swap memory
> to and from disk.
That was my first thought, but the Windows Task Manager showed no
increase in the amount of memory used (and I was loading >100 MB of
images total) and there was not a lot of disk activity, which I always
see when it's swapping to/from disk. There isn't any method in the
plugin like the one you described.

I thought maybe since I was calling the plugin repeatedly, the garbage
collector just wasn't getting around to releasing some used objects, so
I tried adding a System.gc() call at the end of each loop of my plugin,
but that didn't help.

-- Eric

[Eric Olson-3]

Eric Olson said the following on 3/11/2008 5:14 PM:
> I thought maybe since I was calling the plugin repeatedly, the garbage
> collector just wasn't getting around to releasing some used objects,
> so I tried adding a System.gc() call at the end of each loop of my
> plugin, but that didn't help.
Found it. The plugin I was using to do the actual loading had a
temporary variable that wasn't reset each time the method was called. I
had just been assuming my plugin was at fault.