Memory leak when using Macro to process 1000+ images
Locked
 
|
Hi,
I am using ImageJ v1.49e and I wrote a simple macro that will read in 1000+ images from a directory and perform some processing. Once its all over I save the summaries to a file. I do this all in batchmode so the windows don't open and close quickly. So the issue is that the memory usage keeps rising until the program crashes. I really do not know why this is happening and I am opening and closing each image. Can anyone locate in the code below where things are going wrong? Thanks. ----- Dialog.create("Select Options"); Dialog.addSlider("Subtract Background Rolling:", 0.0, 90.0, 30.0); Dialog.addSlider("Min Particle Size (px):", 5, 90, 15); Dialog.addSlider("Min Particle Circularity:", 0.0, 1.0, 0.40); Dialog.addCheckbox("Save Summaries to file", true); Dialog.show(); inputDir = getDirectory("Choose the Input Directory"); inputDir = replace(inputDir, "\\\\", "\\\\\\\\"); if(inputDir != "") { inputList = getFileList(inputDir); rollingValue = Dialog.getNumber(); minParticleSize = Dialog.getNumber(); minParticleCirc = Dialog.getNumber(); saveSummaries = Dialog.getCheckbox(); setBatchMode(true); for (i=0; i < inputList.length; i++) { showProgress(i+1, inputList.length); fileName = inputList[i]; imageAnalysis(inputDir, fileName, rollingValue, minParticleSize, minParticleCirc); } if(saveSummaries) { selectWindow("Summary"); saveAs("Results", inputDir+"summaries.txt"); } call("java.lang.System.gc"); setBatchMode(false); if(isOpen("Summary")) { selectWindow("Summary"); run("Close" ); } } else { exit("Must select an input and output directory"); } function imageAnalysis(inputDir, fileName, rollingValue, circSize, minCircularity) { open(inputDir+fileName); run("Enhance Local Contrast (CLAHE)", "blocksize=127 histogram=256 maximum=3 mask=*None* fast_(less_accurate)"); run("8-bit"); run("Subtract Background...", "rolling="+rollingValue); setAutoThreshold("Triangle"); setOption("BlackBackground", false); run("Convert to Mask"); run("Set Measurements...", "area mean min centroid shape feret's area_fraction redirect=None decimal=3"); run("Analyze Particles...", "size="+circSize+"-Infinity circularity="+minCircularity+"-1.00 exclude clear summarize"); close(); } -- George [hidden email] -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Memory leak when using Macro to process 1000+ images
|
|
On Aug 19, 2014, at 7:14 AM, George H wrote:
> Hi, > > I am using ImageJ v1.49e and I wrote a simple macro that will read in 1000+ > images from a directory and perform some processing. Once its all over I > save the summaries to a file. I do this all in batchmode so the windows > don't open and close quickly. > > So the issue is that the memory usage keeps rising until the program > crashes. I really do not know why this is happening and I am opening and > closing each image. It looks like you are using Fiji so you should verify that "Use SCIFIO when opening files" is disabled in Edit>Options>ImageJ2. SCIFIO is known to cause memory leaks. The SCIFIO option is disabled by default in Fiji/ImageJ2 2.0.0-rc14 and later. -wayne > Can anyone locate in the code below where things are going wrong? Thanks. > > ----- > Dialog.create("Select Options"); > Dialog.addSlider("Subtract Background Rolling:", 0.0, 90.0, 30.0); > Dialog.addSlider("Min Particle Size (px):", 5, 90, 15); > Dialog.addSlider("Min Particle Circularity:", 0.0, 1.0, 0.40); > Dialog.addCheckbox("Save Summaries to file", true); > Dialog.show(); > > inputDir = getDirectory("Choose the Input Directory"); > inputDir = replace(inputDir, "\\\\", "\\\\\\\\"); > > if(inputDir != "") { > inputList = getFileList(inputDir); > rollingValue = Dialog.getNumber(); > minParticleSize = Dialog.getNumber(); > minParticleCirc = Dialog.getNumber(); > saveSummaries = Dialog.getCheckbox(); > setBatchMode(true); > for (i=0; i < inputList.length; i++) { > showProgress(i+1, inputList.length); > fileName = inputList[i]; > imageAnalysis(inputDir, fileName, rollingValue, minParticleSize, > minParticleCirc); > } > if(saveSummaries) { > selectWindow("Summary"); > saveAs("Results", inputDir+"summaries.txt"); > } > > call("java.lang.System.gc"); > setBatchMode(false); > > if(isOpen("Summary")) { > selectWindow("Summary"); > run("Close" ); > } > } > else { > exit("Must select an input and output directory"); > } > > > function imageAnalysis(inputDir, fileName, rollingValue, circSize, > minCircularity) { > open(inputDir+fileName); > run("Enhance Local Contrast (CLAHE)", "blocksize=127 histogram=256 > maximum=3 mask=*None* fast_(less_accurate)"); > run("8-bit"); > run("Subtract Background...", "rolling="+rollingValue); > setAutoThreshold("Triangle"); > setOption("BlackBackground", false); > run("Convert to Mask"); > run("Set Measurements...", "area mean min centroid shape feret's > area_fraction redirect=None decimal=3"); > run("Analyze Particles...", "size="+circSize+"-Infinity > circularity="+minCircularity+"-1.00 exclude clear summarize"); > close(); > } > > > -- > George > [hidden email] > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Memory leak when using Macro to process 1000+ images
|
|
Hi all,
>SCIFIO is known to cause memory leaks. I agree with Wayne's assessment that you should try running with "Use SCIFIO when opening files" disabled. Furthermore, in recent versions of Fiji all known SCIFIO-related memory leaks have been addressed. >Can anyone locate in the code below where things are going wrong? The macro looks correct. I tested with and without SCIFIO enabled and did not see a significant memory leak. If you still see crashes with an up-to-date Fiji, you should try: * Running with Java 7 <http://fiji.sc/Frequently_Asked_Questions#How_do_I_launch_ImageJ_with_a_different_version_of_Java.3F> - we've seen this fix hard crashes that were occurring during long image processing runs. * Launching in debug mode <http://imagej.net/Debugging#Launching_ImageJ_in_debug_mode> from the console and recording the exact error on crashing. * Trying solutions in the debugging hard JVM crashes <http://imagej.net/Debugging#Debugging_hard_JVM_crashes> guide. Let us know how things work out. - Mark On Tue, Aug 19, 2014 at 11:58 AM, Rasband, Wayne (NIH/NIMH) [E] < [hidden email]> wrote: > On Aug 19, 2014, at 7:14 AM, George H wrote: > > > Hi, > > > > I am using ImageJ v1.49e and I wrote a simple macro that will read in > 1000+ > > images from a directory and perform some processing. Once its all over I > > save the summaries to a file. I do this all in batchmode so the windows > > don't open and close quickly. > > > > So the issue is that the memory usage keeps rising until the program > > crashes. I really do not know why this is happening and I am opening and > > closing each image. > > It looks like you are using Fiji so you should verify that "Use SCIFIO > when opening files" is disabled in Edit>Options>ImageJ2. SCIFIO is known to > cause memory leaks. The SCIFIO option is disabled by default in > Fiji/ImageJ2 2.0.0-rc14 and later. > > -wayne > > > Can anyone locate in the code below where things are going wrong? > Thanks. > > > > ----- > > Dialog.create("Select Options"); > > Dialog.addSlider("Subtract Background Rolling:", 0.0, 90.0, 30.0); > > Dialog.addSlider("Min Particle Size (px):", 5, 90, 15); > > Dialog.addSlider("Min Particle Circularity:", 0.0, 1.0, 0.40); > > Dialog.addCheckbox("Save Summaries to file", true); > > Dialog.show(); > > > > inputDir = getDirectory("Choose the Input Directory"); > > inputDir = replace(inputDir, "\\\\", "\\\\\\\\"); > > > > if(inputDir != "") { > > inputList = getFileList(inputDir); > > rollingValue = Dialog.getNumber(); > > minParticleSize = Dialog.getNumber(); > > minParticleCirc = Dialog.getNumber(); > > saveSummaries = Dialog.getCheckbox(); > > setBatchMode(true); > > for (i=0; i < inputList.length; i++) { > > showProgress(i+1, inputList.length); > > fileName = inputList[i]; > > imageAnalysis(inputDir, fileName, rollingValue, minParticleSize, > > minParticleCirc); > > } > > if(saveSummaries) { > > selectWindow("Summary"); > > saveAs("Results", inputDir+"summaries.txt"); > > } > > > > call("java.lang.System.gc"); > > setBatchMode(false); > > > > if(isOpen("Summary")) { > > selectWindow("Summary"); > > run("Close" ); > > } > > } > > else { > > exit("Must select an input and output directory"); > > } > > > > > > function imageAnalysis(inputDir, fileName, rollingValue, circSize, > > minCircularity) { > > open(inputDir+fileName); > > run("Enhance Local Contrast (CLAHE)", "blocksize=127 histogram=256 > > maximum=3 mask=*None* fast_(less_accurate)"); > > run("8-bit"); > > run("Subtract Background...", "rolling="+rollingValue); > > setAutoThreshold("Triangle"); > > setOption("BlackBackground", false); > > run("Convert to Mask"); > > run("Set Measurements...", "area mean min centroid shape feret's > > area_fraction redirect=None decimal=3"); > > run("Analyze Particles...", "size="+circSize+"-Infinity > > circularity="+minCircularity+"-1.00 exclude clear summarize"); > > close(); > > } > > > > > > -- > > George > > [hidden email] > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Yes, you're all right.
I didn't mention that I was using Fiji, I actually thought it was the same as ImageJ so apologies for that. With the 1.49f update it showed me that warning on restart about getting buggy behavior if SCIFIO was enabled so I disabled it. and everything went fine. I think I might have accidentally enabled it from some previous update. Thanks. -- George [hidden email] On Tue, Aug 19, 2014 at 11:19 PM, Mark Hiner <[hidden email]> wrote: > Hi all, > > >SCIFIO is known to cause memory leaks. > > I agree with Wayne's assessment that you should try running with "Use > SCIFIO when opening files" disabled. Furthermore, in recent versions of > Fiji all known SCIFIO-related memory leaks have been addressed. > > >Can anyone locate in the code below where things are going wrong? > > The macro looks correct. I tested with and without SCIFIO enabled and did > not see a significant memory leak. If you still see crashes with an > up-to-date Fiji, you should try: > > * Running with Java 7 > < > http://fiji.sc/Frequently_Asked_Questions#How_do_I_launch_ImageJ_with_a_different_version_of_Java.3F > > > - we've seen this fix hard crashes that were occurring during long image > processing runs. > * Launching in debug mode > <http://imagej.net/Debugging#Launching_ImageJ_in_debug_mode> from the > console and recording the exact error on crashing. > * Trying solutions in the debugging hard JVM crashes > <http://imagej.net/Debugging#Debugging_hard_JVM_crashes> guide. > > Let us know how things work out. > > - Mark > > On Tue, Aug 19, 2014 at 11:58 AM, Rasband, Wayne (NIH/NIMH) [E] < > [hidden email]> wrote: > > > On Aug 19, 2014, at 7:14 AM, George H wrote: > > > > > Hi, > > > > > > I am using ImageJ v1.49e and I wrote a simple macro that will read in > > 1000+ > > > images from a directory and perform some processing. Once its all over > I > > > save the summaries to a file. I do this all in batchmode so the windows > > > don't open and close quickly. > > > > > > So the issue is that the memory usage keeps rising until the program > > > crashes. I really do not know why this is happening and I am opening > and > > > closing each image. > > > > It looks like you are using Fiji so you should verify that "Use SCIFIO > > when opening files" is disabled in Edit>Options>ImageJ2. SCIFIO is known > to > > cause memory leaks. The SCIFIO option is disabled by default in > > Fiji/ImageJ2 2.0.0-rc14 and later. > > > > -wayne > > > > > Can anyone locate in the code below where things are going wrong? > > Thanks. > > > > > > ----- > > > Dialog.create("Select Options"); > > > Dialog.addSlider("Subtract Background Rolling:", 0.0, 90.0, 30.0); > > > Dialog.addSlider("Min Particle Size (px):", 5, 90, 15); > > > Dialog.addSlider("Min Particle Circularity:", 0.0, 1.0, 0.40); > > > Dialog.addCheckbox("Save Summaries to file", true); > > > Dialog.show(); > > > > > > inputDir = getDirectory("Choose the Input Directory"); > > > inputDir = replace(inputDir, "\\\\", "\\\\\\\\"); > > > > > > if(inputDir != "") { > > > inputList = getFileList(inputDir); > > > rollingValue = Dialog.getNumber(); > > > minParticleSize = Dialog.getNumber(); > > > minParticleCirc = Dialog.getNumber(); > > > saveSummaries = Dialog.getCheckbox(); > > > setBatchMode(true); > > > for (i=0; i < inputList.length; i++) { > > > showProgress(i+1, inputList.length); > > > fileName = inputList[i]; > > > imageAnalysis(inputDir, fileName, rollingValue, minParticleSize, > > > minParticleCirc); > > > } > > > if(saveSummaries) { > > > selectWindow("Summary"); > > > saveAs("Results", inputDir+"summaries.txt"); > > > } > > > > > > call("java.lang.System.gc"); > > > setBatchMode(false); > > > > > > if(isOpen("Summary")) { > > > selectWindow("Summary"); > > > run("Close" ); > > > } > > > } > > > else { > > > exit("Must select an input and output directory"); > > > } > > > > > > > > > function imageAnalysis(inputDir, fileName, rollingValue, circSize, > > > minCircularity) { > > > open(inputDir+fileName); > > > run("Enhance Local Contrast (CLAHE)", "blocksize=127 histogram=256 > > > maximum=3 mask=*None* fast_(less_accurate)"); > > > run("8-bit"); > > > run("Subtract Background...", "rolling="+rollingValue); > > > setAutoThreshold("Triangle"); > > > setOption("BlackBackground", false); > > > run("Convert to Mask"); > > > run("Set Measurements...", "area mean min centroid shape feret's > > > area_fraction redirect=None decimal=3"); > > > run("Analyze Particles...", "size="+circSize+"-Infinity > > > circularity="+minCircularity+"-1.00 exclude clear summarize"); > > > close(); > > > } > > > > > > > > > -- > > > George > > > [hidden email] > > > > > > -- > > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi George,
On Wed, 20 Aug 2014, George H wrote: > I didn't mention that I was using Fiji, I actually thought it was the > same as ImageJ so apologies for that. No worries. In general, Fiji Is Just ImageJ (hence the acronym). Even more concretely, due to the needs of the projects in which I am involved, Fiji Is Just ImageJ2 (which includes all of ImageJ, therefore you were not at all off the mark). Ciao, Johannes -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
«
Return to ImageJ
|
1 view|%1 views
| Free forum by Nabble | Edit this page |