Macro slowing down
Locked
 
|
I hate to ping the list twice in two days, but this one is a puzzle. I wrote a macro that measures the difference between each frame of a time series with frame 1, with the ultimate goal of measuring the periodicity of the average difference value (with Fourier analysis in Excel) to measure ciliary beat frequency. The macro uses nested for() loops to open each file in a folder, process it framewise and build a results table.
The macro runs pretty fast, but if I walk away from the computer for a while at some point it will slow down by almost ten fold, making it impractical to run through large data sets overnight. Each stack is about 200-500 kb and Fiji has 10 GB of RAM allocated, so I doubt that RAM is my problem. I quit all other programs, prevented sleep and Fiji remains foregrounded the whole time. I am using the latest Fiji update on OSX 10.10.5. It would be great to know what might cause the slowdown. At the same time, odds are my approach could be streamlined quite a bit. Right now the macro duplicates out frame 1 and frame n from the movie, makes an image of the difference and records the average pixel value of the difference image to a results table. Making the new images and closing them again seems to take time, even with batch mode on. Does anyone know how to measure the average difference between frame 1 and frame n of a stack without duplicating them out? I pasted the macro below. Thanks again, T Timothy Feinstein, Ph.D. Research Scientist University of Pittsburgh Department of Developmental Biology ------------------------------------------------------------ //Analyze ciliary beat frequency using cropped movies //Requires stable movies. Use the stackreg plugin on //uncropped originals if you can see sample or camera motion. setBatchMode(true); if(isOpen("diff_Quant")){ selectWindow("diff_Quant"); run("Close"); } if(isOpen("Results")) { selectWindow("Results"); run("Close"); } run("Clear Results"); run("Set Measurements...", "mean limit display redirect=None decimal=5"); //Find the movies to analyze files = getDirectory("Movies"); count = 0; list = getFileList(files); n = lengthOf(list); //Set up the results table for (i=0; i < n; i++) { result = 0; fileName = list[i]; setResult(fileName, result, 1); updateResults(); } IJ.renameResults("diff_Quant"); //Duplicate out the 1st and nth frame of each movie //and measure the average difference for (i=0; i <= n; i++) { open(files+list[count]); stack = getTitle(); time = 0; fileName = list[i]; result = 0; run("Gaussian Blur 3D...", "x=1 y=1 z=1"); n = nSlices(); for (slice=1; slice<=(300); slice++){ selectWindow(stack); setSlice(1); run("Duplicate...", "use"); first = getTitle(); selectWindow(stack); setSlice(slice + 1); run("Duplicate...", "use"); test = getTitle(); imageCalculator("Difference create", first, test); diff = getTitle(); run("Measure"); mean = getResult("Mean", 0); selectWindow("Results"); run("Clear Results"); run("Close"); selectWindow("diff_Quant"); IJ.renameResults("Results"); result = slice; setResult(fileName, result, mean); updateResults(); IJ.renameResults("diff_Quant"); selectWindow(first); run("Close"); selectWindow(test); run("Close"); selectWindow(diff); run("Close"); time = time + 0.005; } selectWindow(stack); run("Close"); count++; } -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Timothy - This sounds like a unix/linux question. See the documentation on priorities in linux processes.
As a process ages, its priority for processing may slip downward. Try using the ‘nice’ command with a negative setting (RTM before you start fiddling). - Jim > On Jan 12, 2017, at 9:19 AM, Feinstein, Timothy N <[hidden email]> wrote: > > I hate to ping the list twice in two days, but this one is a puzzle. I wrote a macro that measures the difference between each frame of a time series with frame 1, with the ultimate goal of measuring the periodicity of the average difference value (with Fourier analysis in Excel) to measure ciliary beat frequency. The macro uses nested for() loops to open each file in a folder, process it framewise and build a results table. > > The macro runs pretty fast, but if I walk away from the computer for a while at some point it will slow down by almost ten fold, making it impractical to run through large data sets overnight. Each stack is about 200-500 kb and Fiji has 10 GB of RAM allocated, so I doubt that RAM is my problem. I quit all other programs, prevented sleep and Fiji remains foregrounded the whole time. I am using the latest Fiji update on OSX 10.10.5. > > It would be great to know what might cause the slowdown. At the same time, odds are my approach could be streamlined quite a bit. Right now the macro duplicates out frame 1 and frame n from the movie, makes an image of the difference and records the average pixel value of the difference image to a results table. Making the new images and closing them again seems to take time, even with batch mode on. Does anyone know how to measure the average difference between frame 1 and frame n of a stack without duplicating them out? I pasted the macro below. > > Thanks again, > > > T > > Timothy Feinstein, Ph.D. > Research Scientist > University of Pittsburgh Department of Developmental Biology > > ------------------------------------------------------------ > > > //Analyze ciliary beat frequency using cropped movies > > //Requires stable movies. Use the stackreg plugin on > //uncropped originals if you can see sample or camera motion. > > setBatchMode(true); > > if(isOpen("diff_Quant")){ > selectWindow("diff_Quant"); > run("Close"); > } > > if(isOpen("Results")) { > selectWindow("Results"); > run("Close"); > } > > run("Clear Results"); > > run("Set Measurements...", "mean limit display redirect=None decimal=5"); > > //Find the movies to analyze > > files = getDirectory("Movies"); > count = 0; > list = getFileList(files); > n = lengthOf(list); > > //Set up the results table > > for (i=0; i < n; i++) { > > result = 0; > fileName = list[i]; > setResult(fileName, result, 1); > updateResults(); > } > > IJ.renameResults("diff_Quant"); > > //Duplicate out the 1st and nth frame of each movie > //and measure the average difference > > for (i=0; i <= n; i++) { > > open(files+list[count]); > stack = getTitle(); > time = 0; > fileName = list[i]; > result = 0; > run("Gaussian Blur 3D...", "x=1 y=1 z=1"); > > n = nSlices(); > > for (slice=1; slice<=(300); slice++){ > > selectWindow(stack); > setSlice(1); > run("Duplicate...", "use"); > first = getTitle(); > > selectWindow(stack); > setSlice(slice + 1); > run("Duplicate...", "use"); > test = getTitle(); > > imageCalculator("Difference create", first, test); > diff = getTitle(); > run("Measure"); > mean = getResult("Mean", 0); > > selectWindow("Results"); > run("Clear Results"); > run("Close"); > > selectWindow("diff_Quant"); > IJ.renameResults("Results"); > result = slice; > setResult(fileName, result, mean); > updateResults(); > IJ.renameResults("diff_Quant"); > > > selectWindow(first); > run("Close"); > selectWindow(test); > run("Close"); > selectWindow(diff); > run("Close"); > time = time + 0.005; > > } > > selectWindow(stack); > run("Close"); > count++; > } > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Timothy, Jim,
in my experience, Linux does not have a problem of slowing down processes unless other programs/processes are competing for CPU time. You could run 'top' in a terminal window in parallel to monitor CPU usage. Did you check that all windows are properly closed, so memory usage does not go up? You might print nImages() from time to time to make sure. Plugins>Utilites>Montor Memory might be also helpful to see whether memory is an issue. Click into the status line of ImageJ from time to time to enforce garbage collection (otherwise memory usage goes up gradually even if everything is properly closed). -- Concerning the average between two different frames of a stack: In a macro, it would be enough to duplicate one of the frames, and set the stack slice to the other, then calculate the difference. In a java plugin or javascript you could get the two ImageProcessors and run the appropriate process/Blitter class, then get the mean of the pixels. Michael ________________________________________________________________ On 12/01/2017 15:50, Ewing James wrote: > Timothy - This sounds like a unix/linux question. See the > documentation on priorities in linux processes. > > As a process ages, its priority for processing may slip downward. > Try using the ‘nice’ command with a negative setting (RTM before you > start fiddling). > > - Jim > >> On Jan 12, 2017, at 9:19 AM, Feinstein, Timothy N <[hidden email]> >> wrote: >> >> I hate to ping the list twice in two days, but this one is a >> puzzle. I wrote a macro that measures the difference between each >> frame of a time series with frame 1, with the ultimate goal of >> measuring the periodicity of the average difference value (with >> Fourier analysis in Excel) to measure ciliary beat frequency. The >> macro uses nested for() loops to open each file in a folder, >> process it framewise and build a results table. >> >> The macro runs pretty fast, but if I walk away from the computer >> for a while at some point it will slow down by almost ten fold, >> making it impractical to run through large data sets overnight. >> Each stack is about 200-500 kb and Fiji has 10 GB of RAM allocated, >> so I doubt that RAM is my problem. I quit all other programs, >> prevented sleep and Fiji remains foregrounded the whole time. I am >> using the latest Fiji update on OSX 10.10.5. >> >> It would be great to know what might cause the slowdown. At the >> same time, odds are my approach could be streamlined quite a bit. >> Right now the macro duplicates out frame 1 and frame n from the >> movie, makes an image of the difference and records the average >> pixel value of the difference image to a results table. Making the >> new images and closing them again seems to take time, even with >> batch mode on. Does anyone know how to measure the average >> difference between frame 1 and frame n of a stack without >> duplicating them out? I pasted the macro below. >> >> Thanks again, >> >> >> T >> >> Timothy Feinstein, Ph.D. Research Scientist University of >> Pittsburgh Department of Developmental Biology >> >> ------------------------------------------------------------ >> >> > //Analyze ciliary beat frequency using cropped movies > > //Requires stable movies. Use the stackreg plugin on > //uncropped originals if you can see sample or camera motion. > > setBatchMode(true); > > if(isOpen("diff_Quant")){ > selectWindow("diff_Quant"); > run("Close"); > } > > if(isOpen("Results")) { > selectWindow("Results"); > run("Close"); > } > > run("Clear Results"); > > run("Set Measurements...", "mean limit display redirect=None decimal=5"); > > //Find the movies to analyze > > files = getDirectory("Movies"); > count = 0; > list = getFileList(files); > n = lengthOf(list); > > //Set up the results table > > for (i=0; i < n; i++) { > > result = 0; > fileName = list[i]; > setResult(fileName, result, 1); > updateResults(); > } > > IJ.renameResults("diff_Quant"); > > //Duplicate out the 1st and nth frame of each movie > //and measure the average difference > > for (i=0; i <= n; i++) { > > open(files+list[count]); > stack = getTitle(); > time = 0; > fileName = list[i]; > result = 0; > run("Gaussian Blur 3D...", "x=1 y=1 z=1"); > > n = nSlices(); > > for (slice=1; slice<=(300); slice++){ > > selectWindow(stack); > setSlice(1); > run("Duplicate...", "use"); > first = getTitle(); > > selectWindow(stack); > setSlice(slice + 1); > run("Duplicate...", "use"); > test = getTitle(); > > imageCalculator("Difference create", first, test); > diff = getTitle(); > run("Measure"); > mean = getResult("Mean", 0); > > selectWindow("Results"); > run("Clear Results"); > run("Close"); > > selectWindow("diff_Quant"); > IJ.renameResults("Results"); > result = slice; > setResult(fileName, result, mean); > updateResults(); > IJ.renameResults("diff_Quant"); > > > selectWindow(first); > run("Close"); > selectWindow(test); > run("Close"); > selectWindow(diff); > run("Close"); > time = time + 0.005; > > } > > selectWindow(stack); > run("Close"); > count++; > } -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
Hi all,
Tim is using MacOS, not Linux. One issue (of several) with Java on MacOS is the "App Nap" problem. It might be on 10.10 that your machine is napping the process, even though it is still in the foreground. See this article: http://osxdaily.com/2014/05/13/disable-app-nap-mac-os-x/ Regards, Curtis -- Curtis Rueden LOCI software architect - http://loci.wisc.edu/software ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden Did you know ImageJ has a forum? http://forum.imagej.net/ On Thu, Jan 12, 2017 at 10:02 AM, Michael Schmid <[hidden email]> wrote: > Hi Timothy, Jim, > > in my experience, Linux does not have a problem of slowing down > processes unless other programs/processes are competing for CPU time. > You could run 'top' in a terminal window in parallel to monitor CPU usage. > > Did you check that all windows are properly closed, so memory usage does > not go up? You might print nImages() from time to time to make sure. > > Plugins>Utilites>Montor Memory might be also helpful to see whether memory > is an issue. Click into the status line of ImageJ from time to time to > enforce garbage collection (otherwise memory usage goes up gradually even > if everything is properly closed). > > -- > Concerning the average between two different frames of a stack: In a > macro, it would be enough to duplicate one of the frames, and set the stack > slice to the other, then calculate the difference. In a java plugin or > javascript you could get the two ImageProcessors and run the appropriate > process/Blitter class, then get the mean of the pixels. > > Michael > ________________________________________________________________ > > On 12/01/2017 15:50, Ewing James wrote: > >> Timothy - This sounds like a unix/linux question. See the >> documentation on priorities in linux processes. >> >> As a process ages, its priority for processing may slip downward. >> Try using the ‘nice’ command with a negative setting (RTM before you >> start fiddling). >> >> - Jim >> >> On Jan 12, 2017, at 9:19 AM, Feinstein, Timothy N <[hidden email]> >>> wrote: >>> >>> I hate to ping the list twice in two days, but this one is a >>> puzzle. I wrote a macro that measures the difference between each >>> frame of a time series with frame 1, with the ultimate goal of >>> measuring the periodicity of the average difference value (with >>> Fourier analysis in Excel) to measure ciliary beat frequency. The >>> macro uses nested for() loops to open each file in a folder, >>> process it framewise and build a results table. >>> >>> The macro runs pretty fast, but if I walk away from the computer >>> for a while at some point it will slow down by almost ten fold, >>> making it impractical to run through large data sets overnight. >>> Each stack is about 200-500 kb and Fiji has 10 GB of RAM allocated, >>> so I doubt that RAM is my problem. I quit all other programs, >>> prevented sleep and Fiji remains foregrounded the whole time. I am >>> using the latest Fiji update on OSX 10.10.5. >>> >>> It would be great to know what might cause the slowdown. At the >>> same time, odds are my approach could be streamlined quite a bit. >>> Right now the macro duplicates out frame 1 and frame n from the >>> movie, makes an image of the difference and records the average >>> pixel value of the difference image to a results table. Making the >>> new images and closing them again seems to take time, even with >>> batch mode on. Does anyone know how to measure the average >>> difference between frame 1 and frame n of a stack without >>> duplicating them out? I pasted the macro below. >>> >>> Thanks again, >>> >>> >>> T >>> >>> Timothy Feinstein, Ph.D. Research Scientist University of >>> Pittsburgh Department of Developmental Biology >>> >>> ------------------------------------------------------------ >>> >>> >>> //Analyze ciliary beat frequency using cropped movies >> >> //Requires stable movies. Use the stackreg plugin on >> //uncropped originals if you can see sample or camera motion. >> >> setBatchMode(true); >> >> if(isOpen("diff_Quant")){ >> selectWindow("diff_Quant"); >> run("Close"); >> } >> >> if(isOpen("Results")) { >> selectWindow("Results"); >> run("Close"); >> } >> >> run("Clear Results"); >> >> run("Set Measurements...", "mean limit display redirect=None decimal=5"); >> >> //Find the movies to analyze >> >> files = getDirectory("Movies"); >> count = 0; >> list = getFileList(files); >> n = lengthOf(list); >> >> //Set up the results table >> >> for (i=0; i < n; i++) { >> >> result = 0; >> fileName = list[i]; >> setResult(fileName, result, 1); >> updateResults(); >> } >> >> IJ.renameResults("diff_Quant"); >> >> //Duplicate out the 1st and nth frame of each movie >> //and measure the average difference >> >> for (i=0; i <= n; i++) { >> >> open(files+list[count]); >> stack = getTitle(); >> time = 0; >> fileName = list[i]; >> result = 0; >> run("Gaussian Blur 3D...", "x=1 y=1 z=1"); >> >> n = nSlices(); >> >> for (slice=1; slice<=(300); slice++){ >> >> selectWindow(stack); >> setSlice(1); >> run("Duplicate...", "use"); >> first = getTitle(); >> >> selectWindow(stack); >> setSlice(slice + 1); >> run("Duplicate...", "use"); >> test = getTitle(); >> >> imageCalculator("Difference create", first, test); >> diff = getTitle(); >> run("Measure"); >> mean = getResult("Mean", 0); >> >> selectWindow("Results"); >> run("Clear Results"); >> run("Close"); >> >> selectWindow("diff_Quant"); >> IJ.renameResults("Results"); >> result = slice; >> setResult(fileName, result, mean); >> updateResults(); >> IJ.renameResults("diff_Quant"); >> >> >> selectWindow(first); >> run("Close"); >> selectWindow(test); >> run("Close"); >> selectWindow(diff); >> run("Close"); >> time = time + 0.005; >> >> } >> >> selectWindow(stack); >> run("Close"); >> count++; >> } >> > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Curtis is right that it's OSX and I used to have a lot of serious issues
like is mentioned and they had to do with window refreshing the menu bar. Wayne put in a temporary fix a while ago and it's been a lot better since then. To the point where I don't experience it much any more at all. When FIJI/ImageJ is out of focus (like you're trying to do something else with your computer) and you're *not* in batch mode it slows down a lot when you switch windows a lot. This I can't figure out because I turned off AppNap long ago and I can't find any other settings within OSX to turn off speed reductions when an App is out of focus. The key is to get to where your macros are fully automatic and you can run in batch mode. Once you're there, it runs great, and if anything, it likes to steal focus while you do other things sometimes closing windows when you click on the wrong thing. HTH, B On Thu, Jan 12, 2017 at 8:36 AM, Curtis Rueden <[hidden email]> wrote: > Hi all, > > Tim is using MacOS, not Linux. > > One issue (of several) with Java on MacOS is the "App Nap" problem. It > might be on 10.10 that your machine is napping the process, even though it > is still in the foreground. See this article: > > http://osxdaily.com/2014/05/13/disable-app-nap-mac-os-x/ > > Regards, > Curtis > > -- > Curtis Rueden > LOCI software architect - http://loci.wisc.edu/software > ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden > Did you know ImageJ has a forum? http://forum.imagej.net/ > > > On Thu, Jan 12, 2017 at 10:02 AM, Michael Schmid <[hidden email]> > wrote: > > > Hi Timothy, Jim, > > > > in my experience, Linux does not have a problem of slowing down > > processes unless other programs/processes are competing for CPU time. > > You could run 'top' in a terminal window in parallel to monitor CPU > usage. > > > > Did you check that all windows are properly closed, so memory usage does > > not go up? You might print nImages() from time to time to make sure. > > > > Plugins>Utilites>Montor Memory might be also helpful to see whether > memory > > is an issue. Click into the status line of ImageJ from time to time to > > enforce garbage collection (otherwise memory usage goes up gradually even > > if everything is properly closed). > > > > -- > > Concerning the average between two different frames of a stack: In a > > macro, it would be enough to duplicate one of the frames, and set the > stack > > slice to the other, then calculate the difference. In a java plugin or > > javascript you could get the two ImageProcessors and run the appropriate > > process/Blitter class, then get the mean of the pixels. > > > > Michael > > ________________________________________________________________ > > > > On 12/01/2017 15:50, Ewing James wrote: > > > >> Timothy - This sounds like a unix/linux question. See the > >> documentation on priorities in linux processes. > >> > >> As a process ages, its priority for processing may slip downward. > >> Try using the ‘nice’ command with a negative setting (RTM before you > >> start fiddling). > >> > >> - Jim > >> > >> On Jan 12, 2017, at 9:19 AM, Feinstein, Timothy N <[hidden email]> > >>> wrote: > >>> > >>> I hate to ping the list twice in two days, but this one is a > >>> puzzle. I wrote a macro that measures the difference between each > >>> frame of a time series with frame 1, with the ultimate goal of > >>> measuring the periodicity of the average difference value (with > >>> Fourier analysis in Excel) to measure ciliary beat frequency. The > >>> macro uses nested for() loops to open each file in a folder, > >>> process it framewise and build a results table. > >>> > >>> The macro runs pretty fast, but if I walk away from the computer > >>> for a while at some point it will slow down by almost ten fold, > >>> making it impractical to run through large data sets overnight. > >>> Each stack is about 200-500 kb and Fiji has 10 GB of RAM allocated, > >>> so I doubt that RAM is my problem. I quit all other programs, > >>> prevented sleep and Fiji remains foregrounded the whole time. I am > >>> using the latest Fiji update on OSX 10.10.5. > >>> > >>> It would be great to know what might cause the slowdown. At the > >>> same time, odds are my approach could be streamlined quite a bit. > >>> Right now the macro duplicates out frame 1 and frame n from the > >>> movie, makes an image of the difference and records the average > >>> pixel value of the difference image to a results table. Making the > >>> new images and closing them again seems to take time, even with > >>> batch mode on. Does anyone know how to measure the average > >>> difference between frame 1 and frame n of a stack without > >>> duplicating them out? I pasted the macro below. > >>> > >>> Thanks again, > >>> > >>> > >>> T > >>> > >>> Timothy Feinstein, Ph.D. Research Scientist University of > >>> Pittsburgh Department of Developmental Biology > >>> > >>> ------------------------------------------------------------ > >>> > >>> > >>> //Analyze ciliary beat frequency using cropped movies > >> > >> //Requires stable movies. Use the stackreg plugin on > >> //uncropped originals if you can see sample or camera motion. > >> > >> setBatchMode(true); > >> > >> if(isOpen("diff_Quant")){ > >> selectWindow("diff_Quant"); > >> run("Close"); > >> } > >> > >> if(isOpen("Results")) { > >> selectWindow("Results"); > >> run("Close"); > >> } > >> > >> run("Clear Results"); > >> > >> run("Set Measurements...", "mean limit display redirect=None > decimal=5"); > >> > >> //Find the movies to analyze > >> > >> files = getDirectory("Movies"); > >> count = 0; > >> list = getFileList(files); > >> n = lengthOf(list); > >> > >> //Set up the results table > >> > >> for (i=0; i < n; i++) { > >> > >> result = 0; > >> fileName = list[i]; > >> setResult(fileName, result, 1); > >> updateResults(); > >> } > >> > >> IJ.renameResults("diff_Quant"); > >> > >> //Duplicate out the 1st and nth frame of each movie > >> //and measure the average difference > >> > >> for (i=0; i <= n; i++) { > >> > >> open(files+list[count]); > >> stack = getTitle(); > >> time = 0; > >> fileName = list[i]; > >> result = 0; > >> run("Gaussian Blur 3D...", "x=1 y=1 z=1"); > >> > >> n = nSlices(); > >> > >> for (slice=1; slice<=(300); slice++){ > >> > >> selectWindow(stack); > >> setSlice(1); > >> run("Duplicate...", "use"); > >> first = getTitle(); > >> > >> selectWindow(stack); > >> setSlice(slice + 1); > >> run("Duplicate...", "use"); > >> test = getTitle(); > >> > >> imageCalculator("Difference create", first, test); > >> diff = getTitle(); > >> run("Measure"); > >> mean = getResult("Mean", 0); > >> > >> selectWindow("Results"); > >> run("Clear Results"); > >> run("Close"); > >> > >> selectWindow("diff_Quant"); > >> IJ.renameResults("Results"); > >> result = slice; > >> setResult(fileName, result, mean); > >> updateResults(); > >> IJ.renameResults("diff_Quant"); > >> > >> > >> selectWindow(first); > >> run("Close"); > >> selectWindow(test); > >> run("Close"); > >> selectWindow(diff); > >> run("Close"); > >> time = time + 0.005; > >> > >> } > >> > >> selectWindow(stack); > >> run("Close"); > >> count++; > >> } > >> > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi folks,
Thanks to everyone who replied on and off list. People are no doubt right that OSX (and possibly Windows?) does not respond well to a Java macro that opens and closes and switches between windows thousands of times at a fever speed. I was able to streamline my macro dramatically when I found out that you can subtract one image from a whole stack: http://imagej.1557.x6.nabble.com/Image-Calculator-Image-Slices-minus-Single-Image-td5015901.html I also gave up on creating a custom results table with a large 2D array of measurement results (sample ID x slice number). Instead I recorded one long results table and split out the individual stacks in Excel with some help from the INDIRECT() command. The updated macro is below. All the best, Tim Timothy Feinstein, Ph.D. Research Scientist University of Pittsburgh Department of Developmental Biology ---------------------------------------------------- //Analyze ciliary beat frequency using cropped movies //Requires stable movies. Use the stackreg plugin on //uncropped originals if you can see sample or camera motion. setBatchMode(true); if(isOpen("Results")) { selectWindow("Results"); run("Close"); } run("Clear Results"); setResult("Title", 0, 1); run("Set Measurements...", "mean redirect=None decimal=5"); //Find the movies to analyze files = getDirectory("Movies"); count = 0; list = getFileList(files); n = lengthOf(list); //Set up the results table //Duplicate out the 1st and nth frame of each movie //and measure the average difference for (i=0; i < n; i++) { open(files+list[count]); stack = getTitle(); fileName = list[i]; slices = nSlices(); if (slices < 310) { selectWindow(stack); run("Close"); count++; } else { run("Gaussian Blur 3D...", "x=1 y=1 z=1"); selectWindow(stack); setSlice(1); run("Duplicate...", "use"); first = getTitle(); imageCalculator("Difference create stack", stack, first); diff = getTitle(); for (slice=1; slice<=300; slice++){ selectWindow(diff); setSlice(slice+10); run("Measure"); number = (nResults-1); setResult("Title", number, stack); } selectWindow(stack); run("Close"); selectWindow(diff); run("Close"); selectWindow(first); run("Close"); count++; } } -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
In reply to this post by Brandon Hurr
Brandon Hurr said:
> -----Original Message----- > Sent: 12 January 2017 17:52 > Subject: Re: Macro slowing down > > Curtis is right that it's OSX and I used to have a lot of serious > issues like is mentioned and they had to do with window refreshing the menu bar. I often have a problem if using ImageJ on Linux via an X-tunnel connection. I do this in order to run the ImageJ on a much more capable server with a faster connection to the data storage, but which lives in a rack in a computer cabinet. After a little while, though the PROCESSING of data by macro or builtin function is nice and fast, the OPENING of dialog boxes for parameter input, or file selection, etc . becomes extremely slow. I suspect it also has to do with window refreshing menu bar because I see some icons in the menu bar moving or resizing during the wait. Is there a workaround, does anyone also see this happen, is there a way to prevent it? I found one workaround, to put any command that opens a dialog box for parameters into a macro of one line, and just run that; if it avoids opening the dialog box then it runs very fast. Oddly, opening IMAGE display box doesn't seem to have this problem . Thanks Robert ____________________ Dr. Robert C. Atwood Senior Support Scientist Beamline I12-JEEP The Joint Engineering and Environmental Processing Beamline Diamond Light Source The Harwell Science and Innovation Campus Didcot,OXON OX11 0DE +44 (0) 1235 778 670 -----Original Message----- From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Brandon Hurr Sent: Thursday, January 12, 2017 5:52 PM To: [hidden email] Subject: Re: Macro slowing down Curtis is right that it's OSX and I used to have a lot of serious issues like is mentioned and they had to do with window refreshing the menu bar. Wayne put in a temporary fix a while ago and it's been a lot better since then. To the point where I don't experience it much any more at all. When FIJI/ImageJ is out of focus (like you're trying to do something else with your computer) and you're *not* in batch mode it slows down a lot when you switch windows a lot. This I can't figure out because I turned off AppNap long ago and I can't find any other settings within OSX to turn off speed reductions when an App is out of focus. The key is to get to where your macros are fully automatic and you can run in batch mode. Once you're there, it runs great, and if anything, it likes to steal focus while you do other things sometimes closing windows when you click on the wrong thing. HTH, B On Thu, Jan 12, 2017 at 8:36 AM, Curtis Rueden <[hidden email]> wrote: > Hi all, > > Tim is using MacOS, not Linux. > > One issue (of several) with Java on MacOS is the "App Nap" problem. It > might be on 10.10 that your machine is napping the process, even > though it is still in the foreground. See this article: > > http://osxdaily.com/2014/05/13/disable-app-nap-mac-os-x/ > > Regards, > Curtis > > -- > Curtis Rueden > LOCI software architect - http://loci.wisc.edu/software > ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden Did you > know ImageJ has a forum? http://forum.imagej.net/ > > > On Thu, Jan 12, 2017 at 10:02 AM, Michael Schmid > <[hidden email]> > wrote: > > > Hi Timothy, Jim, > > > > in my experience, Linux does not have a problem of slowing down > > processes unless other programs/processes are competing for CPU time. > > You could run 'top' in a terminal window in parallel to monitor CPU > usage. > > > > Did you check that all windows are properly closed, so memory usage > > does not go up? You might print nImages() from time to time to make sure. > > > > Plugins>Utilites>Montor Memory might be also helpful to see whether > memory > > is an issue. Click into the status line of ImageJ from time to time > > to enforce garbage collection (otherwise memory usage goes up > > gradually even if everything is properly closed). > > > > -- > > Concerning the average between two different frames of a stack: In a > > macro, it would be enough to duplicate one of the frames, and set > > the > stack > > slice to the other, then calculate the difference. In a java plugin > > or javascript you could get the two ImageProcessors and run the > > appropriate process/Blitter class, then get the mean of the pixels. > > > > Michael > > ________________________________________________________________ > > > > On 12/01/2017 15:50, Ewing James wrote: > > > >> Timothy - This sounds like a unix/linux question. See the > >> documentation on priorities in linux processes. > >> > >> As a process ages, its priority for processing may slip downward. > >> Try using the ‘nice’ command with a negative setting (RTM before > >> you start fiddling). > >> > >> - Jim > >> > >> On Jan 12, 2017, at 9:19 AM, Feinstein, Timothy N <[hidden email]> > >>> wrote: > >>> > >>> I hate to ping the list twice in two days, but this one is a > >>> puzzle. I wrote a macro that measures the difference between each > >>> frame of a time series with frame 1, with the ultimate goal of > >>> measuring the periodicity of the average difference value (with > >>> Fourier analysis in Excel) to measure ciliary beat frequency. The > >>> macro uses nested for() loops to open each file in a folder, > >>> process it framewise and build a results table. > >>> > >>> The macro runs pretty fast, but if I walk away from the computer > >>> for a while at some point it will slow down by almost ten fold, > >>> making it impractical to run through large data sets overnight. > >>> Each stack is about 200-500 kb and Fiji has 10 GB of RAM > >>> allocated, so I doubt that RAM is my problem. I quit all other > >>> programs, prevented sleep and Fiji remains foregrounded the whole > >>> time. I am using the latest Fiji update on OSX 10.10.5. > >>> > >>> It would be great to know what might cause the slowdown. At the > >>> same time, odds are my approach could be streamlined quite a bit. > >>> Right now the macro duplicates out frame 1 and frame n from the > >>> movie, makes an image of the difference and records the average > >>> pixel value of the difference image to a results table. Making > >>> the new images and closing them again seems to take time, even > >>> with batch mode on. Does anyone know how to measure the average > >>> difference between frame 1 and frame n of a stack without > >>> duplicating them out? I pasted the macro below. > >>> > >>> Thanks again, > >>> > >>> > >>> T > >>> > >>> Timothy Feinstein, Ph.D. Research Scientist University of > >>> Pittsburgh Department of Developmental Biology > >>> > >>> ------------------------------------------------------------ > >>> > >>> > >>> //Analyze ciliary beat frequency using cropped movies > >> > >> //Requires stable movies. Use the stackreg plugin on //uncropped > >> originals if you can see sample or camera motion. > >> > >> setBatchMode(true); > >> > >> if(isOpen("diff_Quant")){ > >> selectWindow("diff_Quant"); > >> run("Close"); > >> } > >> > >> if(isOpen("Results")) { > >> selectWindow("Results"); > >> run("Close"); > >> } > >> > >> run("Clear Results"); > >> > >> run("Set Measurements...", "mean limit display redirect=None > decimal=5"); > >> > >> //Find the movies to analyze > >> > >> files = getDirectory("Movies"); > >> count = 0; > >> list = getFileList(files); > >> n = lengthOf(list); > >> > >> //Set up the results table > >> > >> for (i=0; i < n; i++) { > >> > >> result = 0; > >> fileName = list[i]; > >> setResult(fileName, result, 1); > >> updateResults(); > >> } > >> > >> IJ.renameResults("diff_Quant"); > >> > >> //Duplicate out the 1st and nth frame of each movie //and measure > >> the average difference > >> > >> for (i=0; i <= n; i++) { > >> > >> open(files+list[count]); > >> stack = getTitle(); > >> time = 0; > >> fileName = list[i]; > >> result = 0; > >> run("Gaussian Blur 3D...", "x=1 y=1 z=1"); > >> > >> n = nSlices(); > >> > >> for (slice=1; slice<=(300); slice++){ > >> > >> selectWindow(stack); > >> setSlice(1); > >> run("Duplicate...", "use"); > >> first = getTitle(); > >> > >> selectWindow(stack); > >> setSlice(slice + 1); > >> run("Duplicate...", "use"); > >> test = getTitle(); > >> > >> imageCalculator("Difference create", first, test); > >> diff = getTitle(); > >> run("Measure"); > >> mean = getResult("Mean", 0); > >> > >> selectWindow("Results"); > >> run("Clear Results"); > >> run("Close"); > >> > >> selectWindow("diff_Quant"); > >> IJ.renameResults("Results"); > >> result = slice; > >> setResult(fileName, result, mean); > >> updateResults(); > >> IJ.renameResults("diff_Quant"); > >> > >> > >> selectWindow(first); > >> run("Close"); > >> selectWindow(test); > >> run("Close"); > >> selectWindow(diff); > >> run("Close"); > >> time = time + 0.005; > >> > >> } > >> > >> selectWindow(stack); > >> run("Close"); > >> count++; > >> } > >> > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Robert,
You could try generating a stack trace when things start to get very slow; see: http://imagej.net/Troubleshooting#If_ImageJ_freezes_or_hangs The stack trace may give good hints as to which code is the culprit. My guess is that it will be something deep in Java AWT, which would need to fixed in Java itself, or else worked around in the ImageJ codebase somehow. But we won't know until you check. Regards, Curtis -- Curtis Rueden LOCI software architect - https://loci.wisc.edu/software ImageJ2 lead, Fiji maintainer - https://imagej.net/User:Rueden Did you know ImageJ has a forum? http://forum.imagej.net/ On Tue, Jan 31, 2017 at 8:32 AM, Robert Atwood <[hidden email]> wrote: > Brandon Hurr said: > > > > -----Original Message----- > > Sent: 12 January 2017 17:52 > > > Subject: Re: Macro slowing down > > > > Curtis is right that it's OSX and I used to have a lot of serious > > issues like is mentioned and they had to do with window refreshing the > menu bar. > > I often have a problem if using ImageJ on Linux via an X-tunnel > connection. I do this in order to run the ImageJ on a much more capable > server with a faster connection to the data storage, but which lives in a > rack in a computer cabinet. > > After a little while, though the PROCESSING of data by macro or builtin > function is nice and fast, the OPENING of dialog boxes for parameter input, > or file selection, etc . becomes extremely slow. I suspect it also has to > do with window refreshing menu bar because I see some icons in the menu bar > moving or resizing during the wait. > > Is there a workaround, does anyone also see this happen, is there a way to > prevent it? I found one workaround, to put any command that opens a dialog > box for parameters into a macro of one line, and just run that; if it > avoids opening the dialog box then it runs very fast. Oddly, opening IMAGE > display box doesn't seem to have this problem . > > > Thanks > Robert > > ____________________ > > Dr. Robert C. Atwood > Senior Support Scientist > Beamline I12-JEEP > The Joint Engineering and Environmental Processing Beamline > Diamond Light Source > The Harwell Science and Innovation Campus > Didcot,OXON > OX11 0DE > +44 (0) 1235 778 670 > > -----Original Message----- > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > Brandon Hurr > Sent: Thursday, January 12, 2017 5:52 PM > To: [hidden email] > Subject: Re: Macro slowing down > > Curtis is right that it's OSX and I used to have a lot of serious issues > like is mentioned and they had to do with window refreshing the menu bar. > Wayne put in a temporary fix a while ago and it's been a lot better since > then. To the point where I don't experience it much any more at all. > > When FIJI/ImageJ is out of focus (like you're trying to do something else > with your computer) and you're *not* in batch mode it slows down a lot when > you switch windows a lot. This I can't figure out because I turned off > AppNap long ago and I can't find any other settings within OSX to turn off > speed reductions when an App is out of focus. > > The key is to get to where your macros are fully automatic and you can run > in batch mode. Once you're there, it runs great, and if anything, it likes > to steal focus while you do other things sometimes closing windows when you > click on the wrong thing. > > HTH, > B > > On Thu, Jan 12, 2017 at 8:36 AM, Curtis Rueden <[hidden email]> wrote: > > > Hi all, > > > > Tim is using MacOS, not Linux. > > > > One issue (of several) with Java on MacOS is the "App Nap" problem. It > > might be on 10.10 that your machine is napping the process, even > > though it is still in the foreground. See this article: > > > > http://osxdaily.com/2014/05/13/disable-app-nap-mac-os-x/ > > > > Regards, > > Curtis > > > > -- > > Curtis Rueden > > LOCI software architect - http://loci.wisc.edu/software > > ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden Did you > > know ImageJ has a forum? http://forum.imagej.net/ > > > > > > On Thu, Jan 12, 2017 at 10:02 AM, Michael Schmid > > <[hidden email]> > > wrote: > > > > > Hi Timothy, Jim, > > > > > > in my experience, Linux does not have a problem of slowing down > > > processes unless other programs/processes are competing for CPU time. > > > You could run 'top' in a terminal window in parallel to monitor CPU > > usage. > > > > > > Did you check that all windows are properly closed, so memory usage > > > does not go up? You might print nImages() from time to time to make > sure. > > > > > > Plugins>Utilites>Montor Memory might be also helpful to see whether > > memory > > > is an issue. Click into the status line of ImageJ from time to time > > > to enforce garbage collection (otherwise memory usage goes up > > > gradually even if everything is properly closed). > > > > > > -- > > > Concerning the average between two different frames of a stack: In a > > > macro, it would be enough to duplicate one of the frames, and set > > > the > > stack > > > slice to the other, then calculate the difference. In a java plugin > > > or javascript you could get the two ImageProcessors and run the > > > appropriate process/Blitter class, then get the mean of the pixels. > > > > > > Michael > > > ________________________________________________________________ > > > > > > On 12/01/2017 15:50, Ewing James wrote: > > > > > >> Timothy - This sounds like a unix/linux question. See the > > >> documentation on priorities in linux processes. > > >> > > >> As a process ages, its priority for processing may slip downward. > > >> Try using the ‘nice’ command with a negative setting (RTM before > > >> you start fiddling). > > >> > > >> - Jim > > >> > > >> On Jan 12, 2017, at 9:19 AM, Feinstein, Timothy N <[hidden email]> > > >>> wrote: > > >>> > > >>> I hate to ping the list twice in two days, but this one is a > > >>> puzzle. I wrote a macro that measures the difference between each > > >>> frame of a time series with frame 1, with the ultimate goal of > > >>> measuring the periodicity of the average difference value (with > > >>> Fourier analysis in Excel) to measure ciliary beat frequency. The > > >>> macro uses nested for() loops to open each file in a folder, > > >>> process it framewise and build a results table. > > >>> > > >>> The macro runs pretty fast, but if I walk away from the computer > > >>> for a while at some point it will slow down by almost ten fold, > > >>> making it impractical to run through large data sets overnight. > > >>> Each stack is about 200-500 kb and Fiji has 10 GB of RAM > > >>> allocated, so I doubt that RAM is my problem. I quit all other > > >>> programs, prevented sleep and Fiji remains foregrounded the whole > > >>> time. I am using the latest Fiji update on OSX 10.10.5. > > >>> > > >>> It would be great to know what might cause the slowdown. At the > > >>> same time, odds are my approach could be streamlined quite a bit. > > >>> Right now the macro duplicates out frame 1 and frame n from the > > >>> movie, makes an image of the difference and records the average > > >>> pixel value of the difference image to a results table. Making > > >>> the new images and closing them again seems to take time, even > > >>> with batch mode on. Does anyone know how to measure the average > > >>> difference between frame 1 and frame n of a stack without > > >>> duplicating them out? I pasted the macro below. > > >>> > > >>> Thanks again, > > >>> > > >>> > > >>> T > > >>> > > >>> Timothy Feinstein, Ph.D. Research Scientist University of > > >>> Pittsburgh Department of Developmental Biology > > >>> > > >>> ------------------------------------------------------------ > > >>> > > >>> > > >>> //Analyze ciliary beat frequency using cropped movies > > >> > > >> //Requires stable movies. Use the stackreg plugin on //uncropped > > >> originals if you can see sample or camera motion. > > >> > > >> setBatchMode(true); > > >> > > >> if(isOpen("diff_Quant")){ > > >> selectWindow("diff_Quant"); > > >> run("Close"); > > >> } > > >> > > >> if(isOpen("Results")) { > > >> selectWindow("Results"); > > >> run("Close"); > > >> } > > >> > > >> run("Clear Results"); > > >> > > >> run("Set Measurements...", "mean limit display redirect=None > > decimal=5"); > > >> > > >> //Find the movies to analyze > > >> > > >> files = getDirectory("Movies"); > > >> count = 0; > > >> list = getFileList(files); > > >> n = lengthOf(list); > > >> > > >> //Set up the results table > > >> > > >> for (i=0; i < n; i++) { > > >> > > >> result = 0; > > >> fileName = list[i]; > > >> setResult(fileName, result, 1); > > >> updateResults(); > > >> } > > >> > > >> IJ.renameResults("diff_Quant"); > > >> > > >> //Duplicate out the 1st and nth frame of each movie //and measure > > >> the average difference > > >> > > >> for (i=0; i <= n; i++) { > > >> > > >> open(files+list[count]); > > >> stack = getTitle(); > > >> time = 0; > > >> fileName = list[i]; > > >> result = 0; > > >> run("Gaussian Blur 3D...", "x=1 y=1 z=1"); > > >> > > >> n = nSlices(); > > >> > > >> for (slice=1; slice<=(300); slice++){ > > >> > > >> selectWindow(stack); > > >> setSlice(1); > > >> run("Duplicate...", "use"); > > >> first = getTitle(); > > >> > > >> selectWindow(stack); > > >> setSlice(slice + 1); > > >> run("Duplicate...", "use"); > > >> test = getTitle(); > > >> > > >> imageCalculator("Difference create", first, test); > > >> diff = getTitle(); > > >> run("Measure"); > > >> mean = getResult("Mean", 0); > > >> > > >> selectWindow("Results"); > > >> run("Clear Results"); > > >> run("Close"); > > >> > > >> selectWindow("diff_Quant"); > > >> IJ.renameResults("Results"); > > >> result = slice; > > >> setResult(fileName, result, mean); > > >> updateResults(); > > >> IJ.renameResults("diff_Quant"); > > >> > > >> > > >> selectWindow(first); > > >> run("Close"); > > >> selectWindow(test); > > >> run("Close"); > > >> selectWindow(diff); > > >> run("Close"); > > >> time = time + 0.005; > > >> > > >> } > > >> > > >> selectWindow(stack); > > >> run("Close"); > > >> count++; > > >> } > > >> > > > > > > -- > > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > -- > This e-mail and any attachments may contain confidential, copyright and or > privileged material, and are for the use of the intended addressee only. If > you are not the intended addressee or an authorised recipient of the > addressee please notify us of receipt by returning the e-mail and do not > use, copy, retain, distribute or disclose the information in or attached to > the e-mail. > Any opinions expressed within this e-mail are those of the individual and > not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for any > damage which you may sustain as a result of software viruses which may be > transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in England > and Wales with its registered office at Diamond House, Harwell Science and > Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
«
Return to ImageJ
|
1 view|%1 views
| Free forum by Nabble | Edit this page |