Workflow for importing DV video.
Locked
Workflow for importing DV video.
 
|
I am looking for the "best" way to import DV formatted video into ImageJ. My
current workflow consists: 1. Grabbing the video in raw mode using dvgrab on Linux or Vidi on OSX. 2. Extracting the raw RGB data using transcode 3. Discarding the GB channels, retaining the raw file containing 8 bit red channel data. 4. Importing these raw frames into ImageJ I would like to switch my workflow (in order to deal with the interlacing of the source video) 1. Grab raw video 2. Deinterlace and extract raw data (YV12 format, using ffmpeg) 3. Discard the chroma channels (UV, keeping Y') 4. Import the luma channel into ImageJ However I'm wondering about 3 and 4. It seems like I might be missing something obvious, and before I go writing a filter to discard the UV data, I thought I'd ask here in the hopes that someone else can offer advice on a more effective workflow, or at least let me know if this is even going to work. Regards, William Day [hidden email] |
|
|
Hi!
I have the following machine: Dell, dual core Pentium, 4Gb RAM. My ImageJ version is the lates one (w) and my memory is set to the max allowed (1.7G). I am trying to run Deconvolve3D plugin on a 16bit TIFF file and I have the following problem. First I make a PSF (I usually get R values of 3. something) and then I use that as the PSF input in th eplugin. Then the plugin crashes with the following message: >>>>>>>>>>>>>>>>>>>>>>>>>>> <Out of memory> <All available memory (1587MB) has been> <used. Instructions for making more> <available can be found in the "Memory" > <sections of the installation notes at> <http://rsb.info.nih.gov/ij/docs/install/> >>>>>>>>>>>>>>>>>>>>>>>>>>> I tried it with other bit depths as well but the problem persisted. Could anyone help me out? Thank you! Przemko -- Przemko Tylzanowski Ph.D. LSD & Joint, box 813-GREEN O & N University of Leuven Herestraat 49 3000 Leuven Belgium phone: (32-16)34-61-96 fax : (32-16)34-62-00 Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm |
Re: Deconvolve 3D plugin problem
|
|
Przemko,
Bit depths don't help because the plugin converts the input to 32-bit floating point for its internal calculations. At the moment, the only option that I know of is to process smaller image stacks. Bob Robert P. Dougherty, Ph.D. President, OptiNav, Inc. Phone (425) 990-5912 Fax (425) 467-1119 www.optinav.com > -----Original Message----- > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > Przemko > Sent: Wednesday, July 04, 2007 4:24 AM > To: [hidden email] > Subject: Deconvolve 3D plugin problem > > Hi! > I have the following machine: Dell, dual core Pentium, 4Gb RAM. My > ImageJ version is the lates one (w) and my memory is set to the max > allowed (1.7G). I am trying to run Deconvolve3D plugin on a 16bit TIFF > file and I have the following problem. First I make a PSF (I usually get > R values of 3. something) and then I use that as the PSF input in th > eplugin. Then the plugin crashes with the following message: > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > <Out of memory> > <All available memory (1587MB) has been> > <used. Instructions for making more> > <available can be found in the "Memory" > > <sections of the installation notes at> > <http://rsb.info.nih.gov/ij/docs/install/> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > I tried it with other bit depths as well but the problem persisted. > Could anyone help me out? > Thank you! > Przemko > > > > -- > Przemko Tylzanowski Ph.D. > LSD & Joint, box 813-GREEN > O & N > University of Leuven > Herestraat 49 > 3000 Leuven > Belgium > > phone: (32-16)34-61-96 > fax : (32-16)34-62-00 > > > > Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm |
Re: Deconvolve 3D plugin problem
|
|
In reply to this post by Przemko
Hello Przemko,
The issue you are experiencing occurs because the plugin doubles the size of the original image when working with it, and there is an overhead operating space that is required for ImageJ. I am going to assume that you are using biomedical 3D image files. If this is the case, a simple work-around would be to File>Save As>Image Sequence for the large multi-slice TIFF file, after which you can load (File>Import>Image Sequence) smaller subsets of the original and subsequently run the plugin. Be sure to only load 1/2 of your max (1587 MB max, leaving ~794 MB) in any one subset. After the plugin is run, you can save the output as an image sequence again and numerate appropriately, which will allow you to load the entire image as a whole at a later point in time. As for a out-and-out fix to the issue, you would require a 'partial-loader' that only works with the set of data being directly transformed by the plugin (i.e. non-RAM resident data). If one exists, I would like to see the code for it as well. Regards, Tom Elliot http://gis.lrs.uoguelph.ca/soilct/ -----Original Message----- From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Przemko Sent: July 4, 2007 7:24 AM To: [hidden email] Subject: Deconvolve 3D plugin problem Hi! I have the following machine: Dell, dual core Pentium, 4Gb RAM. My ImageJ version is the lates one (w) and my memory is set to the max allowed (1.7G). I am trying to run Deconvolve3D plugin on a 16bit TIFF file and I have the following problem. First I make a PSF (I usually get R values of 3. something) and then I use that as the PSF input in th eplugin. Then the plugin crashes with the following message: >>>>>>>>>>>>>>>>>>>>>>>>>>> <Out of memory> <All available memory (1587MB) has been> <used. Instructions for making more> <available can be found in the "Memory" > <sections of the installation notes at> <http://rsb.info.nih.gov/ij/docs/install/> >>>>>>>>>>>>>>>>>>>>>>>>>>> I tried it with other bit depths as well but the problem persisted. Could anyone help me out? Thank you! Przemko -- Przemko Tylzanowski Ph.D. LSD & Joint, box 813-GREEN O & N University of Leuven Herestraat 49 3000 Leuven Belgium phone: (32-16)34-61-96 fax : (32-16)34-62-00 Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm |
Re: Deconvolve 3D plugin problem
|
|
Hi Przemko
You don't say how large your image is, in terms of pixels. As Tom pointed out, the deconvolution plugincreates a space that is at least four times that of your original image, so that it can use the FFT algorithm. That algorithm requires that the image be a power of 2 square. In order to satisfy this criterion, the original image is padded in each direction. You might be able to find more memory by scaling your images down, by a factor of 2, or perhaps 4, if they can take it with no loss of detail. Joel Date sent: Wed, 4 Jul 2007 10:23:34 -0400 Send reply to: ImageJ Interest Group <[hidden email]> From: Tom Elliot <[hidden email]> Subject: Re: Deconvolve 3D plugin problem To: [hidden email] > Hello Przemko, > > The issue you are experiencing occurs because the plugin doubles the size of > the original image when working with it, and there is an overhead operating > space that is required for ImageJ. > > I am going to assume that you are using biomedical 3D image files. If this > is the case, a simple work-around would be to File>Save As>Image Sequence > for the large multi-slice TIFF file, after which you can load > (File>Import>Image Sequence) smaller subsets of the original and > subsequently run the plugin. Be sure to only load 1/2 of your max (1587 MB > max, leaving ~794 MB) in any one subset. After the plugin is run, you can > save the output as an image sequence again and numerate appropriately, which > will allow you to load the entire image as a whole at a later point in time. > > As for a out-and-out fix to the issue, you would require a 'partial-loader' > that only works with the set of data being directly transformed by the > plugin (i.e. non-RAM resident data). If one exists, I would like to see the > code for it as well. > > Regards, > Tom Elliot > > http://gis.lrs.uoguelph.ca/soilct/ > > > > > -----Original Message----- > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > Przemko > Sent: July 4, 2007 7:24 AM > To: [hidden email] > Subject: Deconvolve 3D plugin problem > > Hi! > I have the following machine: Dell, dual core Pentium, 4Gb RAM. My ImageJ > version is the lates one (w) and my memory is set to the max allowed (1.7G). > I am trying to run Deconvolve3D plugin on a 16bit TIFF file and I have the > following problem. First I make a PSF (I usually get R values of 3. > something) and then I use that as the PSF input in th eplugin. Then the > plugin crashes with the following message: > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > <Out of memory> > <All available memory (1587MB) has been> <used. Instructions for making > more> <available can be found in the "Memory" > <sections of the > installation notes at> <http://rsb.info.nih.gov/ij/docs/install/> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > I tried it with other bit depths as well but the problem persisted. > Could anyone help me out? > Thank you! > Przemko > > > > -- > Przemko Tylzanowski Ph.D. > LSD & Joint, box 813-GREEN > O & N > University of Leuven > Herestraat 49 > 3000 Leuven > Belgium > > phone: (32-16)34-61-96 > fax : (32-16)34-62-00 > > > > Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm Joel B. Sheffield, Ph.D Department of Biology Temple University Philadelphia, PA 19122 Voice: 215 204 8839 e-mail: [hidden email] |
|
|
In reply to this post by Przemko
Gentlemen, thank you for the prompt response. Here is some more info
about my images and what happens. So, first of all I am not doing a 3D medical imaging but immunofluorescence. Normally I use the Iterative Deconvolve plugin and it works on my images rather well. I have decided however to give it a try with Deconvolve 3D because some of my images would not be processed to the point that I was happy with them. Namely, I am looking at the colocalization of two proteins and sometimes, with insufficient deconvolution, I had troubles deciding if the two proteins colocalized or not. My images are 1315x1033 pixels. If I do Diffraction Limit PSF and then Iterative Deconvolution, the rpogram works and on my machine it takes about 60 seconds to 5 rounds of deconvolution. Now if I go with the same file to Diffraction PSF 3D and Iterative Deconvolution 3D, I get the crash as described before. If I decrease the number of pixel by half (loosing some of the resolution which I need), it seems that the program goes a bit longer (2-3 min) before crashing as mentioned earlier. The reason to go for the 3D plugin comes from the info on the plugin site stating "This plugin is intended to replace both the Iterative Deconvolution and Iterative Deconvolution 2 plugins for 2D applications, as well as extending the function to 3D" So, here I am. What I wonder is, as one of you suggested, so do the processing in parts so that the memory usage is better controlled. Would that be somehow possible? Again, thank you for your help! Przemko -- Przemko Tylzanowski Ph.D. LSD & Joint, box 813-GREEN O & N University of Leuven Herestraat 49 3000 Leuven Belgium phone: (32-16)34-61-96 fax : (32-16)34-62-00 -- Przemko Tylzanowski Ph.D. LSD & Joint, box 813-GREEN O & N University of Leuven Herestraat 49 3000 Leuven Belgium phone: (32-16)34-61-96 fax : (32-16)34-62-00 Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm |
Re: Deconvolve 3D plugin problem
|
|
How many images are you trying to process at once? Similarly, how
many slices are taken up by the 3D psf? You might be able to get the result you want by defining the psf in fewer slices. Joel Date sent: Thu, 5 Jul 2007 11:20:40 +0200 Send reply to: ImageJ Interest Group <[hidden email]> From: Przemko <[hidden email]> Subject: Re: Deconvolve 3D plugin problem To: [hidden email] > Gentlemen, thank you for the prompt response. Here is some more info > about my images and what happens. So, first of all I am not doing a 3D > medical imaging but immunofluorescence. Normally I use the Iterative > Deconvolve plugin and it works on my images rather well. I have decided > however to give it a try with Deconvolve 3D because some of my images > would not be processed to the point that I was happy with them. Namely, > I am looking at the colocalization of two proteins and sometimes, with > insufficient deconvolution, I had troubles deciding if the two proteins > colocalized or not. > My images are 1315x1033 pixels. If I do Diffraction Limit PSF and then > Iterative Deconvolution, the rpogram works and on my machine it takes > about 60 seconds to 5 rounds of deconvolution. Now if I go with the same > file to Diffraction PSF 3D and Iterative Deconvolution 3D, I get the > crash as described before. If I decrease the number of pixel by half > (loosing some of the resolution which I need), it seems that the program > goes a bit longer (2-3 min) before crashing as mentioned earlier. The > reason to go for the 3D plugin comes from the info on the plugin site > stating "This plugin is intended to replace both the Iterative > Deconvolution and Iterative Deconvolution 2 plugins for 2D applications, > as well as extending the function to 3D" > > So, here I am. What I wonder is, as one of you suggested, so do the > processing in parts so that the memory usage is better controlled. Would > that be somehow possible? > > Again, thank you for your help! > > Przemko > > -- > Przemko Tylzanowski Ph.D. > LSD & Joint, box 813-GREEN > O & N > University of Leuven > Herestraat 49 > 3000 Leuven > Belgium > > phone: (32-16)34-61-96 > fax : (32-16)34-62-00 > > > > > > -- > Przemko Tylzanowski Ph.D. > LSD & Joint, box 813-GREEN > O & N > University of Leuven > Herestraat 49 > 3000 Leuven > Belgium > > phone: (32-16)34-61-96 > fax : (32-16)34-62-00 > > > > Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm -- Joel B. Sheffield, Ph.D. Biology Department, Temple University 1900 North 12th Street Philadelphia, PA 19122 [hidden email] (215) 204 8839, fax (215) 204 0486 http://astro.temple.edu/~jbs |
|
|
Joel Sheffield wrote:
> How many images are you trying to process at once? Similarly, how > many slices are taken up by the 3D psf? You might be able to get the > result you want by defining the psf in fewer slices. > I started with one and never moved to more because of the crash. My PSF has 34 slices. Interestingly I looked into the task manager. When I run Diffraction Limit PSF and then Iterative Deconvolution the memory usage was around 1G but when I do the 3D the memory usage goes above the magic value of 1.7. Another thing that I noticed was that I could the 3D plugin to work on an image of 300x300 pixels. This however is way too small for my applications. Przemko -- Przemko Tylzanowski Ph.D. LSD & Joint, box 813-GREEN O & N University of Leuven Herestraat 49 3000 Leuven Belgium phone: (32-16)34-61-96 fax : (32-16)34-62-00 Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm |
Re: Deconvolve 3D plugin problem
|
|
See what happens if you make a smaller psf. my guess is that even
for a single image, you are creating a volume of at least 64 times the 2D volume based on the need for a power of 2 in the z dimension.. So, if your original image were 1000x1000, that would be padded in the x,y dimension to 2048x2048 (i.e. approx 4 mb). This would then have to be multiplied by 64, yielding a minimum of 266 mb. If the volume needs to be a cube (I don't know the algorithm well enough), you need even more. Re the 300x300 success, can you extract that sized subset from your original image, and not have to scale it? Joel Date sent: Thu, 5 Jul 2007 16:23:38 +0200 Send reply to: ImageJ Interest Group <[hidden email]> From: Przemko <[hidden email]> Subject: Re: Deconvolve 3D plugin problem To: [hidden email] > Joel Sheffield wrote: > > How many images are you trying to process at once? Similarly, how > > many slices are taken up by the 3D psf? You might be able to get the > > result you want by defining the psf in fewer slices. > > > > I started with one and never moved to more because of the crash. My PSF > has 34 slices. Interestingly I looked into the task manager. When I run > Diffraction Limit PSF and then Iterative Deconvolution the memory usage > was around 1G but when I do the 3D the memory usage goes above the magic > value of 1.7. > Another thing that I noticed was that I could the 3D plugin to work on > an image of 300x300 pixels. This however is way too small for my > applications. > > Przemko > > > -- > Przemko Tylzanowski Ph.D. > LSD & Joint, box 813-GREEN > O & N > University of Leuven > Herestraat 49 > 3000 Leuven > Belgium > > phone: (32-16)34-61-96 > fax : (32-16)34-62-00 > > > > Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm -- Joel B. Sheffield, Ph.D. Biology Department, Temple University 1900 North 12th Street Philadelphia, PA 19122 [hidden email] (215) 204 8839, fax (215) 204 0486 http://astro.temple.edu/~jbs |
Re: Deconvolve 3D plugin problem
|
|
In reply to this post by Przemko
Przemko,
When I wrote the documentation for Deconvolve 3D, I hoped that the plugin would replace the earlier 2D versions. This has not fully happened yet because the 3D code is not efficient with memory on 2D problems (it multiplies the problem size by 4 for no good reason in this case), and users have reported better results from Iterative Deconvolution 2 in some cases. I have updated the documentation page. Is your problem 2D or 3D? If it is 3D, then the 2D plugins do not apply, and I'm not sure how we got here. If it is 2D, then I don't understand the recent post about the number of slices in the PSF. Bob Robert P. Dougherty, Ph.D. President, OptiNav, Inc. Phone (425) 990-5912 Fax (425) 467-1119 www.optinav.com > "This plugin is intended to replace both the Iterative > Deconvolution and Iterative Deconvolution 2 plugins for 2D applications, > as well as extending the function to 3D" > |
|
|
OK, here we go.
First of all I noticed that when I use single image, my PSF was based on 46 slices (or so- the default value). I got it down to one (since I have only one image) and PRESTO, Deconvolve3D works! And it is fast, and it does not use a lot of memory. So, my conclusion here is that once you go to real 3D (so more than one image) the plugin falters. Other than that, it works without a crash. I used 3D since it said on the plugin page that 3D would supersede 2D. Now, when I copare the quality of the results, at this point 2D is vastly superior to 3D. This may of course be due to the fact that I have to play with the parameters a lot more. But, "just out of the box", 2D is better, BUT NOT FASTER. Guys, THANK YOU FOR THE HELP!!!! Przemko -- Przemko Tylzanowski Ph.D. LSD & Joint, box 813-GREEN O & N University of Leuven Herestraat 49 3000 Leuven Belgium phone: (32-16)34-61-96 fax : (32-16)34-62-00 Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm |
«
Return to ImageJ
|
1 view|%1 views
| Free forum by Nabble | Edit this page |