AVI re-opening problem
Locked
 
|
I am working with large (20 GB) AVI files. I save them with ImageJ as uncompressed AVI's.
But often when I open them later, only part of the file loads. For example, I saved an AVI with 5746 frames. I opened it, and only 970 frames opened. I have plenty of RAM. Any solution to this? Thank you, Frank Shaffer Research Engineer USDOE National Energy Technology Laboratory Office of Research & Development [hidden email] -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
We have the same problem opening large ND2 and CZI files.
========================================================================= Michael Cammer, Microscopy Core & Skirball Institute, NYU Langone Medical Center Cell: 914-309-3270 ** MY OFFICE HAS MOVED TO SKIRBALL 2nd FLOOR, Back right ** http://ocs.med.nyu.edu/microscopy & http://microscopynotes.com/ -----Original Message----- From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Shaffer, Franklin D. Sent: Thursday, September 24, 2015 11:52 AM To: [hidden email] Subject: AVI re-opening problem I am working with large (20 GB) AVI files. I save them with ImageJ as uncompressed AVI's. But often when I open them later, only part of the file loads. For example, I saved an AVI with 5746 frames. I opened it, and only 970 frames opened. I have plenty of RAM. Any solution to this? Thank you, Frank Shaffer Research Engineer USDOE National Energy Technology Laboratory Office of Research & Development [hidden email] -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html ------------------------------------------------------------ This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is proprietary, confidential, and exempt from disclosure under applicable law. Any unauthorized review, use, disclosure, or distribution is prohibited. If you have received this email in error please notify the sender by return email and delete the original message. Please note, the recipient should check this email and any attachments for the presence of viruses. The organization accepts no liability for any damage caused by any virus transmitted by this email. ================================= -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
Hi Franklin,
> I am working with large (20 GB) AVI files. I save them with ImageJ as > uncompressed AVI's. But often when I open them later, only part of the > file loads. For example, I saved an AVI with 5746 frames. I opened > it, and only 970 frames opened. This is a known limitation/bug of ImageJ 1.x's AVI Writer [1]. Did you try exporting your data using the Bio-Formats Exporter [2] plugin? Off the top of my head, I do not know whether it supports proper exporting of >2GB AVI files, but it is worth a shot. Regards, Curtis P.S. to Michael Cammer: problems with ND2 and CZI are unrelated; those formats are not AVI-based. [1] http://imagej.1557.x6.nabble.com/Not-able-to-open-full-length-of-large-AVI-file-tt4496180.html [2] http://imagej.net/Bio-Formats#Bio-Formats_Exporter On Thu, Sep 24, 2015 at 11:06 AM, Cammer, Michael < [hidden email]> wrote: > We have the same problem opening large ND2 and CZI files. > ========================================================================= > Michael Cammer, Microscopy Core & Skirball Institute, NYU Langone Medical > Center > Cell: 914-309-3270 ** MY OFFICE HAS MOVED TO > SKIRBALL 2nd FLOOR, Back right ** > http://ocs.med.nyu.edu/microscopy & http://microscopynotes.com/ > > > -----Original Message----- > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > Shaffer, Franklin D. > Sent: Thursday, September 24, 2015 11:52 AM > To: [hidden email] > Subject: AVI re-opening problem > > I am working with large (20 GB) AVI files. I save them with ImageJ as > uncompressed AVI's. > But often when I open them later, only part of the file loads. For > example, I saved an AVI with 5746 frames. I opened it, and only 970 frames > opened. > I have plenty of RAM. > > Any solution to this? > > Thank you, > Frank Shaffer > Research Engineer > USDOE National Energy Technology Laboratory Office of Research & > Development [hidden email] > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > ------------------------------------------------------------ > This email message, including any attachments, is for the sole use of the > intended recipient(s) and may contain information that is proprietary, > confidential, and exempt from disclosure under applicable law. Any > unauthorized review, use, disclosure, or distribution is prohibited. If you > have received this email in error please notify the sender by return email > and delete the original message. Please note, the recipient should check > this email and any attachments for the presence of viruses. The > organization accepts no liability for any damage caused by any virus > transmitted by this email. > ================================= > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: AVI re-opening problem
|
|
If I run the AVI through VirtualDub, then ImageJ will read the entire AVI.
Virtual Dub says something is wrong with the AVI indexing in the AVI's written from ImageJ. It's not a RAM limitation. I can get it to work, it's just an extra hassle to have to go through virtualDub everytime I want to open a large AVI. -----Original Message----- From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Curtis Rueden Sent: Thursday, September 24, 2015 12:48 PM To: [hidden email] Subject: Re: AVI re-opening problem Hi Franklin, > I am working with large (20 GB) AVI files. I save them with ImageJ as > uncompressed AVI's. But often when I open them later, only part of the > file loads. For example, I saved an AVI with 5746 frames. I opened > it, and only 970 frames opened. This is a known limitation/bug of ImageJ 1.x's AVI Writer [1]. Did you try exporting your data using the Bio-Formats Exporter [2] plugin? Off the top of my head, I do not know whether it supports proper exporting of >2GB AVI files, but it is worth a shot. Regards, Curtis P.S. to Michael Cammer: problems with ND2 and CZI are unrelated; those formats are not AVI-based. [1] http://imagej.1557.x6.nabble.com/Not-able-to-open-full-length-of-large-AVI-file-tt4496180.html [2] http://imagej.net/Bio-Formats#Bio-Formats_Exporter On Thu, Sep 24, 2015 at 11:06 AM, Cammer, Michael < [hidden email]> wrote: > We have the same problem opening large ND2 and CZI files. > ====================================================================== > === Michael Cammer, Microscopy Core & Skirball Institute, NYU Langone > Medical Center > Cell: 914-309-3270 ** MY OFFICE HAS MOVED TO > SKIRBALL 2nd FLOOR, Back right ** > http://ocs.med.nyu.edu/microscopy & > http://microscopynotes.com/ > > > -----Original Message----- > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > Shaffer, Franklin D. > Sent: Thursday, September 24, 2015 11:52 AM > To: [hidden email] > Subject: AVI re-opening problem > > I am working with large (20 GB) AVI files. I save them with ImageJ as > uncompressed AVI's. > But often when I open them later, only part of the file loads. For > example, I saved an AVI with 5746 frames. I opened it, and only 970 > frames opened. > I have plenty of RAM. > > Any solution to this? > > Thank you, > Frank Shaffer > Research Engineer > USDOE National Energy Technology Laboratory Office of Research & > Development [hidden email] > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > ------------------------------------------------------------ > This email message, including any attachments, is for the sole use of > the intended recipient(s) and may contain information that is > proprietary, confidential, and exempt from disclosure under applicable > law. Any unauthorized review, use, disclosure, or distribution is > prohibited. If you have received this email in error please notify the > sender by return email and delete the original message. Please note, > the recipient should check this email and any attachments for the > presence of viruses. The organization accepts no liability for any > damage caused by any virus transmitted by this email. > ================================= > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Frank,
> I can get it to work, it's just an extra hassle to have to go through > virtualDub everytime I want to open a large AVI. I filed an issue for it in the ImageJ 1.x issue tracker on GitHub: https://github.com/imagej/imagej1/issues/11 Regards, Curtis On Thu, Sep 24, 2015 at 12:06 PM, Shaffer, Franklin D. < [hidden email]> wrote: > If I run the AVI through VirtualDub, then ImageJ will read the entire AVI. > > Virtual Dub says something is wrong with the AVI indexing in the AVI's > written from ImageJ. It's not a RAM limitation. > > I can get it to work, it's just an extra hassle to have to go through > virtualDub everytime I want to open a large AVI. > > > -----Original Message----- > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > Curtis Rueden > Sent: Thursday, September 24, 2015 12:48 PM > To: [hidden email] > Subject: Re: AVI re-opening problem > > Hi Franklin, > > > I am working with large (20 GB) AVI files. I save them with ImageJ as > > uncompressed AVI's. But often when I open them later, only part of the > > file loads. For example, I saved an AVI with 5746 frames. I opened > > it, and only 970 frames opened. > > This is a known limitation/bug of ImageJ 1.x's AVI Writer [1]. > > Did you try exporting your data using the Bio-Formats Exporter [2] plugin? > Off the top of my head, I do not know whether it supports proper exporting > of >2GB AVI files, but it is worth a shot. > > Regards, > Curtis > > P.S. to Michael Cammer: problems with ND2 and CZI are unrelated; those > formats are not AVI-based. > > [1] > > http://imagej.1557.x6.nabble.com/Not-able-to-open-full-length-of-large-AVI-file-tt4496180.html > [2] http://imagej.net/Bio-Formats#Bio-Formats_Exporter > > On Thu, Sep 24, 2015 at 11:06 AM, Cammer, Michael < > [hidden email]> wrote: > > > We have the same problem opening large ND2 and CZI files. > > ====================================================================== > > === Michael Cammer, Microscopy Core & Skirball Institute, NYU Langone > > Medical Center > > Cell: 914-309-3270 ** MY OFFICE HAS MOVED TO > > SKIRBALL 2nd FLOOR, Back right ** > > http://ocs.med.nyu.edu/microscopy & > > http://microscopynotes.com/ > > > > > > -----Original Message----- > > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > > Shaffer, Franklin D. > > Sent: Thursday, September 24, 2015 11:52 AM > > To: [hidden email] > > Subject: AVI re-opening problem > > > > I am working with large (20 GB) AVI files. I save them with ImageJ as > > uncompressed AVI's. > > But often when I open them later, only part of the file loads. For > > example, I saved an AVI with 5746 frames. I opened it, and only 970 > > frames opened. > > I have plenty of RAM. > > > > Any solution to this? > > > > Thank you, > > Frank Shaffer > > Research Engineer > > USDOE National Energy Technology Laboratory Office of Research & > > Development [hidden email] > > > > > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > ------------------------------------------------------------ > > This email message, including any attachments, is for the sole use of > > the intended recipient(s) and may contain information that is > > proprietary, confidential, and exempt from disclosure under applicable > > law. Any unauthorized review, use, disclosure, or distribution is > > prohibited. If you have received this email in error please notify the > > sender by return email and delete the original message. Please note, > > the recipient should check this email and any attachments for the > > presence of viruses. The organization accepts no liability for any > > damage caused by any virus transmitted by this email. > > ================================= > > > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: AVI re-opening problem
|
|
In reply to this post by Franklin Shaffer-2
> On Sep 24, 2015, at 11:52 AM, Shaffer, Franklin D. <[hidden email]> wrote:
> > I am working with large (20 GB) AVI files. I save them with ImageJ as uncompressed AVI's. > But often when I open them later, only part of the file loads. For example, I saved an AVI with 5746 frames. I opened it, and only 970 frames opened. > I have plenty of RAM. > > Any solution to this? Save in TIFF format. Other TIFF readers will not be able to open a 20 GB TIFF created by ImageJ, except as a “raw” file using information displayed by ImageJ when the stack is saved, for example: Stack is larger than 4GB. Most TIFF readers will only open the first image. Use this information to open as raw: name=Untitled, dir=, width=1024, height=1024, nImages=5000, offset=183, type=byte, byteOrder=big, format=0, url=, whiteIsZero=f, lutSize=256, comp=1, samples=1 -wayne -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
In reply to this post by Franklin Shaffer-2
Hi Frank,
you don't write about this, but I presume that not only the RAM but also the memory assigned to ImageJ (Edit>Option>Memory&Threads) is enough to hold all images? How much free memory does ImageJ report when reading the movie has stopped? (Click on the status line or look at Help>About ImageJ) If memory is not the problem: Could you switch debug mode on (Edit>Options>Misc) and report (a) at the beginning of the Log with all the parameters of the movie, and (b) the last lines of the log, with the information about the last slices read? Does it work when opening the AVI as a virtual stack? Michael ________________________________________________________________ On Sep 24, 2015, at 17:52, Shaffer, Franklin D. wrote: > I am working with large (20 GB) AVI files. I save them with ImageJ as uncompressed AVI's. > But often when I open them later, only part of the file loads. For example, I saved an AVI with 5746 frames. I opened it, and only 970 frames opened. > I have plenty of RAM. > > Any solution to this? > > Thank you, > Frank Shaffer > Research Engineer > USDOE National Energy Technology Laboratory > Office of Research & Development > [hidden email] > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: AVI re-opening problem
|
|
Michael,
Here's a test I did based on your request: 1. I set ImageJ memory at 80 GB 2. I opened an AVI file with size of 6.9 GB. I had saved it with ImageJ as AVI uncompressed. 3. The file opened to 1063 of 2501 frames and stopped. So ImageJ opened 2.9 GB of the 6.9 GB AVI file. 4. In Help-->About ImageJ it says it has used "3051 of 80000 MB" 5. I then closed the AVI file and turned on debug mode. 6. Opened the AVI file again, same thing happened. The entire ImageJ log is pasted below. 7. Opened the AVI in virtualDub. It opened but had to be repaired. Here's what virtualDub said: " [!] AVI: Index not found or damaged -- reconstructing via file scan. [!] AVI: Keyframe flag reconstruction was not specified in open options and the video stream is not a known keyframe-only type. Seeking in the video stream may be extremely slow." Thanks, Frank The ImageJ log: setKeyUp: -1 actionPerformed: time=1443198565264, java.awt.event.ActionEvent[ACTION_PERFORMED,cmd=Open...,when=1443198565264,modifiers=] on menuitem0 runPlugIn: ij.plugin.Commands("open") openImage: "ojj", E:\OHMSETT\Oct29\proc'd_cines\Test_19_3in_OIL_419GPM_500fs\img_seq\Test_19_3in_OIL_419GPM_500fs.avi OPEN AND READ AVI FILE HEADER (t=0 ms) File header: File type='RIFF' (should be 'RIFF') (t=0 ms) File header: RIFF type='AVI ' (should be 'AVI ') Searching for 'hdrl', found hdrl' 0x14-0x4eb (1240 Bytes) Searching for 'avih', found avih' 0x20-0x57 (56 Bytes) AVI HEADER (avih): (t=10 ms) dwMicroSecPerFrame=142857 dwMaxBytesPerSec=0 dwReserved1=0 dwFlags=16 dwTotalFrames=2501 dwInitialFrames=0 dwStreams=1 dwSuggestedBufferSize=0 dwWidth=2494 setKeyUp: 10 dwHeight=1196 Searching for 'strl', found strl' 0x60-0x4eb (1164 Bytes) Searching for 'strh', found strh' 0x6c-0xa3 (56 Bytes) VIDEO STREAM HEADER (strh): fccStreamHandler='DIB ' dwStreamFlags=0 wPriority,wLanguage=0 dwStreamInitialFrames=0 dwStreamScale=1 dwStreamRate=7 dwStreamStart=0 dwStreamLength=2501 dwStreamSuggestedBufferSize=0 dwStreamQuality=-1 dwStreamSampleSize=0 Searching for 'strf', found strf' 0xac-0x4d3 (1064 Bytes) biSize=40 biWidth=2494 biHeight=1196 biPlanes=1 biBitCount=8 biCompression=0x0 ' Searching for 'indx', found strn' 0x4dc-0x4eb (16 Bytes) Discarded 'strn': Contents does not fit GUI.getMaxWindowBounds: java.awt.Rectangle[x=0,y=0,width=1600,height=1172] Skip JUNK: 0x4f4-0xfff (2828 Bytes) Searching for 'movi', found movi' 0x1008-0xbd02df33 (3171077932 Bytes) MOVIE DATA 0x100c-0xbd02df33 (3171077928 Bytes) (t=100 ms) Searching for stream 0: '00db' or '00dc' chunks 1 movie data '00db' 0x1014-0x2d9d13 (2985216 Bytes) (t=100 ms) . . . 1063 movie data '00db' 0xbcf71f44-0xbd24ac43 (2985216 Bytes) (t=9860 ms) AVI HEADER (avih): (t=10 ms) dwMicroSecPerFrame=142857 dwMaxBytesPerSec=0 dwReserved1=0 dwFlags=16 dwTotalFrames=2501 dwInitialFrames=0 dwStreams=1 dwSuggestedBufferSize=0 dwWidth=2494 setKeyUp: 10 dwHeight=1196 Searching for 'strl', found strl' 0x60-0x4eb (1164 Bytes) Searching for 'strh', found strh' 0x6c-0xa3 (56 Bytes) VIDEO STREAM HEADER (strh): fccStreamHandler='DIB ' dwStreamFlags=0 wPriority,wLanguage=0 dwStreamInitialFrames=0 dwStreamScale=1 dwStreamRate=7 dwStreamStart=0 dwStreamLength=2501 dwStreamSuggestedBufferSize=0 dwStreamQuality=-1 dwStreamSampleSize=0 Searching for 'strf', found strf' 0xac-0x4d3 (1064 Bytes) biSize=40 biWidth=2494 biHeight=1196 biPlanes=1 biBitCount=8 biCompression=0x0 ' -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Frank,
looking at the code, I can confirm that this is a problem of the AVI_Writer of ImageJ. Here are a few more details on what happens: If the size of the data (plus a small bit of overhead) is above 2 GB, but below 4 GB, it creates an AVI file that does not strictly adhere to the standard, but is still readable by many programs (including ImageJ itself). If it's above 4 GB, it does not work any more. That's because the AVI_Writer still writes everything into a single 'movi' chunk, which has a 32-bit integer for the length of the 'movi' chunk. The consequences of a file size > 4GB, when reading the AVI file: (1) Jumping over the 'movi' chunk to find the index 'idx1' does not work (the AVI-type 1 index must be at the end), and (2) When reading the frames sequentially, ImageJ stops when it reaches the length modulo 4 GB, because it only reads up to the length reported in the Workaround: For an uncompressed file size of 6.9 GB, it will usually help to use 'png' compression when saving (this is lossless for 8-bit and RGB images). If the data are not too noisy, it will reduce the file size to less than 4 GB. But it will be slower and won't help for much larger movies. In my view, the only clean solution, adding AVI 2 capabilities to the the AVI_Writer, requires some work. I'll have a look at this, but I can't promise anything. Michael ________________________________________________________________ On Sep 25, 2015, at 18:41, Shaffer, Franklin D. wrote: > Michael, > Here's a test I did based on your request: > > 1. I set ImageJ memory at 80 GB > 2. I opened an AVI file with size of 6.9 GB. I had saved it with ImageJ as AVI uncompressed. > 3. The file opened to 1063 of 2501 frames and stopped. So ImageJ opened 2.9 GB of the 6.9 GB AVI file. > 4. In Help-->About ImageJ it says it has used "3051 of 80000 MB" > 5. I then closed the AVI file and turned on debug mode. > 6. Opened the AVI file again, same thing happened. The entire ImageJ log is pasted below. > 7. Opened the AVI in virtualDub. It opened but had to be repaired. Here's what virtualDub said: > " [!] AVI: Index not found or damaged -- reconstructing via file scan. > [!] AVI: Keyframe flag reconstruction was not specified in open options and > the video stream is not a known keyframe-only type. Seeking in the video > stream may be extremely slow." > > Thanks, Frank > > The ImageJ log: > setKeyUp: -1 > actionPerformed: time=1443198565264, java.awt.event.ActionEvent[ACTION_PERFORMED,cmd=Open...,when=1443198565264,modifiers=] on menuitem0 > runPlugIn: ij.plugin.Commands("open") > openImage: "ojj", E:\OHMSETT\Oct29\proc'd_cines\Test_19_3in_OIL_419GPM_500fs\img_seq\Test_19_3in_OIL_419GPM_500fs.avi > OPEN AND READ AVI FILE HEADER (t=0 ms) > File header: File type='RIFF' (should be 'RIFF') (t=0 ms) > File header: RIFF type='AVI ' (should be 'AVI ') > Searching for 'hdrl', found hdrl' 0x14-0x4eb (1240 Bytes) > Searching for 'avih', found avih' 0x20-0x57 (56 Bytes) > AVI HEADER (avih): (t=10 ms) > dwMicroSecPerFrame=142857 > dwMaxBytesPerSec=0 > dwReserved1=0 > dwFlags=16 > dwTotalFrames=2501 > dwInitialFrames=0 > dwStreams=1 > dwSuggestedBufferSize=0 > dwWidth=2494 > setKeyUp: 10 > dwHeight=1196 > Searching for 'strl', found strl' 0x60-0x4eb (1164 Bytes) > Searching for 'strh', found strh' 0x6c-0xa3 (56 Bytes) > VIDEO STREAM HEADER (strh): > fccStreamHandler='DIB ' > dwStreamFlags=0 > wPriority,wLanguage=0 > dwStreamInitialFrames=0 > dwStreamScale=1 > dwStreamRate=7 > dwStreamStart=0 > dwStreamLength=2501 > dwStreamSuggestedBufferSize=0 > dwStreamQuality=-1 > dwStreamSampleSize=0 > Searching for 'strf', found strf' 0xac-0x4d3 (1064 Bytes) > biSize=40 > biWidth=2494 > biHeight=1196 > biPlanes=1 > biBitCount=8 > biCompression=0x0 ' > Searching for 'indx', found strn' 0x4dc-0x4eb (16 Bytes) > Discarded 'strn': Contents does not fit > GUI.getMaxWindowBounds: java.awt.Rectangle[x=0,y=0,width=1600,height=1172] > Skip JUNK: 0x4f4-0xfff (2828 Bytes) > Searching for 'movi', found movi' 0x1008-0xbd02df33 (3171077932 Bytes) > MOVIE DATA 0x100c-0xbd02df33 (3171077928 Bytes) (t=100 ms) > Searching for stream 0: '00db' or '00dc' chunks > 1 movie data '00db' 0x1014-0x2d9d13 (2985216 Bytes) (t=100 ms) > . > . > . > 1063 movie data '00db' 0xbcf71f44-0xbd24ac43 (2985216 Bytes) (t=9860 ms) > AVI HEADER (avih): (t=10 ms) > dwMicroSecPerFrame=142857 > dwMaxBytesPerSec=0 > dwReserved1=0 > dwFlags=16 > dwTotalFrames=2501 > dwInitialFrames=0 > dwStreams=1 > dwSuggestedBufferSize=0 > dwWidth=2494 > setKeyUp: 10 > dwHeight=1196 > Searching for 'strl', found strl' 0x60-0x4eb (1164 Bytes) > Searching for 'strh', found strh' 0x6c-0xa3 (56 Bytes) > VIDEO STREAM HEADER (strh): > fccStreamHandler='DIB ' > dwStreamFlags=0 > wPriority,wLanguage=0 > dwStreamInitialFrames=0 > dwStreamScale=1 > dwStreamRate=7 > dwStreamStart=0 > dwStreamLength=2501 > dwStreamSuggestedBufferSize=0 > dwStreamQuality=-1 > dwStreamSampleSize=0 > Searching for 'strf', found strf' 0xac-0x4d3 (1064 Bytes) > biSize=40 > biWidth=2494 > biHeight=1196 > biPlanes=1 > biBitCount=8 > biCompression=0x0 ' > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: AVI re-opening problem
|
|
Michael,
Thank you for looking into this and explaining. For legal reasons, I must use a completely uncompressed format. I can work around this by using TIFF sequences until I must go to uncompressed AVI. Then I can use virtualDub to repair the AVI so ImageJ will read the entire AVI. You all will have to decide where this ranks in your priority of work to do on ImageJ. ImageJ is an excellent tool. It's a great example of federally funded R&D. It's also a great example of a community of researchers working together from all over the world. Frank -----Original Message----- From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Michael Schmid Sent: Friday, September 25, 2015 3:10 PM To: [hidden email] Subject: Re: AVI re-opening problem Hi Frank, looking at the code, I can confirm that this is a problem of the AVI_Writer of ImageJ. Here are a few more details on what happens: If the size of the data (plus a small bit of overhead) is above 2 GB, but below 4 GB, it creates an AVI file that does not strictly adhere to the standard, but is still readable by many programs (including ImageJ itself). If it's above 4 GB, it does not work any more. That's because the AVI_Writer still writes everything into a single 'movi' chunk, which has a 32-bit integer for the length of the 'movi' chunk. The consequences of a file size > 4GB, when reading the AVI file: (1) Jumping over the 'movi' chunk to find the index 'idx1' does not work (the AVI-type 1 index must be at the end), and (2) When reading the frames sequentially, ImageJ stops when it reaches the length modulo 4 GB, because it only reads up to the length reported in the Workaround: For an uncompressed file size of 6.9 GB, it will usually help to use 'png' compression when saving (this is lossless for 8-bit and RGB images). If the data are not too noisy, it will reduce the file size to less than 4 GB. But it will be slower and won't help for much larger movies. In my view, the only clean solution, adding AVI 2 capabilities to the the AVI_Writer, requires some work. I'll have a look at this, but I can't promise anything. Michael ________________________________________________________________ On Sep 25, 2015, at 18:41, Shaffer, Franklin D. wrote: > Michael, > Here's a test I did based on your request: > > 1. I set ImageJ memory at 80 GB > 2. I opened an AVI file with size of 6.9 GB. I had saved it with ImageJ as AVI uncompressed. > 3. The file opened to 1063 of 2501 frames and stopped. So ImageJ opened 2.9 GB of the 6.9 GB AVI file. > 4. In Help-->About ImageJ it says it has used "3051 of 80000 MB" > 5. I then closed the AVI file and turned on debug mode. > 6. Opened the AVI file again, same thing happened. The entire ImageJ log is pasted below. > 7. Opened the AVI in virtualDub. It opened but had to be repaired. Here's what virtualDub said: > " [!] AVI: Index not found or damaged -- reconstructing via file scan. > [!] AVI: Keyframe flag reconstruction was not specified in open options and > the video stream is not a known keyframe-only type. Seeking in the video > stream may be extremely slow." > > Thanks, Frank > > The ImageJ log: > setKeyUp: -1 > actionPerformed: time=1443198565264, > java.awt.event.ActionEvent[ACTION_PERFORMED,cmd=Open...,when=144319856 > 5264,modifiers=] on menuitem0 > runPlugIn: ij.plugin.Commands("open") > openImage: "ojj", > E:\OHMSETT\Oct29\proc'd_cines\Test_19_3in_OIL_419GPM_500fs\img_seq\Tes > t_19_3in_OIL_419GPM_500fs.avi OPEN AND READ AVI FILE HEADER (t=0 ms) > File header: File type='RIFF' (should be 'RIFF') (t=0 ms) File header: > RIFF type='AVI ' (should be 'AVI ') Searching for 'hdrl', found hdrl' > 0x14-0x4eb (1240 Bytes) Searching for 'avih', found avih' 0x20-0x57 > (56 Bytes) AVI HEADER (avih): (t=10 ms) > dwMicroSecPerFrame=142857 > dwMaxBytesPerSec=0 > dwReserved1=0 > dwFlags=16 > dwTotalFrames=2501 > dwInitialFrames=0 > dwStreams=1 > dwSuggestedBufferSize=0 > dwWidth=2494 > setKeyUp: 10 > dwHeight=1196 > Searching for 'strl', found strl' 0x60-0x4eb (1164 Bytes) Searching > for 'strh', found strh' 0x6c-0xa3 (56 Bytes) VIDEO STREAM HEADER > (strh): > fccStreamHandler='DIB ' > dwStreamFlags=0 > wPriority,wLanguage=0 > dwStreamInitialFrames=0 > dwStreamScale=1 > dwStreamRate=7 > dwStreamStart=0 > dwStreamLength=2501 > dwStreamSuggestedBufferSize=0 > dwStreamQuality=-1 > dwStreamSampleSize=0 > Searching for 'strf', found strf' 0xac-0x4d3 (1064 Bytes) > biSize=40 > biWidth=2494 > biHeight=1196 > biPlanes=1 > biBitCount=8 > biCompression=0x0 ' > Searching for 'indx', found strn' 0x4dc-0x4eb (16 Bytes) Discarded > 'strn': Contents does not fit > GUI.getMaxWindowBounds: > java.awt.Rectangle[x=0,y=0,width=1600,height=1172] > Skip JUNK: 0x4f4-0xfff (2828 Bytes) > Searching for 'movi', found movi' 0x1008-0xbd02df33 (3171077932 Bytes) > MOVIE DATA 0x100c-0xbd02df33 (3171077928 Bytes) (t=100 ms) Searching > for stream 0: '00db' or '00dc' chunks > 1 movie data '00db' 0x1014-0x2d9d13 (2985216 Bytes) (t=100 ms) . > . > . > 1063 movie data '00db' 0xbcf71f44-0xbd24ac43 (2985216 Bytes) (t=9860 > ms) AVI HEADER (avih): (t=10 ms) > dwMicroSecPerFrame=142857 > dwMaxBytesPerSec=0 > dwReserved1=0 > dwFlags=16 > dwTotalFrames=2501 > dwInitialFrames=0 > dwStreams=1 > dwSuggestedBufferSize=0 > dwWidth=2494 > setKeyUp: 10 > dwHeight=1196 > Searching for 'strl', found strl' 0x60-0x4eb (1164 Bytes) Searching > for 'strh', found strh' 0x6c-0xa3 (56 Bytes) VIDEO STREAM HEADER > (strh): > fccStreamHandler='DIB ' > dwStreamFlags=0 > wPriority,wLanguage=0 > dwStreamInitialFrames=0 > dwStreamScale=1 > dwStreamRate=7 > dwStreamStart=0 > dwStreamLength=2501 > dwStreamSuggestedBufferSize=0 > dwStreamQuality=-1 > dwStreamSampleSize=0 > Searching for 'strf', found strf' 0xac-0x4d3 (1064 Bytes) > biSize=40 > biWidth=2494 > biHeight=1196 > biPlanes=1 > biBitCount=8 > biCompression=0x0 ' > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
«
Return to ImageJ
|
1 view|%1 views
| Free forum by Nabble | Edit this page |