Fiji Bio-Formats is not opening large czi?
Locked
Fiji Bio-Formats is not opening large czi?
 
|
Hi,
I have a 85 GB CZI file (one out of a series of 40 files in the same folder) that I wanted to open with latest Fiji (updates from today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: Plugins > Bio-Formats > Bio-Formats Importer with options: - View with Hyperstack - Open files individually - Use virtual stack - all other options unchecked Fiji shows "Analyzing XXX" in the status bar, then nothing happens. I've waited 20 minutes now, then gave up. About Bio-Formats says "5.0.7 built 3.2.2015 revision 274796b". The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with the --debug option, but the command prompt console does not show any messages (to track down whats happening). Thanks for your help, and all the best, Mario -- Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch D-81669 München http://www.biodataanalysis.de/ -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
Hi Mario,
> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > I've waited 20 minutes now, then gave up. Did you try reading it with the Bio-Formats showinf command line tool? http://openmicroscopy.org/info/bio-formats/users/comlinetools/ Did you check out the "Debugging JVM hangs" section of the Debugging page? http://imagej.net/Debugging#Debugging_JVM_hangs Getting a stack trace will tell you more about what is going on -- i.e., what Bio-Formats is doing internally, whether it has crashed, etc. Did you try with the Bio-Formats update site enabled? This enables the 5.1.x development builds, which contain additional bug-fixes. http://fiji.sc/Bio-Formats#Daily_builds If this is a bug in Bio-Formats (if showinf also does not work, etc.), then please report this bug to the OME team. http://openmicroscopy.org/info/bio-formats/about/bug-reporting.html Regards, Curtis On Tue, Mar 3, 2015 at 8:45 AM, Mario Emmenlauer <[hidden email]> wrote: > Hi, > > I have a 85 GB CZI file (one out of a series of 40 files in the > same folder) that I wanted to open with latest Fiji (updates from > today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: > Plugins > Bio-Formats > Bio-Formats Importer > > with options: > - View with Hyperstack > - Open files individually > - Use virtual stack > - all other options unchecked > > Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > I've waited 20 minutes now, then gave up. About Bio-Formats says > "5.0.7 built 3.2.2015 revision 274796b". > > The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with > the --debug option, but the command prompt console does not show any > messages (to track down whats happening). > > Thanks for your help, and all the best, > > Mario > > > > > -- > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > D-81669 München http://www.biodataanalysis.de/ > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
In reply to this post by Mario Emmenlauer-3
Hi Mario,
Is this a Lightsheet file that's been separated by time (or color, view, etc.) so that you have a series of files in the format: filename.czi; filename(1).czi; filename(2).czi? If so, open the complete file (filename.czi) with Bioformats. You will then see a number of thumbnail images representing filename(1).czi, filename(2).czi, etc. Check the one you want to open. If you do not have enough RAM to open the file, make sure you check "Open in virtual Stack." This will take Fiji a long time to open (possibly minutes) as it has to read all of the header data. Even longer than ZEN. -Doug On Tue, Mar 3, 2015 at 9:45 AM, Mario Emmenlauer <[hidden email]> wrote: > Hi, > > I have a 85 GB CZI file (one out of a series of 40 files in the > same folder) that I wanted to open with latest Fiji (updates from > today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: > Plugins > Bio-Formats > Bio-Formats Importer > > with options: > - View with Hyperstack > - Open files individually > - Use virtual stack > - all other options unchecked > > Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > I've waited 20 minutes now, then gave up. About Bio-Formats says > "5.0.7 built 3.2.2015 revision 274796b". > > The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with > the --debug option, but the command prompt console does not show any > messages (to track down whats happening). > > Thanks for your help, and all the best, > > Mario > > > > > -- > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > D-81669 München http://www.biodataanalysis.de/ > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
Hi Mario,
I did not experience this problem using Bioformats so far, even for very long timelapses acquired with the Lightsheet Z.1. You can try my Multiview Reconstruction pipeline which is part of Fiji (http://fiji.sc/Multiview-Reconstruction <http://fiji.sc/Multiview-Reconstruction>) to test it. Just 1) Define a new XML dataset (Plugins > Multiview Reconstruction > Define Multi-View Dataset). Press OK, then choose the first filename.CZI (the one without (1), (2), etc.) 2) It will show all the metadata and you should just have to click OK. 3) Then, for example display one of the views (Plugins > Multiview Reconstruction > Tools > Display View) You can also resave the entire or parts of the dataset as TIFF or HDF5 (Plugins > Multiview Reconstruction > Resave > As TIFF). It also supports registration, fusion, deconvolution of the dataset! Hope this helps, Stephan --- Dr. Stephan Preibisch HFSP Fellow Robert H. Singer / Eugene Myers lab Albert Einstein College of Medicine / HHMI Janelia Farm / MPI-CBG email: [hidden email] <mailto:[hidden email]> / [hidden email] <mailto:[hidden email]> / [hidden email] <mailto:[hidden email]> web: http://www.preibisch.net/ <http://fly.mpi-cbg.de/preibisch> > On Mar 3, 2015, at 11:49 , Douglas Richardson <[hidden email]> wrote: > > Hi Mario, > > Is this a Lightsheet file that's been separated by time (or color, view, > etc.) so that you have a series of files in the format: filename.czi; > filename(1).czi; filename(2).czi? If so, open the complete file > (filename.czi) with Bioformats. You will then see a number of thumbnail > images representing filename(1).czi, filename(2).czi, etc. Check the one > you want to open. If you do not have enough RAM to open the file, make > sure you check "Open in virtual Stack." > > This will take Fiji a long time to open (possibly minutes) as it has to > read all of the header data. Even longer than ZEN. > > -Doug > > > On Tue, Mar 3, 2015 at 9:45 AM, Mario Emmenlauer <[hidden email]> > wrote: > >> Hi, >> >> I have a 85 GB CZI file (one out of a series of 40 files in the >> same folder) that I wanted to open with latest Fiji (updates from >> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: >> Plugins > Bio-Formats > Bio-Formats Importer >> >> with options: >> - View with Hyperstack >> - Open files individually >> - Use virtual stack >> - all other options unchecked >> >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >> I've waited 20 minutes now, then gave up. About Bio-Formats says >> "5.0.7 built 3.2.2015 revision 274796b". >> >> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with >> the --debug option, but the command prompt console does not show any >> messages (to track down whats happening). >> >> Thanks for your help, and all the best, >> >> Mario >> >> >> >> >> -- >> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >> D-81669 München http://www.biodataanalysis.de/ >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
In reply to this post by Douglas Richardson
Hi,
On 03.03.2015 17:49, Douglas Richardson wrote: > Is this a Lightsheet file that's been separated by time (or color, view, > etc.) so that you have a series of files in the format: filename.czi; > filename(1).czi; filename(2).czi? If so, open the complete file > (filename.czi) with Bioformats. You will then see a number of thumbnail > images representing filename(1).czi, filename(2).czi, etc. Check the one > you want to open. If you do not have enough RAM to open the file, make > sure you check "Open in virtual Stack." Yes, its a Lightsheet file that has been separated by time. I tried to open the complete file drlGFPlmo2Red7ss.czi with Bio-Formats, and nothing shows up, even after many minutes (no thumnails, no popup, no status message, nothing). Its as if I never tried "open" at all. Fiji just stands there, idle... Could it be that opening files from a (very fast fiber-connected) network share (mounted as drive letter L:) is not supported? All the best, Mario > This will take Fiji a long time to open (possibly minutes) as it has to > read all of the header data. Even longer than ZEN. > > -Doug > > > On Tue, Mar 3, 2015 at 9:45 AM, Mario Emmenlauer <[hidden email]> > wrote: > >> Hi, >> >> I have a 85 GB CZI file (one out of a series of 40 files in the >> same folder) that I wanted to open with latest Fiji (updates from >> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: >> Plugins > Bio-Formats > Bio-Formats Importer >> >> with options: >> - View with Hyperstack >> - Open files individually >> - Use virtual stack >> - all other options unchecked >> >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >> I've waited 20 minutes now, then gave up. About Bio-Formats says >> "5.0.7 built 3.2.2015 revision 274796b". >> >> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with >> the --debug option, but the command prompt console does not show any >> messages (to track down whats happening). >> >> Thanks for your help, and all the best, >> >> Mario >> >> >> >> >> -- >> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >> D-81669 München http://www.biodataanalysis.de/ >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email? Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch D-81669 München http://www.biodataanalysis.de/ -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
In reply to this post by Stephan Preibisch
Hi,
On 03.03.2015 18:45, Stephan Preibisch wrote: > I did not experience this problem using Bioformats so far, even for very long timelapses acquired with the Lightsheet Z.1. > > You can try my Multiview Reconstruction pipeline which is part of Fiji (http://fiji.sc/Multiview-Reconstruction <http://fiji.sc/Multiview-Reconstruction>) to test it. Just > > 1) Define a new XML dataset (Plugins > Multiview Reconstruction > Define Multi-View Dataset). Press OK, then choose the first filename.CZI (the one without (1), (2), etc.) > 2) It will show all the metadata and you should just have to click OK. > 3) Then, for example display one of the views (Plugins > Multiview Reconstruction > Tools > Display View) I tried this, also this seems not to work. Here is what I see: 1) Define a new XML dataset (Plugins > Multiview Reconstruction > Define Multi-View Dataset). Press OK, then choose drlGFPlmo2Red7ss.czi 2) It shows in the log window: 2 Investigating file 'L:\MarioEmmenlauer\Lightsheet_RAW\drlGFPlmo2Red7ss.czi'. No output comes after that in a very long time. All the best, Mario > You can also resave the entire or parts of the dataset as TIFF or HDF5 (Plugins > Multiview Reconstruction > Resave > As TIFF). It also supports registration, fusion, deconvolution of the dataset! > > Hope this helps, > Stephan > --- > > Dr. Stephan Preibisch > HFSP Fellow > Robert H. Singer / Eugene Myers lab > > Albert Einstein College of Medicine / HHMI Janelia Farm / MPI-CBG > > email: [hidden email] <mailto:[hidden email]> / [hidden email] <mailto:[hidden email]> / [hidden email] <mailto:[hidden email]> > web: http://www.preibisch.net/ <http://fly.mpi-cbg.de/preibisch> >> On Mar 3, 2015, at 11:49 , Douglas Richardson <[hidden email]> wrote: >> >> Hi Mario, >> >> Is this a Lightsheet file that's been separated by time (or color, view, >> etc.) so that you have a series of files in the format: filename.czi; >> filename(1).czi; filename(2).czi? If so, open the complete file >> (filename.czi) with Bioformats. You will then see a number of thumbnail >> images representing filename(1).czi, filename(2).czi, etc. Check the one >> you want to open. If you do not have enough RAM to open the file, make >> sure you check "Open in virtual Stack." >> >> This will take Fiji a long time to open (possibly minutes) as it has to >> read all of the header data. Even longer than ZEN. >> >> -Doug >> >> >> On Tue, Mar 3, 2015 at 9:45 AM, Mario Emmenlauer <[hidden email]> >> wrote: >> >>> Hi, >>> >>> I have a 85 GB CZI file (one out of a series of 40 files in the >>> same folder) that I wanted to open with latest Fiji (updates from >>> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: >>> Plugins > Bio-Formats > Bio-Formats Importer >>> >>> with options: >>> - View with Hyperstack >>> - Open files individually >>> - Use virtual stack >>> - all other options unchecked >>> >>> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >>> I've waited 20 minutes now, then gave up. About Bio-Formats says >>> "5.0.7 built 3.2.2015 revision 274796b". >>> >>> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with >>> the --debug option, but the command prompt console does not show any >>> messages (to track down whats happening). >>> >>> Thanks for your help, and all the best, >>> >>> Mario >>> >>> >>> >>> >>> -- >>> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >>> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >>> D-81669 München http://www.biodataanalysis.de/ >>> >>> -- >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>> >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email? Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch D-81669 München http://www.biodataanalysis.de/ -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
In reply to this post by Mario Emmenlauer-3
Yes, this might be the issue … over the network it might be much slower from my experience.
Cheers, Stephan > On Mar 4, 2015, at 7:28 , Mario Emmenlauer <[hidden email]> wrote: > > Hi, > > On 03.03.2015 17:49, Douglas Richardson wrote: >> Is this a Lightsheet file that's been separated by time (or color, view, >> etc.) so that you have a series of files in the format: filename.czi; >> filename(1).czi; filename(2).czi? If so, open the complete file >> (filename.czi) with Bioformats. You will then see a number of thumbnail >> images representing filename(1).czi, filename(2).czi, etc. Check the one >> you want to open. If you do not have enough RAM to open the file, make >> sure you check "Open in virtual Stack." > > Yes, its a Lightsheet file that has been separated by time. I tried > to open the complete file drlGFPlmo2Red7ss.czi with Bio-Formats, and > nothing shows up, even after many minutes (no thumnails, no popup, > no status message, nothing). Its as if I never tried "open" at all. > Fiji just stands there, idle... > > Could it be that opening files from a (very fast fiber-connected) > network share (mounted as drive letter L:) is not supported? > > All the best, > > Mario > > > >> This will take Fiji a long time to open (possibly minutes) as it has to >> read all of the header data. Even longer than ZEN. >> >> -Doug >> >> >> On Tue, Mar 3, 2015 at 9:45 AM, Mario Emmenlauer <[hidden email] <mailto:[hidden email]>> >> wrote: >> >>> Hi, >>> >>> I have a 85 GB CZI file (one out of a series of 40 files in the >>> same folder) that I wanted to open with latest Fiji (updates from >>> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: >>> Plugins > Bio-Formats > Bio-Formats Importer >>> >>> with options: >>> - View with Hyperstack >>> - Open files individually >>> - Use virtual stack >>> - all other options unchecked >>> >>> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >>> I've waited 20 minutes now, then gave up. About Bio-Formats says >>> "5.0.7 built 3.2.2015 revision 274796b". >>> >>> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with >>> the --debug option, but the command prompt console does not show any >>> messages (to track down whats happening). >>> >>> Thanks for your help, and all the best, >>> >>> Mario >>> >>> >>> >>> >>> -- >>> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >>> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >>> D-81669 München http://www.biodataanalysis.de/ >>> >>> -- >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>> >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> > > -- > A: Yes. >> Q: Are you sure? >>> A: Because it reverses the logical flow of conversation. >>>> Q: Why is top posting annoying in email? > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch <http://unibas.ch/> > D-81669 München http://www.biodataanalysis.de/ <http://www.biodataanalysis.de/> > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html <http://imagej.nih.gov/ij/list.html> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
In reply to this post by ctrueden
Hi Curtis,
On 03.03.2015 17:11, Curtis Rueden wrote: >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >> I've waited 20 minutes now, then gave up. > > Did you try reading it with the Bio-Formats showinf command line tool? > http://openmicroscopy.org/info/bio-formats/users/comlinetools/ I tried that now, and it seems Stephan Preibisch might be right that opening the files just takes forever. But why? The network mount is reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the file in ~30sec, and I can scroll through the timepoints and stacks at reasonable speed. I opened the czi with "Open files individually" and "Use virtual stack" in Fiji, and I hoped this might speed things up. So in Fiji, I can select the thumbnails finally after > 2hrs. Thats 25% of the time it would take to read *all* the files over the 100MB/s network. I can not think Bio-Formats would not read 25% of the 4TB data, just to show thumbnails? When using showinf, it prints first: Checking file format [Zeiss CZI] Initializing reader ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP lmo2Red 7ss .czi ... and then after around 1 hour, it fails with out-of-memory. Can it really take >2 hrs before I can see anything? Should the option "Open files individually" not limit it to the first timepoint? Cheers, Mario > Did you check out the "Debugging JVM hangs" section of the Debugging page? > http://imagej.net/Debugging#Debugging_JVM_hangs > > Getting a stack trace will tell you more about what is going on -- i.e., > what Bio-Formats is doing internally, whether it has crashed, etc. > > Did you try with the Bio-Formats update site enabled? This enables the > 5.1.x development builds, which contain additional bug-fixes. > http://fiji.sc/Bio-Formats#Daily_builds > > If this is a bug in Bio-Formats (if showinf also does not work, etc.), then > please report this bug to the OME team. > http://openmicroscopy.org/info/bio-formats/about/bug-reporting.html > > Regards, > Curtis > > On Tue, Mar 3, 2015 at 8:45 AM, Mario Emmenlauer <[hidden email]> > wrote: > >> Hi, >> >> I have a 85 GB CZI file (one out of a series of 40 files in the >> same folder) that I wanted to open with latest Fiji (updates from >> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: >> Plugins > Bio-Formats > Bio-Formats Importer >> >> with options: >> - View with Hyperstack >> - Open files individually >> - Use virtual stack >> - all other options unchecked >> >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >> I've waited 20 minutes now, then gave up. About Bio-Formats says >> "5.0.7 built 3.2.2015 revision 274796b". >> >> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with >> the --debug option, but the command prompt console does not show any >> messages (to track down whats happening). >> >> Thanks for your help, and all the best, >> >> Mario >> >> >> >> >> -- >> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >> D-81669 München http://www.biodataanalysis.de/ >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email? Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch D-81669 München http://www.biodataanalysis.de/ -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Mario,
> Stephan Preibisch might be right that opening the files just takes > forever. But why? The network mount is reasonably fast, it achieves > easily 100MB/s. There are multiple sorts of "fast" at work here: burst throughput, sustained throughput, latency, etc. If you really want to know why it is slow, then take a few stack traces while you are waiting for the files to be read. Just press Ctrl+\ while showinf is thinking. Cut and paste them from your console, saving them to files. Look for the string "loci.formats." and find out which Bio-Formats classes/method/lines are being hit. That will be 90% likely to answer your "why" question in this instance. Regards, Curtis On Wed, Mar 4, 2015 at 9:26 AM, Mario Emmenlauer <[hidden email]> wrote: > Hi Curtis, > > On 03.03.2015 17:11, Curtis Rueden wrote: > >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > >> I've waited 20 minutes now, then gave up. > > > > Did you try reading it with the Bio-Formats showinf command line tool? > > http://openmicroscopy.org/info/bio-formats/users/comlinetools/ > > I tried that now, and it seems Stephan Preibisch might be right that > opening the files just takes forever. But why? The network mount is > reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the > file in ~30sec, and I can scroll through the timepoints and stacks at > reasonable speed. I opened the czi with "Open files individually" and > "Use virtual stack" in Fiji, and I hoped this might speed things up. > > So in Fiji, I can select the thumbnails finally after > 2hrs. Thats > 25% of the time it would take to read *all* the files over the 100MB/s > network. I can not think Bio-Formats would not read 25% of the 4TB > data, just to show thumbnails? > > > When using showinf, it prints first: > Checking file format [Zeiss CZI] > Initializing reader > ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP > lmo2Red 7ss .czi > > ... and then after around 1 hour, it fails with out-of-memory. Can > it really take >2 hrs before I can see anything? Should the option > "Open files individually" not limit it to the first timepoint? > > Cheers, > > Mario > > > > > Did you check out the "Debugging JVM hangs" section of the Debugging > page? > > http://imagej.net/Debugging#Debugging_JVM_hangs > > > > Getting a stack trace will tell you more about what is going on -- i.e., > > what Bio-Formats is doing internally, whether it has crashed, etc. > > > > Did you try with the Bio-Formats update site enabled? This enables the > > 5.1.x development builds, which contain additional bug-fixes. > > http://fiji.sc/Bio-Formats#Daily_builds > > > > If this is a bug in Bio-Formats (if showinf also does not work, etc.), > then > > please report this bug to the OME team. > > http://openmicroscopy.org/info/bio-formats/about/bug-reporting.html > > > > Regards, > > Curtis > > > > On Tue, Mar 3, 2015 at 8:45 AM, Mario Emmenlauer <[hidden email]> > > wrote: > > > >> Hi, > >> > >> I have a 85 GB CZI file (one out of a series of 40 files in the > >> same folder) that I wanted to open with latest Fiji (updates from > >> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: > >> Plugins > Bio-Formats > Bio-Formats Importer > >> > >> with options: > >> - View with Hyperstack > >> - Open files individually > >> - Use virtual stack > >> - all other options unchecked > >> > >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > >> I've waited 20 minutes now, then gave up. About Bio-Formats says > >> "5.0.7 built 3.2.2015 revision 274796b". > >> > >> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with > >> the --debug option, but the command prompt console does not show any > >> messages (to track down whats happening). > >> > >> Thanks for your help, and all the best, > >> > >> Mario > >> > >> > >> > >> > >> -- > >> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > >> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > >> D-81669 München http://www.biodataanalysis.de/ > >> > >> -- > >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > >> > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > A: Yes. > > Q: Are you sure? > >> A: Because it reverses the logical flow of conversation. > >>> Q: Why is top posting annoying in email? > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > D-81669 München http://www.biodataanalysis.de/ > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
Slow file opening from a network drive via Bioformats is a know issue:
http://lists.openmicroscopy.org.uk/pipermail/ome-users/2014-December/004952.html Performance can be improved slightly by reducing the number of sub-directories your file in placed in. -Doug On Wed, Mar 4, 2015 at 12:50 PM, Curtis Rueden <[hidden email]> wrote: > Hi Mario, > > > Stephan Preibisch might be right that opening the files just takes > > forever. But why? The network mount is reasonably fast, it achieves > > easily 100MB/s. > > There are multiple sorts of "fast" at work here: burst throughput, > sustained throughput, latency, etc. > > If you really want to know why it is slow, then take a few stack traces > while you are waiting for the files to be read. Just press Ctrl+\ while > showinf is thinking. Cut and paste them from your console, saving them to > files. Look for the string "loci.formats." and find out which Bio-Formats > classes/method/lines are being hit. That will be 90% likely to answer your > "why" question in this instance. > > Regards, > Curtis > > On Wed, Mar 4, 2015 at 9:26 AM, Mario Emmenlauer <[hidden email]> > wrote: > > > Hi Curtis, > > > > On 03.03.2015 17:11, Curtis Rueden wrote: > > >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > > >> I've waited 20 minutes now, then gave up. > > > > > > Did you try reading it with the Bio-Formats showinf command line tool? > > > http://openmicroscopy.org/info/bio-formats/users/comlinetools/ > > > > I tried that now, and it seems Stephan Preibisch might be right that > > opening the files just takes forever. But why? The network mount is > > reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the > > file in ~30sec, and I can scroll through the timepoints and stacks at > > reasonable speed. I opened the czi with "Open files individually" and > > "Use virtual stack" in Fiji, and I hoped this might speed things up. > > > > So in Fiji, I can select the thumbnails finally after > 2hrs. Thats > > 25% of the time it would take to read *all* the files over the 100MB/s > > network. I can not think Bio-Formats would not read 25% of the 4TB > > data, just to show thumbnails? > > > > > > When using showinf, it prints first: > > Checking file format [Zeiss CZI] > > Initializing reader > > ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP > > lmo2Red 7ss .czi > > > > ... and then after around 1 hour, it fails with out-of-memory. Can > > it really take >2 hrs before I can see anything? Should the option > > "Open files individually" not limit it to the first timepoint? > > > > Cheers, > > > > Mario > > > > > > > > > Did you check out the "Debugging JVM hangs" section of the Debugging > > page? > > > http://imagej.net/Debugging#Debugging_JVM_hangs > > > > > > Getting a stack trace will tell you more about what is going on -- > i.e., > > > what Bio-Formats is doing internally, whether it has crashed, etc. > > > > > > Did you try with the Bio-Formats update site enabled? This enables the > > > 5.1.x development builds, which contain additional bug-fixes. > > > http://fiji.sc/Bio-Formats#Daily_builds > > > > > > If this is a bug in Bio-Formats (if showinf also does not work, etc.), > > then > > > please report this bug to the OME team. > > > http://openmicroscopy.org/info/bio-formats/about/bug-reporting.html > > > > > > Regards, > > > Curtis > > > > > > On Tue, Mar 3, 2015 at 8:45 AM, Mario Emmenlauer <[hidden email]> > > > wrote: > > > > > >> Hi, > > >> > > >> I have a 85 GB CZI file (one out of a series of 40 files in the > > >> same folder) that I wanted to open with latest Fiji (updates from > > >> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: > > >> Plugins > Bio-Formats > Bio-Formats Importer > > >> > > >> with options: > > >> - View with Hyperstack > > >> - Open files individually > > >> - Use virtual stack > > >> - all other options unchecked > > >> > > >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > > >> I've waited 20 minutes now, then gave up. About Bio-Formats says > > >> "5.0.7 built 3.2.2015 revision 274796b". > > >> > > >> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with > > >> the --debug option, but the command prompt console does not show any > > >> messages (to track down whats happening). > > >> > > >> Thanks for your help, and all the best, > > >> > > >> Mario > > >> > > >> > > >> > > >> > > >> -- > > >> Mario Emmenlauer BioDataAnalysis Mobil: > +49-(0)151-68108489 > > >> Balanstrasse 43 mailto: mario.emmenlauer * > unibas.ch > > >> D-81669 München > http://www.biodataanalysis.de/ > > >> > > >> -- > > >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > >> > > > > > > -- > > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > > > > -- > > A: Yes. > > > Q: Are you sure? > > >> A: Because it reverses the logical flow of conversation. > > >>> Q: Why is top posting annoying in email? > > > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > > D-81669 München http://www.biodataanalysis.de/ > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
In reply to this post by Mario Emmenlauer-3
Hi Mario,
Try moving one of those files to either your local file system or to another folder on the remote machine. The idea is to remove it from the place where the first .czi file lives. I think what takes a lot of time is the analysis of master .czi. Why is another question, probably best asked at the Bio-formats mailing list. All the best PAvel Why this happens is another question, probably > Hi Curtis, > > On 03.03.2015 17:11, Curtis Rueden wrote: >>> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >>> I've waited 20 minutes now, then gave up. >> >> Did you try reading it with the Bio-Formats showinf command line tool? >> http://openmicroscopy.org/info/bio-formats/users/comlinetools/ > > I tried that now, and it seems Stephan Preibisch might be right that > opening the files just takes forever. But why? The network mount is > reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the > file in ~30sec, and I can scroll through the timepoints and stacks at > reasonable speed. I opened the czi with "Open files individually" and > "Use virtual stack" in Fiji, and I hoped this might speed things up. > > So in Fiji, I can select the thumbnails finally after > 2hrs. Thats > 25% of the time it would take to read *all* the files over the 100MB/s > network. I can not think Bio-Formats would not read 25% of the 4TB > data, just to show thumbnails? > > > When using showinf, it prints first: > Checking file format [Zeiss CZI] > Initializing reader > ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP lmo2Red 7ss .czi > > ... and then after around 1 hour, it fails with out-of-memory. Can > it really take >2 hrs before I can see anything? Should the option > "Open files individually" not limit it to the first timepoint? > > Cheers, > > Mario > > > >> Did you check out the "Debugging JVM hangs" section of the Debugging page? >> http://imagej.net/Debugging#Debugging_JVM_hangs >> >> Getting a stack trace will tell you more about what is going on -- i.e., >> what Bio-Formats is doing internally, whether it has crashed, etc. >> >> Did you try with the Bio-Formats update site enabled? This enables the >> 5.1.x development builds, which contain additional bug-fixes. >> http://fiji.sc/Bio-Formats#Daily_builds >> >> If this is a bug in Bio-Formats (if showinf also does not work, etc.), then >> please report this bug to the OME team. >> http://openmicroscopy.org/info/bio-formats/about/bug-reporting.html >> >> Regards, >> Curtis >> >> On Tue, Mar 3, 2015 at 8:45 AM, Mario Emmenlauer <[hidden email]> >> wrote: >> >>> Hi, >>> >>> I have a 85 GB CZI file (one out of a series of 40 files in the >>> same folder) that I wanted to open with latest Fiji (updates from >>> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: >>> Plugins > Bio-Formats > Bio-Formats Importer >>> >>> with options: >>> - View with Hyperstack >>> - Open files individually >>> - Use virtual stack >>> - all other options unchecked >>> >>> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. >>> I've waited 20 minutes now, then gave up. About Bio-Formats says >>> "5.0.7 built 3.2.2015 revision 274796b". >>> >>> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with >>> the --debug option, but the command prompt console does not show any >>> messages (to track down whats happening). >>> >>> Thanks for your help, and all the best, >>> >>> Mario >>> >>> >>> >>> >>> -- >>> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >>> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >>> D-81669 München http://www.biodataanalysis.de/ >>> >>> -- >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>> >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> > > -- > A: Yes. >> Q: Are you sure? >>> A: Because it reverses the logical flow of conversation. >>>> Q: Why is top posting annoying in email? > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > D-81669 München http://www.biodataanalysis.de/ > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Fiji Bio-Formats is not opening large czi?
|
|
In reply to this post by Mario Emmenlauer-3
Hi Mario,
> I tried that now, and it seems Stephan Preibisch might be right that > opening the files just takes forever. But why? The network mount is > reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the > file in ~30sec, and I can scroll through the timepoints and stacks at > reasonable speed. I opened the czi with "Open files individually" and > "Use virtual stack" in Fiji, and I hoped this might speed things up. > > So in Fiji, I can select the thumbnails finally after > 2hrs. Thats > 25% of the time it would take to read *all* the files over the 100MB/s > network. I can not think Bio-Formats would not read 25% of the 4TB > data, just to show thumbnails? Reading large files over NFS on Windows is known to be slow in general. We are currently reviewing improvements to this for the 5.1.0 release of Bio-Formats later this month. Note though that these changes are not yet available through the Bio-Formats update site in Fiji. Please keep in mind that the initialization step isn't just showing thumbnails, it's also reading all of the metadata from the entire dataset and assembling offsets to all of the planes (to improve performance of reading images on demand). > When using showinf, it prints first: > Checking file format [Zeiss CZI] > Initializing reader > ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP lmo2Red 7ss .czi > > ... and then after around 1 hour, it fails with out-of-memory. Can > it really take >2 hrs before I can see anything? Should the option > "Open files individually" not limit it to the first timepoint? If you use the "Open files individually", then it's important to pick one of the files that ends with a number in parentheses. The "master" file (the one without that doesn't end with a number) typically doesn't contain pixel data, and is meant to indicate which files constitute the full dataset - so if you pick the master file, it's going to group all of the files together no matter what. Regards, -Melissa On Wed, Mar 04, 2015 at 04:26:01PM +0100, Mario Emmenlauer wrote: > Hi Curtis, > > On 03.03.2015 17:11, Curtis Rueden wrote: > >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > >> I've waited 20 minutes now, then gave up. > > > > Did you try reading it with the Bio-Formats showinf command line tool? > > http://openmicroscopy.org/info/bio-formats/users/comlinetools/ > > I tried that now, and it seems Stephan Preibisch might be right that > opening the files just takes forever. But why? The network mount is > reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the > file in ~30sec, and I can scroll through the timepoints and stacks at > reasonable speed. I opened the czi with "Open files individually" and > "Use virtual stack" in Fiji, and I hoped this might speed things up. > > So in Fiji, I can select the thumbnails finally after > 2hrs. Thats > 25% of the time it would take to read *all* the files over the 100MB/s > network. I can not think Bio-Formats would not read 25% of the 4TB > data, just to show thumbnails? > > > When using showinf, it prints first: > Checking file format [Zeiss CZI] > Initializing reader > ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP lmo2Red 7ss .czi > > ... and then after around 1 hour, it fails with out-of-memory. Can > it really take >2 hrs before I can see anything? Should the option > "Open files individually" not limit it to the first timepoint? > > Cheers, > > Mario > > > > > Did you check out the "Debugging JVM hangs" section of the Debugging page? > > http://imagej.net/Debugging#Debugging_JVM_hangs > > > > Getting a stack trace will tell you more about what is going on -- i.e., > > what Bio-Formats is doing internally, whether it has crashed, etc. > > > > Did you try with the Bio-Formats update site enabled? This enables the > > 5.1.x development builds, which contain additional bug-fixes. > > http://fiji.sc/Bio-Formats#Daily_builds > > > > If this is a bug in Bio-Formats (if showinf also does not work, etc.), then > > please report this bug to the OME team. > > http://openmicroscopy.org/info/bio-formats/about/bug-reporting.html > > > > Regards, > > Curtis > > > > On Tue, Mar 3, 2015 at 8:45 AM, Mario Emmenlauer <[hidden email]> > > wrote: > > > >> Hi, > >> > >> I have a 85 GB CZI file (one out of a series of 40 files in the > >> same folder) that I wanted to open with latest Fiji (updates from > >> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried: > >> Plugins > Bio-Formats > Bio-Formats Importer > >> > >> with options: > >> - View with Hyperstack > >> - Open files individually > >> - Use virtual stack > >> - all other options unchecked > >> > >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens. > >> I've waited 20 minutes now, then gave up. About Bio-Formats says > >> "5.0.7 built 3.2.2015 revision 274796b". > >> > >> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with > >> the --debug option, but the command prompt console does not show any > >> messages (to track down whats happening). > >> > >> Thanks for your help, and all the best, > >> > >> Mario > >> > >> > >> > >> > >> -- > >> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > >> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > >> D-81669 München http://www.biodataanalysis.de/ > >> > >> -- > >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > >> > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > A: Yes. > > Q: Are you sure? > >> A: Because it reverses the logical flow of conversation. > >>> Q: Why is top posting annoying in email? > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > D-81669 München http://www.biodataanalysis.de/ > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
«
Return to ImageJ
|
1 view|%1 views
| Free forum by Nabble | Edit this page |