Posted by
Melissa Linkert-2 on
URL: http://imagej.273.s1.nabble.com/Fiji-Bio-Formats-is-not-opening-large-czi-tp5011842p5011890.html
Hi Mario,
> I tried that now, and it seems Stephan Preibisch might be right that
> opening the files just takes forever. But why? The network mount is
> reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the
> file in ~30sec, and I can scroll through the timepoints and stacks at
> reasonable speed. I opened the czi with "Open files individually" and
> "Use virtual stack" in Fiji, and I hoped this might speed things up.
>
> So in Fiji, I can select the thumbnails finally after > 2hrs. Thats
> 25% of the time it would take to read *all* the files over the 100MB/s
> network. I can not think Bio-Formats would not read 25% of the 4TB
> data, just to show thumbnails?
Reading large files over NFS on Windows is known to be slow in general.
We are currently reviewing improvements to this for the 5.1.0 release of
Bio-Formats later this month. Note though that these changes are not
yet available through the Bio-Formats update site in Fiji.
Please keep in mind that the initialization step isn't just showing thumbnails,
it's also reading all of the metadata from the entire dataset and assembling
offsets to all of the planes (to improve performance of reading images on demand).
> When using showinf, it prints first:
> Checking file format [Zeiss CZI]
> Initializing reader
> ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP lmo2Red 7ss .czi
>
> ... and then after around 1 hour, it fails with out-of-memory. Can
> it really take >2 hrs before I can see anything? Should the option
> "Open files individually" not limit it to the first timepoint?
If you use the "Open files individually", then it's important to pick
one of the files that ends with a number in parentheses. The "master"
file (the one without that doesn't end with a number) typically doesn't
contain pixel data, and is meant to indicate which files constitute the
full dataset - so if you pick the master file, it's going to group all
of the files together no matter what.
Regards,
-Melissa
On Wed, Mar 04, 2015 at 04:26:01PM +0100, Mario Emmenlauer wrote:
> Hi Curtis,
>
> On 03.03.2015 17:11, Curtis Rueden wrote:
> >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens.
> >> I've waited 20 minutes now, then gave up.
> >
> > Did you try reading it with the Bio-Formats showinf command line tool?
> >
http://openmicroscopy.org/info/bio-formats/users/comlinetools/>
> I tried that now, and it seems Stephan Preibisch might be right that
> opening the files just takes forever. But why? The network mount is
> reasonably fast, it achieves easily 100MB/s. Zeiss Zen 2012 opens the
> file in ~30sec, and I can scroll through the timepoints and stacks at
> reasonable speed. I opened the czi with "Open files individually" and
> "Use virtual stack" in Fiji, and I hoped this might speed things up.
>
> So in Fiji, I can select the thumbnails finally after > 2hrs. Thats
> 25% of the time it would take to read *all* the files over the 100MB/s
> network. I can not think Bio-Formats would not read 25% of the 4TB
> data, just to show thumbnails?
>
>
> When using showinf, it prints first:
> Checking file format [Zeiss CZI]
> Initializing reader
> ZeissCZIReader initializing L:\MarioEmmenlauer\Lightsheet_RAW\drlGFP lmo2Red 7ss .czi
>
> ... and then after around 1 hour, it fails with out-of-memory. Can
> it really take >2 hrs before I can see anything? Should the option
> "Open files individually" not limit it to the first timepoint?
>
> Cheers,
>
> Mario
>
>
>
> > Did you check out the "Debugging JVM hangs" section of the Debugging page?
> >
http://imagej.net/Debugging#Debugging_JVM_hangs> >
> > Getting a stack trace will tell you more about what is going on -- i.e.,
> > what Bio-Formats is doing internally, whether it has crashed, etc.
> >
> > Did you try with the Bio-Formats update site enabled? This enables the
> > 5.1.x development builds, which contain additional bug-fixes.
> >
http://fiji.sc/Bio-Formats#Daily_builds> >
> > If this is a bug in Bio-Formats (if showinf also does not work, etc.), then
> > please report this bug to the OME team.
> >
http://openmicroscopy.org/info/bio-formats/about/bug-reporting.html> >
> > Regards,
> > Curtis
> >
> > On Tue, Mar 3, 2015 at 8:45 AM, Mario Emmenlauer <
[hidden email]>
> > wrote:
> >
> >> Hi,
> >>
> >> I have a 85 GB CZI file (one out of a series of 40 files in the
> >> same folder) that I wanted to open with latest Fiji (updates from
> >> today, on Win7 x64 with 42GB RAM, Sun Java 7, -Xmx 12GB. I tried:
> >> Plugins > Bio-Formats > Bio-Formats Importer
> >>
> >> with options:
> >> - View with Hyperstack
> >> - Open files individually
> >> - Use virtual stack
> >> - all other options unchecked
> >>
> >> Fiji shows "Analyzing XXX" in the status bar, then nothing happens.
> >> I've waited 20 minutes now, then gave up. About Bio-Formats says
> >> "5.0.7 built 3.2.2015 revision 274796b".
> >>
> >> The file opens fine in Zeiss ZEN 2012 in ~30sec. I tried Fiji with
> >> the --debug option, but the command prompt console does not show any
> >> messages (to track down whats happening).
> >>
> >> Thanks for your help, and all the best,
> >>
> >> Mario
> >>
> >>
> >>
> >>
> >> --
> >> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489
> >> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch
> >> D-81669 München
http://www.biodataanalysis.de/> >>
> >> --
> >> ImageJ mailing list:
http://imagej.nih.gov/ij/list.html> >>
> >
> > --
> > ImageJ mailing list:
http://imagej.nih.gov/ij/list.html> >
>
> --
> A: Yes.
> > Q: Are you sure?
> >> A: Because it reverses the logical flow of conversation.
> >>> Q: Why is top posting annoying in email?
>
> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489
> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch
> D-81669 München
http://www.biodataanalysis.de/>
> --
> ImageJ mailing list:
http://imagej.nih.gov/ij/list.html--
ImageJ mailing list:
http://imagej.nih.gov/ij/list.html