Posted by
ctrueden on
Mar 24, 2015; 9:37pm
URL: http://imagej.273.s1.nabble.com/bfconverter-nd2-to-tif-tp5012133p5012151.html
Hi Greg,
> So my target format would definitely be the ImageJs 'raw' TIFF, as all
> operations seem to be much faster with that format.
I understand that your goal is to maximize performance. But please realize
that once the pixels have been read into memory, it does not matter how the
data was stored on disk. There are actually two major cases here:
A) Using a virtual stack. In that case, the file format _could_ matter
because it might be faster to read a plane on demand from a TIFF than from
an ND2. I haven't benchmarked this though, so I don't know.
B) Using swap space as virtual RAM. In this case, ImageJ will read all the
planes upfront, and then start caching them back out to disk as your
virtual RAM usage exceeds your actual amount of RAM. I expect that when you
do things this way, the data is really slow to load, right? For this reason
and others, I recommend using the virtual stack feature rather than this
swap space approach.
In other words: even if you were able to write out the data as an ImageJ1
raw pseudo-TIFF, I doubt it would magically solve all your problems.
> with 'lagginess' I mean especially the performance after loading the
> file, so switching to next frame for example.
With the Bio-Formats "Use virtual stack" option for your ND2 data, how long
does it take to switch to a new frame? It is supposed to take a fraction of
a second—or at most, on the order of a second or two for very large image
planes.
How large are your image planes? More than 1GPix each? If so, that would
explain the lagginess.
> I see no other way as loading the whole Stack into memory
There is another way: the "Specify range for each series" checkbox. With
that, you can open only a subset of your image planes at a time, perform
your plane-wise analysis, write out the results, rinse and repeat. Then you
can keep your processing within the amount of available RAM you have.
Another solution is simply to buy more RAM. It is pretty easy to get a
machine with more than 128GB of RAM these days, which accommodates very
large datasets.
> the Hyperstack information is lost in BigTIFF format
Try exporting to a .ome.tiff rather than only a .tiff. The vanilla TIFF
format (BigTIFF or otherwise) does not support >3 dimensions. But OME-TIFF
does.
> instead of doing nice image analysis with jython in Fiji, I am now
> more or less lost in a jungle of formats and memory problems, I guess
> that is the fate of the life cell image analyst
Sorry, I know it is frustrating. We definitely want to provide better
support for big image processing, TIFF or otherwise -- it is one of
ImageJ2's major goals. But it is not a trivial thing. To that end, I filed
an issue to track the various "better support for big TIFF files" scenarios
people have raised:
https://github.com/imagej/imagej/issues/117Regards,
Curtis
On Tue, Mar 24, 2015 at 10:35 AM, Greg <
[hidden email]> wrote:
> Hi Curtis,
>
> ok thanks for the clarification about the tif formats. So my target format
> would definitely be the ImageJs 'raw' TIFF, as all operations seem to be
> much faster with that format. I still see no way of doing the conversion
> without loading the whole stack into memory. And with 'lagginess' I mean
> especially the performance after loading the file, so switching to next
> frame for example. So bascically I see no performance difference in between
> BigTIFF and the proprietary nd2 format, and it is also essentially the same
> if I open them as virtual stack or not.. so ya, I migth start a thread on
> the ome site you pointed me to.
>
> The Hyperstack problem turned out to be a bit more serious, as again to
> make
> the output of the bfconverter a Hyperstack, I see no other way as loading
> the whole Stack into memory as the Hyperstack information is lost in
> BigTIFF
> format ?!
>
> So instead of doing nice image analysis with jython in Fiji, I am now more
> or less lost in a jungle of formats and memory problems, I guess that is
> the
> fate of the life cell image analyst :P
>
> Best,
> Greg
>
>
>
> --
> View this message in context:
>
http://imagej.1557.x6.nabble.com/bfconverter-nd2-to-tif-tp5012133p5012136.html> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list:
http://imagej.nih.gov/ij/list.html>
--
ImageJ mailing list:
http://imagej.nih.gov/ij/list.html