Posted by
Jeff Brandenburg on
Jan 18, 2006; 8:12pm
URL: http://imagej.273.s1.nabble.com/making-memory-tp3704020p3704022.html
On Jan 16, 2006, at 4:14 PM, Robert Edward Martin wrote:
> Basically, I need to have all 1800 images open in order to perform a
> vertical slice and I'm wondering if this is a feasible idea or if I
> should create some sort of file structure that 3-D data viewer could
> call up data from a disk in a more regular format (I calculate that the
> images would amount to 473MB*1800/85~=10GB Ram which no system has
> these days though I might be able to convert the data to 8 or 16 bit
> unsigned if that helps).
We wrestle with the same problem here. Even if you could get
everything into RAM, this kind of reslicing can be hideously expensive.
It's not so bad to reslice row-wise, but if you have to reslice
column-wise ("rotate a cube left or right"), you essentially make your
system's caching strategies worthless. It's worse if you're working
with ImageJ-style "virtual stacks", because there you're repeatedly
reopening and rereading files, and it can take minutes to generate a
single slice.
I've seen two approaches used with some success. The first, and most
general, recasts the volume as a set of smaller cubes, ideally each
small enough to fit into the fastest cache RAM available -- say 8x8x8
or 16x16x16. This slows things down for some special cases; for
example, if you start with axial data, displaying one axial slice would
require that you read ~2MB of data, while generating an axial slice
from the cube representation would make you read 16MB or 32MB of data.
But generating a slice in *any* of the three basic orientations takes
the same amount of overhead with cubes, whereas the "wrong" reslice of
your original data could force you to read hundreds or thousands of
times more data. Better yet, the cube representation lets you generate
arbitrary oblique slices with about the same overhead. (Does anyone
know if this is what Volume Viewer does?)
The second, and easiest, reslices the volume up-front and stores three
copies, one in each orientation. This means you'd need 30GB rather
than 10GB to store your volume, but it would let you use Virtual Stack
Opener (or any tool that works similarly) to do quick browsing in any
of the three orientations. I'm considering adopting this approach to
let our users browse images in multiple orientations -- it's easy to
understand, easy to implement, and I'd claim that it gives optimal
speed (at the obvious expense of space).
Here's a team at Drexel that's developed a client-server system for
viewing large datasets in cube ("macrovoxel") format, although the
server isn't implemented in Java:
http://www.neuroterrain.org/Here's a project at the Pittsburgh Supercomputing Center that serves up
*really* large datasets from a hierarchical-cube format:
http://www.psc.edu/biomed/research/VB/Here's some more detailed discussion of the PSC system implementation
(again, alas, not in Java):
http://www.psc.edu/biomed/research/VH/PVB/implementation/implement.htmhttp://www.psc.edu/~smp/files/vb-presentation.pdfNote that they're aiming for interactive browsing within a 25-TERABYTE
TEM volume data set, where each "slice" is ~60K pixels on a side -- so
large that even a single slice can't fit into a 32-bit address space!
So, there's plenty of room for our data sets to grow. :-)
--
-jeffB (Jeff Brandenburg, Duke Center for In-Vivo Microscopy)