Reslice crashes on huge stack

[Ronald Wood]

I am encountering a fatal error detected by the Java Runtime Environment running fiji on a large dataset. I used fiji to crop 70mm photographs from NLM's Visible Human female, and saved this as an image sequence.

I read in 5190 (3966x2073) RGB images, 159 GB, and this took 31% of the available memory; this is a red hat linux environment with 500 GB assigned to this node.

Reslicing results in a fatal error regardless of virtual stack/in core, plane of reslice and so forth. This can occur at the end of the first slice, or can proceed for several slices before failing:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00002b2dd4527bd0, pid=28450, tid=48025052264192
#
# JRE version: 6.0_24-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x83bd0]


Just what are the real limits on reslice ?

Information about your version of Java:

  os.arch => amd64
  os.name => Linux
  os.version => 2.6.32-358.el6.x86_64
  java.version => 1.6.0_24
  java.vendor => Sun Microsystems Inc.
  java.runtime.name => Java(TM) SE Runtime Environment
  java.runtime.version => 1.6.0_24-b07
  java.vm.name => Java HotSpot(TM) 64-Bit Server VM
  java.vm.version => 19.1-b02
  java.vm.vendor => Sun Microsystems Inc.
  java.vm.info => mixed mode
  java.awt.graphicsenv => sun.awt.X11GraphicsEnvironment
  java.specification.name => Java Platform API Specification
  java.specification.version => 1.6
  sun.cpu.endian => little
  sun.desktop => gnome
  file.separator => /

The up-to-date check says: READ_ONLY

Information relevant to JAVA_HOME related problems:

  JAVA_HOME is set to: /gpfs/fs1/sfw/fiji/continuous/64bit/java/linux-amd64/jdk1.6.0_24//jre
  ij.dir => /gpfs/fs1/sfw/fiji/continuous/64bit



Ron Wood
university of Rochester

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Saalfeld, Stephan]

Hi Ron,

this is a segmentation fault which is the sort of thing that should
never happen in Java, i.e. it is not too tightly related to size limits
but to a bug in Java.  It'd be great if you could try to run with a
different JRE, i.e OpenJDK 7 that is shipped with your Linux
distribution.

A note on size.  You may indeed run into an out of memory situation when
reslicing the visible human data set in RAM.  Note that the dataset is
non-isotropic, i.e. individual scans are thicker than the x,y-resolution
at which they were imaged, i.e. a reslice that looks `isotropic' is n
times larger than the original series.

You can workaround this by reslicing small crops of the dataset that you
later join into a complete dataset preferably slice by slice.

You could try this tool that uses ImgLib2 to generate isotropic reslices
or crops for our web-based annotation tool CATMAID:

https://github.com/axtimwalde/catmaid-tools

Although the tool is designed to fetch data from a CATMAID stack, you
can use it to fetch from any local series, using the printf-style
sourceURLFormat, e.g.

sourceUrlFormat='file:/my-series/xy/%5$d.tif'

in combination with full-size `tiles', e.g.

sourceTileWidth=20000
sourceTileHeight=30000

If you do not need a full export but want to look at individual slices
only, you may want to use a virtual reslicer like

http://fiji.sc/BigDataViewer

or it's pre-ImgLib2 predecessor

http://fiji.sc/Interactive_Stack_Rotation

Best,
Stephan


On Sun, 2014-12-28 at 12:33 -0500, Ronald Wood wrote:

> I am encountering a fatal error detected by the Java Runtime Environment running fiji on a large dataset. I used fiji to crop 70mm photographs from NLM's Visible Human female, and saved this as an image sequence.
>
> I read in 5190 (3966x2073) RGB images, 159 GB, and this took 31% of the available memory; this is a red hat linux environment with 500 GB assigned to this node.
>
> Reslicing results in a fatal error regardless of virtual stack/in core, plane of reslice and so forth. This can occur at the end of the first slice, or can proceed for several slices before failing:
>
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00002b2dd4527bd0, pid=28450, tid=48025052264192
> #
> # JRE version: 6.0_24-b07
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode linux-amd64 )
> # Problematic frame:
> # C  [libc.so.6+0x83bd0]
>
>
> Just what are the real limits on reslice ?
>
> Information about your version of Java:
>
>   os.arch => amd64
>   os.name => Linux
>   os.version => 2.6.32-358.el6.x86_64
>   java.version => 1.6.0_24
>   java.vendor => Sun Microsystems Inc.
>   java.runtime.name => Java(TM) SE Runtime Environment
>   java.runtime.version => 1.6.0_24-b07
>   java.vm.name => Java HotSpot(TM) 64-Bit Server VM
>   java.vm.version => 19.1-b02
>   java.vm.vendor => Sun Microsystems Inc.
>   java.vm.info => mixed mode
>   java.awt.graphicsenv => sun.awt.X11GraphicsEnvironment
>   java.specification.name => Java Platform API Specification
>   java.specification.version => 1.6
>   sun.cpu.endian => little
>   sun.desktop => gnome
>   file.separator => /
>
> The up-to-date check says: READ_ONLY
>
> Information relevant to JAVA_HOME related problems:
>
>   JAVA_HOME is set to: /gpfs/fs1/sfw/fiji/continuous/64bit/java/linux-amd64/jdk1.6.0_24//jre
>   ij.dir => /gpfs/fs1/sfw/fiji/continuous/64bit
>
>
>
> Ron Wood
> university of Rochester
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--

Stephan Saalfeld, Ph.D.
Group Leader
Janelia Farm Research Campus
19700 Helix Drive | Ashburn, VA 20147
Phone: [phone] | Fax: [phone]
[hidden email]

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html