Import a really big raw image or process it without opening and displaying

[John Oreopoulos]

I have a really big image I want to process in ImageJ. It is a 1-Bit RAW file spanning 116000x116000 pixels. When I try to import into ImageJ I get an error relating boundaries and the image does not open. I'm guessing this means it's too big to open and display.

Is there a way I can work on this image without actually displaying it? I have a macro where I do some fairly basic image operations on a small sub-section of it, but my macro currently requires that the image be opened once to draw a ROI on it.

Any tips are much appreciated.

John Oreopoulos
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Mark J. Chopping]

Hi John,

You can try opening the image as a "raw" image with the Virtual Stack
option. You can open really large images this way with ImageJ. The image
will display unless you run ImageJ in batch mode -- though this is not a
problem; it will just show the top slice, so will not eat up all your
memory. Example macro code:

// open TIFF using the Raw method with the Virtual Stack facility

run("Raw...", "open="+infile+" image=[16-bit Unsigned] width="+srcCols+"
height="+Wsize+" offset="+mOffset+" number="+ NumSlices +" gap=0
little-endian use");

To use this, you have to:

* choose how to divide up your imagery. In the snippet above, Wsize
   determines the height of the slices and NumSlices the number of slices,
   while srcCols is the width of the image, so that NumSlices x Wsize x
   srcCols = the total number of pixels.

* obtain the offset to imagery. I get the #cols, #rows and offset to
   imagery using tiffdump (part of libtiff). This information is written to
   file in advance and then read in and parsed when I run my ImageJ code.
   tiffdump usage (e.g., at Unix prompt):

  % tiffdump my_huge_TIFF_102615.tif > my_huge_TIFF_102615.tdi

(it's just a text file). Then open and parse in your ImageJ macro:

  // == get offset from tiffdump output ============

  tdi = "my_huge_TIFF_102615.tdi";
  TDText  = File.openAsString(tdi);
  ftext   = split(TDText, "\n");

  // Clean the strings, removing ( ) < >
  for (k = 0; k < ftext.length; k++) {
   ftext[k]=replace(ftext[k], "(" , " ");
   ftext[k]=replace(ftext[k], ")" , " ");
   ftext[k]=replace(ftext[k], "<" , " ");
   ftext[k]=replace(ftext[k], ">" , " ");
  }

  for (k = 0; k < ftext.length; k++) {
     if ( lengthOf(ftext[k] ) > 0) {
       sinfo = split(ftext[k]," ");
       if ( lengthOf(sinfo) > 1)  {
         sinfo[0] = replace(sinfo[0], "^\\s*", ""); //removes leading whitespaces
         if ( (startsWith(sinfo[0], "ImageWidth"))   ) srcCols = parseFloat(sinfo[5]);
         if ( (startsWith(sinfo[0], "ImageLength"))  ) srcRows = parseFloat(sinfo[5]);
         if ( (startsWith(sinfo[0], "StripOffsets")) ) mOffset = parseFloat(sinfo[5]);
       } // if len
      } //if len
  } // for k

  if (mOffset < 0) exit("Offset is negative!");
  print("srcCols  ="+srcCols);       // #columns
  print("srcRows  ="+srcRows);       // #rows
  print("mOffset  ="+mOffset);       // offset to imagery
  NumSlices = floor(srcRows/Wsize);  // number of virtual stack slices

  // == end get offset from tiffdump output ==========

  The TIFF Tags plugin will do this for images smaller than 2GB (?) but
  seems to fail for really large images. Just for reference, I used to do
  it like this:

  // get offset from TIFF tag:

  mOffsetS1   = call("TIFF_Tags.getTag", path+infile, 273);
  mOffsetS2   = split(mOffsetS1);
  mOffset     = mOffsetS2[0];
  if (mOffset < 0) exit("Offset is negative!");
  srcCols     = call("TIFF_Tags.getTag", path+infile, 256);
  srcRows     = call("TIFF_Tags.getTag", path+infile, 257);

I hope this helps.

  Best wishes,

   Mark

  Mark Chopping, Ph.D.
  Professor, Department of Earth & Environmental Studies
  Center for Environmental & Life Sciences LS 312
  Montclair State University, Montclair, NJ 07043
  ------------------------------------------------------------
  E: [hidden email] F: [phone]
  W: http://www.montclair.edu/csam/remote-sensing-lab/

On Mon, 26 Oct 2015, John Oreopoulos wrote:

> Date: Mon, 26 Oct 2015 20:33:59 -0400
> From: John Oreopoulos <[hidden email]>
> Reply-To: [hidden email]
> To: [hidden email]
> Subject: Import a really big raw image or process it without opening and
>     displaying
>
> I have a really big image I want to process in ImageJ. It is a 1-Bit RAW file spanning 116000x116000 pixels. When I try to import into ImageJ I get an error relating boundaries and the image does not open. I'm guessing this means it's too big to open and display.
>
> Is there a way I can work on this image without actually displaying it? I have a macro where I do some fairly basic image operations on a small sub-section of it, but my macro currently requires that the image be opened once to draw a ROI on it.
>
> Any tips are much appreciated.
>
> John Oreopoulos
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html