ImageJ reads reversed bits from unsigned 16-bit TIFF saved by imwrite() from GNU Octave.

[Pariksheet Nanda]

Hi all,

ImageJ shows pixel values different from those saved by GNU Octave.
The byte positions are the same, the bits within each byte are
reversed, so I'm not sure if this is a traditional problem of
endianness?  For example the value 274 saved from the Octave matrix
appears as 32480 in ImageJ; the binary representations of which are
below:

Number saved by Octave (decimal 274):  01001000 10000000
Number read by ImageJ (decimal 32840): 00010010 00000001

1) Can anyone else reproduce this problem?  At the end of this e-mail
is the code for Octave to generate the image stacks, and an ImageJ
macro that reads the value of each slice (or frame), and shows the
binary and expected values in the results window.  Also the image
stacks from Octave are uploaded here and compressed to 7 kb; password
is imagej
https://filelocker.uconn.edu/public_download?shareId=2c8c0c3141741de36547a9db46b1f06c

2) Can you help me identify where the problem might be coming from?
GNU Octave has a wrapper for ImageMagick C++ interface, which in turn
uses libtiff.  Perhaps the next logical thing would be to inspect the
binary values from the TIFF file or check the TIFF file headers, etc
but I'm not sure how to approach those investigations.  My application
and library versions are:

- ImageJ 1.49i (bundled with FIJI linux-32)
- Octave 3.8.2
- ImageMagick 6.8.8.10
- libtiff 4.0.3

3) Are there any Octave + ImageJ users out there?  How do you export
images from Octave for ImageJ?  Also do you interface Octave data with
ImageJ / FIJI using the Java Octave Forge package?

Best wishes,
Pariksheet



%%%%%%%%%%
%% ImageJ does not open TIFF images saved by GNU Octave
%% properly. While the endianness is the same, the bits are reversed.So
%% within each byte one has to reverse the bits to view the image in
%% ImageJ correctly. E.g.:
%%
%% Saved by Octave (decimal 274):  01001000 10000000
%%
%% Read by ImageJ (decimal 32840): 00010010 00000001
%%
%% This program demonstrates the problem and workaround.

%% Create image frames with each frame containing uniform values of 0,
%% 1, 2, etc.
im = 0:2^9-1;

function im_save (x, path)
  %% Reshape to Multi-TIFF image.
  w = 1;
  h = 1;
  %% Multi-tiff needs a special third dimension.
  [GREY, RGB, RGBA] = {1, 3, 4}{:};
  im = reshape (x, w, h, GREY, numel(x));
  imwrite (uint16 (im), path, 'Quality', 100);
endfunction

function y = reverse_uint16 (x)
  %% Reverse bits for ImageJ.  Algorithm from:
  %% http://eshare.stust.edu.tw/EshareFile/2012_5/2012_5_516c9900.pdf
  bits = 16;
  b = uint32 (2^(bits-1));
  y = uint32 (zeros (size (x)));
  for i = 1:bits
    y = bitget (x, i) * b + y;
    b = b / 2;
  end
  y = uint16(y);
endfunction

%% Bad image.
im_save (im, '/tmp/bad.tif');

%% Workaround.
im_rev = reverse_uint16 (im);
im_rev = swapbytes (im_rev);
im_save (im_rev, '/tmp/good.tif');


//////////
// Compare the image values intended to be saved by GNU Octave
// against values read by ImageJ.

for (i = 0; i < nSlices; i++) {
    setSlice(i + 1);
    setResult("Octave Value", i, i);
    getStatistics(area, mean, min, max, std, histogram);
    setResult("ImageJ Value", i, mean);
    setResult("Octave Binary Value", i, toBinary(i));
    setResult("ImageJ Binary Value", i, toBinary(mean));
}

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Pariksheet,

when looking at it with a hex editor, your sample file "good.tif" clearly has the correct values (1x1 pixels, pixel value for each stack slice is equal to the slice number, starting at 0) and "bad.tif" has the bits of each byte reversed in the data
With a hex editor, you can find the values at offsets (decimal) 8, 272, 536, 800, etc.

This is not an issue of byte order but reversed bits *inside* a byte. TIFF has a well-defined byte order anyhow (defined by the first two bytes of the TIFF file; big-endian in your case).

So it is not ImageJ that reverses the bits.

Unfortunately, I know nothing about Octave, but it must be Octave or your Octave code that creates these values.

Michael
________________________________________________________________
On Nov 3, 2014, at 12:05, Pariksheet Nanda wrote:

> Hi all,
>
> ImageJ shows pixel values different from those saved by GNU Octave.
> The byte positions are the same, the bits within each byte are
> reversed, so I'm not sure if this is a traditional problem of
> endianness?  For example the value 274 saved from the Octave matrix
> appears as 32480 in ImageJ; the binary representations of which are
> below:
>
> Number saved by Octave (decimal 274):  01001000 10000000
> Number read by ImageJ (decimal 32840): 00010010 00000001
>
> 1) Can anyone else reproduce this problem?  At the end of this e-mail
> is the code for Octave to generate the image stacks, and an ImageJ
> macro that reads the value of each slice (or frame), and shows the
> binary and expected values in the results window.  Also the image
> stacks from Octave are uploaded here and compressed to 7 kb; password
> is imagej
> https://filelocker.uconn.edu/public_download?shareId=2c8c0c3141741de36547a9db46b1f06c
>
> 2) Can you help me identify where the problem might be coming from?
> GNU Octave has a wrapper for ImageMagick C++ interface, which in turn
> uses libtiff.  Perhaps the next logical thing would be to inspect the
> binary values from the TIFF file or check the TIFF file headers, etc
> but I'm not sure how to approach those investigations.  My application
> and library versions are:
>
> - ImageJ 1.49i (bundled with FIJI linux-32)
> - Octave 3.8.2
> - ImageMagick 6.8.8.10
> - libtiff 4.0.3
>
> 3) Are there any Octave + ImageJ users out there?  How do you export
> images from Octave for ImageJ?  Also do you interface Octave data with
> ImageJ / FIJI using the Java Octave Forge package?
>
> Best wishes,
> Pariksheet
>
>
>
> %%%%%%%%%%
> %% ImageJ does not open TIFF images saved by GNU Octave
> %% properly. While the endianness is the same, the bits are reversed.So
> %% within each byte one has to reverse the bits to view the image in
> %% ImageJ correctly. E.g.:
> %%
> %% Saved by Octave (decimal 274):  01001000 10000000
> %%
> %% Read by ImageJ (decimal 32840): 00010010 00000001
> %%
> %% This program demonstrates the problem and workaround.
>
> %% Create image frames with each frame containing uniform values of 0,
> %% 1, 2, etc.
> im = 0:2^9-1;
>
> function im_save (x, path)
>  %% Reshape to Multi-TIFF image.
>  w = 1;
>  h = 1;
>  %% Multi-tiff needs a special third dimension.
>  [GREY, RGB, RGBA] = {1, 3, 4}{:};
>  im = reshape (x, w, h, GREY, numel(x));
>  imwrite (uint16 (im), path, 'Quality', 100);
> endfunction
>
> function y = reverse_uint16 (x)
>  %% Reverse bits for ImageJ.  Algorithm from:
>  %% http://eshare.stust.edu.tw/EshareFile/2012_5/2012_5_516c9900.pdf
>  bits = 16;
>  b = uint32 (2^(bits-1));
>  y = uint32 (zeros (size (x)));
>  for i = 1:bits
>    y = bitget (x, i) * b + y;
>    b = b / 2;
>  end
>  y = uint16(y);
> endfunction
>
> %% Bad image.
> im_save (im, '/tmp/bad.tif');
>
> %% Workaround.
> im_rev = reverse_uint16 (im);
> im_rev = swapbytes (im_rev);
> im_save (im_rev, '/tmp/good.tif');
>
>
> //////////
> // Compare the image values intended to be saved by GNU Octave
> // against values read by ImageJ.
>
> for (i = 0; i < nSlices; i++) {
>    setSlice(i + 1);
>    setResult("Octave Value", i, i);
>    getStatistics(area, mean, min, max, std, histogram);
>    setResult("ImageJ Value", i, mean);
>    setResult("Octave Binary Value", i, toBinary(i));
>    setResult("ImageJ Binary Value", i, toBinary(mean));
> }
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ctrueden]

Hi Pariksheet & Michael,

> This is not an issue of byte order but reversed bits *inside* a byte.

I was not able to download the example files (remote server error), but it
sounds like a FillOrder issue. See page 32 of the TIFF specification [1].

Pariksheet: Did you try reading your TIFFs using SCIFIO [2]? It has code to
handle both FillOrders [3].

Regards,
Curtis

[1] http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf
[2] http://imagej.net/SCIFIO
[3]
https://github.com/scifio/scifio/blob/scifio-0.17.0/src/main/java/io/scif/formats/tiff/TiffParser.java#L1162-L1198


On Mon, Nov 3, 2014 at 10:03 AM, Michael Schmid <[hidden email]>
wrote:

> Hi Pariksheet,
>
> when looking at it with a hex editor, your sample file "good.tif" clearly
> has the correct values (1x1 pixels, pixel value for each stack slice is
> equal to the slice number, starting at 0) and "bad.tif" has the bits of
> each byte reversed in the data
> With a hex editor, you can find the values at offsets (decimal) 8, 272,
> 536, 800, etc.
>
> This is not an issue of byte order but reversed bits *inside* a byte. TIFF
> has a well-defined byte order anyhow (defined by the first two bytes of the
> TIFF file; big-endian in your case).
>
> So it is not ImageJ that reverses the bits.
>
> Unfortunately, I know nothing about Octave, but it must be Octave or your
> Octave code that creates these values.
>
> Michael
> ________________________________________________________________
> On Nov 3, 2014, at 12:05, Pariksheet Nanda wrote:
>
> > Hi all,
> >
> > ImageJ shows pixel values different from those saved by GNU Octave.
> > The byte positions are the same, the bits within each byte are
> > reversed, so I'm not sure if this is a traditional problem of
> > endianness?  For example the value 274 saved from the Octave matrix
> > appears as 32480 in ImageJ; the binary representations of which are
> > below:
> >
> > Number saved by Octave (decimal 274):  01001000 10000000
> > Number read by ImageJ (decimal 32840): 00010010 00000001
> >
> > 1) Can anyone else reproduce this problem?  At the end of this e-mail
> > is the code for Octave to generate the image stacks, and an ImageJ
> > macro that reads the value of each slice (or frame), and shows the
> > binary and expected values in the results window.  Also the image
> > stacks from Octave are uploaded here and compressed to 7 kb; password
> > is imagej
> >
> https://filelocker.uconn.edu/public_download?shareId=2c8c0c3141741de36547a9db46b1f06c
> >
> > 2) Can you help me identify where the problem might be coming from?
> > GNU Octave has a wrapper for ImageMagick C++ interface, which in turn
> > uses libtiff.  Perhaps the next logical thing would be to inspect the
> > binary values from the TIFF file or check the TIFF file headers, etc
> > but I'm not sure how to approach those investigations.  My application
> > and library versions are:
> >
> > - ImageJ 1.49i (bundled with FIJI linux-32)
> > - Octave 3.8.2
> > - ImageMagick 6.8.8.10
> > - libtiff 4.0.3
> >
> > 3) Are there any Octave + ImageJ users out there?  How do you export
> > images from Octave for ImageJ?  Also do you interface Octave data with
> > ImageJ / FIJI using the Java Octave Forge package?
> >
> > Best wishes,
> > Pariksheet
> >
> >
> >
> > %%%%%%%%%%
> > %% ImageJ does not open TIFF images saved by GNU Octave
> > %% properly. While the endianness is the same, the bits are reversed.So
> > %% within each byte one has to reverse the bits to view the image in
> > %% ImageJ correctly. E.g.:
> > %%
> > %% Saved by Octave (decimal 274):  01001000 10000000
> > %%
> > %% Read by ImageJ (decimal 32840): 00010010 00000001
> > %%
> > %% This program demonstrates the problem and workaround.
> >
> > %% Create image frames with each frame containing uniform values of 0,
> > %% 1, 2, etc.
> > im = 0:2^9-1;
> >
> > function im_save (x, path)
> >  %% Reshape to Multi-TIFF image.
> >  w = 1;
> >  h = 1;
> >  %% Multi-tiff needs a special third dimension.
> >  [GREY, RGB, RGBA] = {1, 3, 4}{:};
> >  im = reshape (x, w, h, GREY, numel(x));
> >  imwrite (uint16 (im), path, 'Quality', 100);
> > endfunction
> >
> > function y = reverse_uint16 (x)
> >  %% Reverse bits for ImageJ.  Algorithm from:
> >  %% http://eshare.stust.edu.tw/EshareFile/2012_5/2012_5_516c9900.pdf
> >  bits = 16;
> >  b = uint32 (2^(bits-1));
> >  y = uint32 (zeros (size (x)));
> >  for i = 1:bits
> >    y = bitget (x, i) * b + y;
> >    b = b / 2;
> >  end
> >  y = uint16(y);
> > endfunction
> >
> > %% Bad image.
> > im_save (im, '/tmp/bad.tif');
> >
> > %% Workaround.
> > im_rev = reverse_uint16 (im);
> > im_rev = swapbytes (im_rev);
> > im_save (im_rev, '/tmp/good.tif');
> >
> >
> > //////////
> > // Compare the image values intended to be saved by GNU Octave
> > // against values read by ImageJ.
> >
> > for (i = 0; i < nSlices; i++) {
> >    setSlice(i + 1);
> >    setResult("Octave Value", i, i);
> >    getStatistics(area, mean, min, max, std, histogram);
> >    setResult("ImageJ Value", i, mean);
> >    setResult("Octave Binary Value", i, toBinary(i));
> >    setResult("ImageJ Binary Value", i, toBinary(mean));
> > }
> >
> > --
> > ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Pariksheet Nanda]

Hi Curtis and Michael,

On Tue, Nov 4, 2014 at 2:44 AM, Curtis Rueden <[hidden email]> wrote:
>
On Mon, Nov 3, 2014 at 4:03 PM, Michael Schmid <[hidden email]> wrote:
>>
>> This is not an issue of byte order but reversed bits *inside* a byte.
>
> I was not able to download the example files (remote server error), but it
> sounds like a FillOrder issue. See page 32 of the TIFF specification [1].
>
> Pariksheet: Did you try reading your TIFFs using SCIFIO [2]? It has code to
> handle both FillOrders [3].

Indeed, FIJI reads in the data correctly if I enable "Use SCIFIO when
opening files" checkbox in FIJI from Edit > Options > ImageJ2.

Thanks also for the tip using the hex editor; I can see the bit
reversal by reading the image into emacs, applying hexl-mode and
running M-x ediff-buffers for the image files.  For reading the TIFF
tags, I found tiffdump (packaged with libtiff) convenient inspecting
and diff-ing tags.  Running `tiffdump bad.tif' shows:

   FillOrder (266) SHORT (3) 1<2>

Thus Octave's imwrite() for TIFF images uses FillOrder type <2> (what
SCIFIO calls "reversed") and ImageJ 1.x assumes FillOrder type <1>
("normal") [4].

Since ImageJ2 handles Fill Order in reading the image, I'll ask the
Octave folks if they intended to use the esoteric Fill Order type <2>.

Thanks for all your help!

>> Michael

> Curtis

Pariksheet

[4] https://github.com/imagej/ImageJA/blob/master/src/main/java/ij/io/ImageReader.java#L136

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html