LZW compressed tiffs

[Jon Harman]

Hi

In looking for lossless compression for my large image files, I started
using LZW tiffs. (are there better formats?) I found that different
programs produce very different output files using this and sometimes
ImageJ would fail to read the output.  For instance paintshop pro
sometimes produces a file that can't be read.  Photoshop seems much
better.  Irfanview  produces files that sometimes can't be read, and
additionally aren't much compressed.  So this format seems to be a can
of worms.

At any rate the other programs did not have the problem that ImageJ did
in opening the files, they all could read the files wrote by the others,
unlike ImageJ.  I looked into this and found that making the following
modification to readCompressesChunkyRGB in ImageReader.java fixed the
problem.  Just add a test that j is < nPixels before writing to pixels:

                if(j < nPixels){
                    if (bgr)
                        pixels[j] = 0xff000000 | (blue<<16) | (green<<8)
| red;
                    else
                        pixels[j] = 0xff000000 | (red<<16) | (green<<8)
| blue;
                }

I have no clue as to why j is getting too big, but adding the test seems
to work just fine.

Jon

[ctrueden]

Hi Jon,

I am not sure what's going on here, but I doubt your fix completely
solves the problem (capping off j, while avoiding exceptions, means that
many RGB triples being read in are being thrown away). But I suppose if
you're seeing what looks like the correct picture, it's a step forward.

Also, ImageJ has a restriction with the ordering of the strips within
the TIFF file (LZW or not), and I recently found out that the libtiff
library does not always write LZW-compressed TIFFs conforming to this
assumption. ImageJ uses InputStream, not RandomAccessFile, so if the
image strips are out of order, ImageJ displays an error message (see
TiffDecoder.OpenIFD() in the STRIP_OFFSETS case). ImageJ could probably
be made to read in TIFFs with out-of-order strip offsets by sorting the
strip offsets and then populating the pixels array in that order (since
Java arrays are obviously random access).

Anyway, if you send me a sample image that is causing you difficulties,
I can look into it.

-Curtis

Jon Harman wrote:

> Hi
>
> In looking for lossless compression for my large image files, I
> started using LZW tiffs. (are there better formats?) I found that
> different programs produce very different output files using this and
> sometimes ImageJ would fail to read the output.  For instance
> paintshop pro sometimes produces a file that can't be read.  Photoshop
> seems much better.  Irfanview  produces files that sometimes can't be
> read, and additionally aren't much compressed.  So this format seems
> to be a can of worms.
>
> At any rate the other programs did not have the problem that ImageJ
> did in opening the files, they all could read the files wrote by the
> others, unlike ImageJ.  I looked into this and found that making the
> following modification to readCompressesChunkyRGB in ImageReader.java
> fixed the problem.  Just add a test that j is < nPixels before writing
> to pixels:
>
>                if(j < nPixels){
>                    if (bgr)
>                        pixels[j] = 0xff000000 | (blue<<16) |
> (green<<8) | red;
>                    else
>                        pixels[j] = 0xff000000 | (red<<16) | (green<<8)
> | blue;
>                }
>
> I have no clue as to why j is getting too big, but adding the test
> seems to work just fine.
>
> Jon
>

[Jarek Sacha]

Jon Harman wrote:

> In looking for lossless compression for my large image files, I
> started using LZW tiffs. (are there better formats?) I found that
> different programs produce very different output files using this and
> sometimes ImageJ would fail to read the output.  For instance
> paintshop pro sometimes produces a file that can't be read.  Photoshop
> seems much better.  Irfanview  produces files that sometimes can't be
> read, and additionally aren't much compressed.  So this format seems
> to be a can of worms.

PNG format has good lossless compression. ImageJ can read it with no
problems (at least I never seen any), however, to write PNG you will
need a plugin like one available at:
http://ij-plugins.sourceforge.net/plugins/imageio/index.html
That plugin bundle also have plugins that can correctly read tiled ("out
of order") TIFF images. Current release version does not support some of
the newer PhotoShop LZW variants, and cannot write compressed images.
The version 2.0 (not yet released but available through CVS), can read
PhotoShop LZW, and will be able to write LZW compressed TIFF images. It
will also support lossless JPEG2000 that can compress better than LZW
TIFF or PNG. If you need more info on that or will like to help with new
plugins (implementing or testing), let me know.

Jarek

[Jon Harman]

Hi,

Thanks for the suggestion.  I've checked it out and it seems to work
well, except for the following:
On very large files it can take a while to write out the png image, but
it is possible to close the image before the writing finishes and in
this case the output is only partial.  You have to watch the ImageJ
status line to verify that the write has finished.  Is there a way to
lock the image from closing before the output routine has finished?

Jon

.Jarek Sacha wrote:

> Jon Harman wrote:
>
>> In looking for lossless compression for my large image files, I
>> started using LZW tiffs. (are there better formats?) I found that
>> different programs produce very different output files using this and
>> sometimes ImageJ would fail to read the output.  For instance
>> paintshop pro sometimes produces a file that can't be read.  
>> Photoshop seems much better.  Irfanview  produces files that
>> sometimes can't be read, and additionally aren't much compressed.  So
>> this format seems to be a can of worms.
>
>
> PNG format has good lossless compression. ImageJ can read it with no
> problems (at least I never seen any), however, to write PNG you will
> need a plugin like one available at:
> http://ij-plugins.sourceforge.net/plugins/imageio/index.html
> That plugin bundle also have plugins that can correctly read tiled
> ("out of order") TIFF images. Current release version does not support
> some of the newer PhotoShop LZW variants, and cannot write compressed
> images. The version 2.0 (not yet released but available through CVS),
> can read PhotoShop LZW, and will be able to write LZW compressed TIFF
> images. It will also support lossless JPEG2000 that can compress
> better than LZW TIFF or PNG. If you need more info on that or will
> like to help with new plugins (implementing or testing), let me know.
>
> Jarek
>
>

[Jarek Sacha]

Jon Harman wrote:

> On very large files it can take a while to write out the png image,
> but it is possible to close the image before the writing finishes and
> in this case the output is only partial.  You have to watch the ImageJ
> status line to verify that the write has finished.  Is there a way to
> lock the image from closing before the output routine has finished?

Good question. I do not know the answer. ImagePlus has method lock() to
indicate to other threads that an image is in use. I tried to add it to
writer plugin, it did not help (tested with ImageJ v.1.34h and 1.35c).
Seems that ImageJ is not checking if image is locked when it closes it.
It may be a bug. It is a good question for Wayne Rasband. You may want
to post it in a separate thread.

Jarek