Fixes to PNM/PGM/... reader/writer

[Spencer Eugene Olson]

Hi all,

I've added stack support to the PNM/PGM/... reader/writer.  For the NETPBM
format, stacks are simply stored as one fully formatted image (header and
all) concatenated right after the other in the same file.  I use the PGM
format quite often and hate having hundreds of files hanging around which are
really only slices in stacks of data.

I'd love to see this in future versions so that I don't have to keep replacing
it in the ij.jar file with my own.  And so that co-workers can open my PGM
stacks without having to get my updated ij.jar.  The attached java files are
changed with respect to those distributed with 1.37.  I've bracketed my
changes with something like /* BEGIN CHANGES \n author Spencer Olson */
and /* END CHANGES by Spencer Olson */ just so that it would be clear as too
which parts I changed.

ciao,

Spencer Olson

P.S.  As a side note, I think that it is time to change the labeling of "PGM"
to perhaps "PNM" in the "opener" menu (and other places?).  This would better
reflect the ability of ImageJ to read in other PNM formats.



--
------------
Spencer E. Olson
[hidden email]
http://www.umich.edu/~olsonse/
PGP Fingerprint: A452 312C 73DC 41EF 47F3  EE73 A898 B528 BAD2 0D0F
(See www.pgp.net or pgp.mit.edu for public keys to encrypt messages for me)

user, n.:
        The word computer professionals use when they mean "idiot."
                        -- Dave Barry, "Claw Your Way to the Top"
                         
[I always thought "computer professional" was the phrase hackers used
when they meant "idiot."  Ed.]

[dscho]

Hi,

On Thu, 4 Jan 2007, Spencer Eugene Olson wrote:

> I've added stack support to the PNM/PGM/... reader/writer.  For the
> NETPBM format, stacks are simply stored as one fully formatted image
> (header and all) concatenated right after the other in the same file.  
> I use the PGM format quite often and hate having hundreds of files
> hanging around which are really only slices in stacks of data.

According to

        http://en.wikipedia.org/wiki/Portable_Pixmap_file_format

stacks are not part of the format. Which programs (apart from "cat") do
write such stacks?

Ciao,
Dscho

P.S.: I don't know about others, but IMHO changes to source code should be
posted as diffs, rather than complete files. It makes it easier to see
what actually changed (and there were more changes than just those
enclosed in BEGIN/END CHANGES blocks).

[Harry Parker]

Your email is a good example of why Wikipedia is a good p
Hello Dscho,

Your email is a good example of why Wikipedia is a good place to start searching, but a bad place to end your search. The Wikipedia article does link to the the reference information and software on the PNM formats at the netpbm site hosted on SourceForge. There, at http://netpbm.sourceforge.net/doc/ppm.html , it is written,  "A PPM file consists of a sequence of one or more PPM images." The original pbm format & software (from the 1980's?) may have supported only one image per file, but the PNM formats and Netpbm package have supported multiple images for years now.

A good overview link to the netpbm software is, http://netpbm.sourceforge.net/doc/index.html.
The home page, http://netpbm.sourceforge.net, links to versions of the software for all operating systems.

As the Netpbm site says, these uncompressed image formats are very inefficient, and have no way of including information about the images. But they are the easiest format to read and write, being a "lowest common denominator" of image files. I don't recommended them as an archive format.
 
--  
Harry Parker  
Senior Systems Engineer  
Dialog Imaging Systems, Inc.

P.S.: Sending diffs is obsolete. It is a lousy way to publish software to users. Diffs do not work unless you have the exact same old version. If you want to see differences, do it yourself. That is the only way you really know that the difference between new software and YOUR source code is accurate. (If you want to see differences in context, I recommend using Kdiff3 for Windows or Linux, http://kdiff3.sourceforge.net/.)


----- Original Message ----
From: Johannes Schindelin <[hidden email]>
To: [hidden email]
Sent: Tuesday, January 9, 2007 6:39:07 AM
Subject: Re: Fixes to PNM/PGM/... reader/writer

Hi,

On Thu, 4 Jan 2007, Spencer Eugene Olson wrote:

> I've added stack support to the PNM/PGM/... reader/writer.  For the
> NETPBM format, stacks are simply stored as one fully formatted image
> (header and all) concatenated right after the other in the same file.  
> I use the PGM format quite often and hate having hundreds of files
> hanging around which are really only slices in stacks of data.

According to

    http://en.wikipedia.org/wiki/Portable_Pixmap_file_format

stacks are not part of the format. Which programs (apart from "cat") do
write such stacks?

Ciao,
Dscho

P.S.: I don't know about others, but IMHO changes to source code should be
posted as diffs, rather than complete files. It makes it easier to see
what actually changed (and there were more changes than just those
enclosed in BEGIN/END CHANGES blocks).






__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

[dscho]

Hi,

On Tue, 9 Jan 2007, Harry Parker wrote:

> Your email is a good example of why Wikipedia is a good place to start
> searching, but a bad place to end your search.

But I did not start the search in Wikipedia! Rather, I started with the
files you sent! And there is a link to the wikipedia article, which I then
thought to be authoritative.

> P.S.: Sending diffs is obsolete.

Uhm. No.

> It is a lousy way to publish software to users.

Not true. Besides, if you want people to _use_ them, why not send .class
files to begin with?

> Diffs do not work unless you have the exact same old version.

Not true at all. Besides, you are posting a proposed new version of a
plugin which is _included in ImageJ_.

> If you want to see differences, do it yourself.

Thank you very much for your help. I already did exactly that. But if
_you_ would have shown the diff, you would have realized that your code

        - changes white space,
        - changes import.* statements, and
        - is rather complicated for what it actually changes.

Hth,
Dscho

[Spencer Eugene Olson]

On Tuesday 09 January 2007 11:06, Johannes Schindelin wrote:

> Hi,
>
> On Tue, 9 Jan 2007, Harry Parker wrote:
> > Your email is a good example of why Wikipedia is a good place to start
> > searching, but a bad place to end your search.
>
> But I did not start the search in Wikipedia! Rather, I started with the
> files you sent! And there is a link to the wikipedia article, which I then
> thought to be authoritative.
>
> > P.S.: Sending diffs is obsolete.
>
> Uhm. No.
>
> > It is a lousy way to publish software to users.
>
> Not true. Besides, if you want people to _use_ them, why not send .class
> files to begin with?
>
> > Diffs do not work unless you have the exact same old version.
>
> Not true at all. Besides, you are posting a proposed new version of a
> plugin which is _included in ImageJ_.
>
> > If you want to see differences, do it yourself.
>
> Thank you very much for your help. I already did exactly that. But if
> _you_ would have shown the diff, you would have realized that your code
>
> - changes white space,
> - changes import.* statements, and
> - is rather complicated for what it actually changes.
>
> Hth,
> Dscho

I don't care whether people like diffs or not.  I can accommodate with no
complaints.  I actually prefer diffs, it's just that I was not sure of the
preferred format here (if there is one) on this list--the lack of repository
and especially the lack of "Here is how you submit patches" brought me to
this view.  I figured that if there were some here that preferred
diffs, these would certainly know how to do them, knowing the base
distribution that I specified.

With regards to the important part of my post, yes, stacked PAM (PNM, etc,
etc) images are defined by the format, although there are few of the netpbm
programs that use it.  Of course, the most blatant use of the stacked netpbm
format is in the programs 'pnmsplit', 'pamsplit', 'pampick', ....  Although
this is not the best format to work with (with respect to compression and so
on), it is certainly an easy format to use with the guarantee that my data is
not altered and easily accessible from various other programs of my own
making (c, c++, octave, ...).  I don't think that my changes were too
complicated, but that is just my opinion (especially in the writer).

Oh, and yes, you can actually store information about the image in netpbm.  
This can be done in the form of comments instead of some standard header.  
Just to set the record straight.

ciao,


Spencer

--
------------
Spencer E. Olson
[hidden email]
http://www.umich.edu/~olsonse/
PGP Fingerprint: A452 312C 73DC 41EF 47F3  EE73 A898 B528 BAD2 0D0F
(See www.pgp.net or pgp.mit.edu for public keys to encrypt messages for me)

user, n.:
        The word computer professionals use when they mean "idiot."
                        -- Dave Barry, "Claw Your Way to the Top"
                         
[I always thought "computer professional" was the phrase hackers used
when they meant "idiot."  Ed.]

[dscho]

Hi,

On Wed, 10 Jan 2007, Spencer Eugene Olson wrote:

> I don't care whether people like diffs or not.

That is okay. My last remark on this issue: if you want other people to
look at your work, make it most convenient for them.

> With regards to the important part of my post, yes, stacked PAM (PNM,
> etc, etc) images are defined by the format, although there are few of
> the netpbm programs that use it.

The home page says something about July 2000.

> I don't think that my changes were too complicated, but that is just my
> opinion (especially in the writer).

What I wanted to say is that your patch does not introduce the minimal
change to do what it tries to do. Maybe I did a poor job explaining that.

> Oh, and yes, you can actually store information about the image in
> netpbm.  This can be done in the form of comments instead of some
> standard header.

I did not find any documentation on the preferred way to specify the
bounding box, or alternatively the voxel dimensions. Could you enlighten
me as to what are the common methods?

Ciao,
Dscho

[dscho]

Hi,

On Thu, 11 Jan 2007, Johannes Schindelin wrote:

> On Wed, 10 Jan 2007, Spencer Eugene Olson wrote:
>
> > I don't think that my changes were too complicated, but that is just
> > my opinion (especially in the writer).
>
> What I wanted to say is that your patch does not introduce the minimal
> change to do what it tries to do. Maybe I did a poor job explaining
> that.

FWIW here is what I consider a concise, easy-to-review patch for the
reader part:

---

 ij/plugin/PGM_Reader.java |   18 +++++++++++++-----
 1 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/ij/plugin/PGM_Reader.java b/ij/plugin/PGM_Reader.java
index 5a86806..6eaacf0 100644
--- a/ij/plugin/PGM_Reader.java
+++ b/ij/plugin/PGM_Reader.java
@@ -68,6 +68,7 @@ public class PGM_Reader extends ImagePlus implements PlugIn {
     private boolean isColor;
     private boolean isBlackWhite;
     private int maxValue;
+    private ImageStack stack;
 
     public void run(String arg) {
         OpenDialog od = new OpenDialog("PBM/PGM/PPM Reader...", arg);
@@ -112,18 +113,28 @@ public class PGM_Reader extends ImagePlus implements PlugIn {
         tok.parseNumbers();
         tok.eolIsSignificant(true);
         tok.commentChar('#');
+ try {
+ while (true)
+ readSlice(tok, is);
+ } catch(IOException e) { /* end of file */ }
+ return stack;
+    }
+
+    private ImageStack readSlice(StreamTokenizer tok, InputStream is)
+    throws IOException {
         openHeader(tok);
+ if (stack == null)
+ stack = new ImageStack(width, height);
+
  //IJ.log("PGM_Reader: w="+width+",h="+height+",raw="+rawBits+",16bits="+sixteenBits+",color="+isColor+",b&w="+isBlackWhite+",max="+maxValue);
         
         if (!isColor && sixteenBits) { // 16-bit grayscale
             if (rawBits) {
                 ImageProcessor ip = open16bitRawImage(is, width, height);
-                ImageStack stack = new ImageStack(width, height);
                 stack.addSlice("", ip);
             return stack;
             } else {
                 ImageProcessor ip = open16bitAsciiImage(tok, width, height);
-                ImageStack stack = new ImageStack(width, height);
                 stack.addSlice("", ip);
             return stack;
             }
@@ -149,7 +160,6 @@ public class PGM_Reader extends ImagePlus implements PlugIn {
  } else
  pixels[i] = (byte) (0xff & (255 * (int) (0xff & pixels[i]) / maxValue));
  }
- ImageStack stack = new ImageStack(width, height);
  stack.addSlice("", ip);
  return stack;
  }
@@ -171,7 +181,6 @@ public class PGM_Reader extends ImagePlus implements PlugIn {
  b = (b * 255 / maxValue);
  pixels[i] = 0xFF000000 | r | g | b;
  }
- ImageStack stack = new ImageStack(width, height);
  stack.addSlice("", ip);
  return stack;
  }
@@ -203,7 +212,6 @@ public class PGM_Reader extends ImagePlus implements PlugIn {
  blue[i] = (short)(pixels[i*3+2]&0xffffff);
  }
  }
- ImageStack stack = new ImageStack(width, height);
  stack.addSlice("red", new ShortProcessor(width, height, red, null));
  stack.addSlice("green", new ShortProcessor(width, height, green, null));
  stack.addSlice("blue", new ShortProcessor(width, height, blue, null));

[Spencer Eugene Olson]

On Thursday 11 January 2007 04:10, Johannes Schindelin wrote:

> The home page says something about July 2000.

If you look at the documentation for the programs that do support stacks in
the netpbm package (pamsplit, pampick, pnmsplit, ...?), you will "Updated: 23
October 2005".  So it looks like it is not a retired feature. As stated, I
find the feature useful.

> > Oh, and yes, you can actually store information about the image in
> > netpbm.  This can be done in the form of comments instead of some
> > standard header.
>
> I did not find any documentation on the preferred way to specify the
> bounding box, or alternatively the voxel dimensions. Could you enlighten
> me as to what are the common methods?

I would not claim to know any "common methods" here.  I simply have used
inline comments in the netpbm header to record things that I want to have
automatically associated with the file (camera exposure, date, data type,
various physical parameters of an experiment, etc.) without the need for me
to take the time to find the correct log book and search through it.  It
stands to reason that if someone really wanted, they could have a set of
comment lines that looked like:
# boundingBox=a,b,c,d
and then just parse these comments.  If you look at the PNM_Writer.java that I
sent, you will notice that I added a "# label : <label>" type of comment.  I
had just this in mind, so that the PGM_Reader could set the slice label if it
found this comment.  I just didn't implement it in the reader yet.  

ciao,

Spencer


--
------------
Spencer E. Olson
[hidden email]
http://www.umich.edu/~olsonse/
PGP Fingerprint: A452 312C 73DC 41EF 47F3  EE73 A898 B528 BAD2 0D0F
(See www.pgp.net or pgp.mit.edu for public keys to encrypt messages for me)

user, n.:
        The word computer professionals use when they mean "idiot."
                        -- Dave Barry, "Claw Your Way to the Top"
                         
[I always thought "computer professional" was the phrase hackers used
when they meant "idiot."  Ed.]

[dscho]

Hi,

On Thu, 11 Jan 2007, Spencer Eugene Olson wrote:

> On Thursday 11 January 2007 04:10, Johannes Schindelin wrote:
>
> > > Oh, and yes, you can actually store information about the image in
> > > netpbm.  This can be done in the form of comments instead of some
> > > standard header.
> >
> > I did not find any documentation on the preferred way to specify the
> > bounding box, or alternatively the voxel dimensions. Could you enlighten
> > me as to what are the common methods?
>
> I would not claim to know any "common methods" here.

Okay, then let's have some precedent. And oh, let's load the information
into the Image Info and not forget about it. Likewise, PNM_Writer should
write it.

Ciao,
Dscho