Re: a tutorial on a custom image format file reader / writer

Posted by ctrueden on Apr 15, 2008; 6:37pm
URL: http://imagej.273.s1.nabble.com/a-tutorial-on-a-custom-image-format-file-reader-writer-tp3696492p3696495.html

Hi,

I am very interested in helping to improve the ImageJ I/O API. It would be
nice if ImageJ's file format I/O could -- at some level -- harness the
general framework we have created for Bio-Formats. Even though the
Bio-Formats project itself is geared toward life sciences file formats,
there is nothing in the framework that fundamentally limits it that way.

As I alluded to earlier, I think it would be really cool to expand ImageJ's
I/O architecture a bit to more explicitly identify which plugins fulfill an
"input reader" or "output writer" role. That way, ImageJ could include an
interface specifically for toggling and/or reordering which readers/writers
get called in which order for various file formats.

Obviously, in developing Bio-Formats I have given these issues a lot of
thought, and Bio-Formats has significant chunks of logic for file type
identification and modular extraction of pixels and metadata. It is not
perfect, but maybe we can apply some portion of the BF solution to ImageJ.

My basic suggestion would be to have a Java interface, a subinterface of
PlugIn, that represents an image format reader. When ImageJ scans for
plugins, it could note which ones implement this interface, and
automatically register them with its File/Open command, reducing or
eliminating the need for a manually edited HandleExtraFileTypes. You could
do something similar, but simpler, for writers, by having a PlugIn
subinterface for image format writers.

The hard part is deciding whether a given file is of a particular format.
For many formats you can just use the file extension, but for some formats
it is insufficient. In Bio-Formats we have two levels of detection: a "fast"
way from just the filename, and a "slow" way that opens up the file for
random access and scans it. The logic is pretty smart and only uses the slow
way 1) if the fast way fails, and 2) if we are in a situation where it is
allowed and appropriate.

If we did this, then we could have a GUI for toggling and reordering which
formats are called in which order. I have created something like this for
Bio-Formats -- allowing toggling of each reader individually -- as part of a
"Bio-Formats Configuration" dialog, that will be present in the next
release, and can be seen in its current form using the prerelease build at <
http://skyking.microscopy.wisc.edu/curtis/jar/loci_tools.jar>. (I am also
planning to add an upgrade feature similar to ImageJ's that allows you to
upgrade BF to the latest release or daily build.)

Even if we do not end up doing anything like I've described here, with this
format toggling feature we could now move Bio-Formats up in the priority
list in HandleExtraFileTypes. This solves the problem of having multiple
plugins installed for handling the same file format -- e.g., you can have
both Bio-Formats and the LSM_Reader, and just toggle Bio-Formats's LSM
support in order to test files one way or the other.

This is probably the kind of thing that would be easier to hash out by
getting interested parties together for a phone chat or even in person
meeting. I am insanely busy through Friday (apologies to Wayne, Patrick and
others for being slow in responding to issues), but could invest some time
next week to discuss all of this further, if there is any interest.

-Curtis

On Mon, Apr 14, 2008 at 3:33 PM, Wayne Rasband <[hidden email]> wrote:

> You are probably seeing this problem because you are using the LSM_Toobox,
> which opens files in a separate thread. Instead, use the LSM_Reader plugin,
> which does not return until it has finished reading the file. You will need
> to remove LSM_Toolbox.jar from the plugins folder since the
> Handle_Extra_File_Types plugin uses it if it is present. Or upgrade to the
> latest version of Handle_Extra_File_Types at <
> http://rsb.info.nih.gov/ij/plugins/file-handler.html>, which uses the
> LSM_Reader if it is present.
>
> -wayne
>
>
>
> On Apr 14, 2008, at 1:52 PM, Francis Burton wrote:
>
>  At 12:16 14/04/08 -0500, "Curtis Rueden" <[hidden email]> wrote:
> >
> > > Thanks for this guide. I think it is very useful currently, but
> > > hopefully in
> > > the future we can improve the ImageJ I/O plugin architecture to reduce
> > > the
> > > need for many of these steps. I have some ideas on how to do so, but
> > > have
> > > not yet had time to work on them.
> > >
> >
> > Recently I have been playing around with getting ImageJ to do
> > conversions
> > between file formats via command line invocation and macros. (Many
> > thanks
> > to Patrick Pirrotte for all his help with reading LSM linescan images
> > with
> > his useful LSMReader/LSMToolbox.)
> >
> > One outstanding problem concerns the threaded nature of file reading. In
> > Patrick's own words:
> >
> > "The problem is that somehow in macro headless mode (no gui) opening and
> > saving occur in other threads. This means that the LSM_Reader is not
> > finished opening the image that the macro already tries to save the
> > files.
> > The results are sometimes weird."
> >
> > This has necessitated adding a delay instruction in the macro, to allow
> > reading to finish before any attempt to write the converted data file.
> >
> > Obviously this is less than ideal because this delay will vary depending
> > on the speed of the computer and other running processes. It makes it
> > impossible to write a portable solution unless one incorporates a long
> > enough delay to cover all existing computers (in practice that is quite
> > a few seconds' delay!).
> >
> > Would the solution be to add another command to ImageJ similar to
> > wait(...)
> > that waits for all i/o to complete before continuing execution (of the
> > macro)?
> >
> > I am not in a position to judge if this is a sensible proposition, but
> > if
> > it is I would be very pleased if this was incorporated into the program.
> >
> > Cheers,
> > Francis
> >
> > --
> > Dr Francis L Burton,          |  [hidden email]
> > West Medical Building,        |
> > University of Glasgow,        |  Tel [phone]
> > Glasgow, G12 8QQ, Scotland.   |  Fax [phone]
> >
> >