Image synchronization between ImageJ2 and ImageJ 1.x

[Mark Hiner]

Hi everyone,

As many of you know, ImageJ2 features a redesigned data model built on the
ImgLib2 image processing library. This is a "greenfield" redesign from
ImageJ 1.x. However, ImageJ2 ships with a compatibility later known as ImageJ
Legacy <https://github.com/imagej/imagej-legacy> which is responsible for
enabling interoperability between ImageJ2 and ImageJ1 data structures. The
idea is that you can write an ImageJ2 command which operates on ImageJ2
data structures, which -- when executed from within the ImageJ Legacy UI --
transparently synchronizes the results with an ImageJ1 version of the data.
In this way, ImageJ1 plugins and ImageJ2 commands can be freely mixed and
matched.

Unfortunately, initial versions of this functionality were too harmful to
performance (in both time and space). Hence, the "Enable ImageJ2 data
structures" option of Edit > Options > ImageJ2 is now disabled by default,
which limits this interoperability.

Furthermore, a recent discussion on fiji-devel
<https://groups.google.com/forum/#%21topic/fiji-devel/21cGi6lnMz0> exposed
further limitations of the current design
<https://github.com/imagej/imagej-legacy/blob/af0f1679fd341ccc704514750b2a618aef841b92/src/main/java/net/imagej/legacy/translate/PlaneHarmonizer.java>:
when plugins replace the components of an ImagePlus (e.g., the plane
objects), synchronization is broken.

After internal discussion, we believe we can achieve much better
synchronization, overcoming these limitations with little to no impact on
performance.

*For those interested in technical details:* see below for a brief outline
of our current plan.

*For others:* we just wanted to let you know about these current
limitations, and that improvements are coming.

The tentative plan is to use a bidirectional wrapping scheme: all queries
will pass through to the actual state of the wrapped object, removing the
need for harmonization. If objects need to be calculated/derived from the
wrapped object, it will be done lazily, recomputing and caching any state
flagged as dirty.

This approach should work at least in the ImageJ1 -> ImageJ2 direction.
There may be obstacles for the ImageJ2 -> ImageJ1 direction, due to the
ImageJ1 data model being less expressive, but we will do our best to make
any limitations of the synchronization intuitive and clear in the UI.

If anyone has questions or technical feedback, they are welcome as always!

Regards,
Mark Hiner
ImageJ2 development team

P.S. For those interested, we also considered two other approaches:

1) Using IJ1-patcher <https://github.com/imagej/ij1-patcher>, we could
intercept all calls modifiying the ImageProcessor, stack, or pixels. In all
of these cases we would have to be able to figure out what the
corresponding Dataset is (if any), and check if the object's plane
references are different from the Dataset's, then re-harmonize if needed.

But this method would be more fragile, and likely have long-term
maintainability problems due to the reliance on javassist to intercept
method calls.

2) Drop the LegacyImageMap completely and rely entirely on the Converter
<https://github.com/imagej/imagej-legacy/issues/95> framework. This will
allow plugins/scripts/etc... with ImagePlus or Dataset @Parameters to
consume either, effectively making them interchangeable to the framework.
All converted objects would be transient, existing only for the life of the
command (or as long as the reference is held in the case where the
parameter is also a return object).

This method would be great for some applications, but would not facilitate
automatic conversion in API involving Datasets or ImagePluses (e.g., extra
work would still be needed to make ImageDisplayService#getActiveDisplay
know about the active ImagePlus).

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html