Registration based ONLY on landmarks, from Java plugin - advice needed

[Kenneth Sloan-2]

I need to register images which have grossly different appearances, so much so that any method which uses pixel values is doomed to failure.

Currently, I use "Align Image by line ROI" and depend on the user to provide matching lineROIs.  I have a trivial Java plugin which collects a stack of images and guides the user through multiple pair-wise registrations.

I use multiple pair-wise registrations (using the same "target" image for each pair) because it is generally NOT possible to find landmarks which are common to ALL of the various images.  Typically, the "target" image is clearest, but the matching landmarks in the various "source" images are very variable.  We typically use the same pairs of lines for 75% of the registrations, but need to use   different lineROIs the rest of the time.

The current transformation is limited to Translation, Rotation, and Uniform Scaling.

It's clunky - but it does exactly what I have needed, so far.

Now, I find myself in the position of needing to deal with images which have a more complicated transformation (mostly because of different optics and imaging conditions, partly because of small local warping of the material being imaged).  "TRS" is not quite good enough for these.  So, I'm looking for options that allow for more than 2 landmarks.

So, the requirements are:
        *based on landmarks only - pixel values must be ignored
        *moderate (very controlled) warping, going beyond TRS

At first glance, "Moving Least Squares" appears to be the leading candidate.  Anything that looks at pixel values produces very weird results!

I'm looking for advice and guidance.  First - is "Moving Least Squares" my best choice?  If not, feel free to suggest something better.  Second - does anyone have sample Java code (not javascript, not macro) for invoking your method of choice from a Java plugin.

A stack-based solution would be preferable, but it would be necessary to cope with different numbers of landmarks in each slice (because what is "visibly salient" differs across images).  But, multiple pair-wise registrations is fine - it's what we do now.

I will wrap the  chosen method in a Java plugin to handle the details of loading images, guiding the user in locating matching sets of landmarks, and packaging up the results (as needed by the remainder of my pipeline).  Typically, I write the resulting set of registered images to a folder as an Image Sequence, with specific filenames required by the application.  In principle, the ideal solution would be something that could be dropped in in place of:

                        //  Ask User to define matching lines
                         ...
                        // images have matching lineROIs
                        IJ.run("Align Image by line ROI",
                               "source=[" +
                               sourceLabel +
                               "] target=[" +
                               targetLabel +
                               "] scale rotate");

I would prefer to avoid a control panel popping up - but if this is necessary, I can deal with it.

My Photoshop-loving customer currently uses "PuppetWarp".

--
Kenneth Sloan
[hidden email]
Vision is the art of seeing what is invisible to others.

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Saalfeld, Stephan]

Check out

https://imagej.net/Landmark_Correspondences

Best,
Stephan

On Fri, 2020-11-27 at 12:45 -0600, Kenneth Sloan wrote:

> I need to register images which have grossly different appearances,
> so much so that any method which uses pixel values is doomed to
> failure.
>
> Currently, I use "Align Image by line ROI" and depend on the user to
> provide matching lineROIs.  I have a trivial Java plugin which
> collects a stack of images and guides the user through multiple pair-
> wise registrations.
>
> I use multiple pair-wise registrations (using the same "target" image
> for each pair) because it is generally NOT possible to find landmarks
> which are common to ALL of the various images.  Typically, the
> "target" image is clearest, but the matching landmarks in the various
> "source" images are very variable.  We typically use the same pairs
> of lines for 75% of the registrations, but need to use   different
> lineROIs the rest of the time.
>
> The current transformation is limited to Translation, Rotation, and
> Uniform Scaling.
>
> It's clunky - but it does exactly what I have needed, so far.
>
> Now, I find myself in the position of needing to deal with images
> which have a more complicated transformation (mostly because of
> different optics and imaging conditions, partly because of small
> local warping of the material being imaged).  "TRS" is not quite good
> enough for these.  So, I'm looking for options that allow for more
> than 2 landmarks.
>
> So, the requirements are:
>         *based on landmarks only - pixel values must be ignored
>         *moderate (very controlled) warping, going beyond TRS
>
> At first glance, "Moving Least Squares" appears to be the leading
> candidate.  Anything that looks at pixel values produces very weird
> results!
>
> I'm looking for advice and guidance.  First - is "Moving Least
> Squares" my best choice?  If not, feel free to suggest something
> better.  Second - does anyone have sample Java code (not javascript,
> not macro) for invoking your method of choice from a Java plugin.
>
> A stack-based solution would be preferable, but it would be necessary
> to cope with different numbers of landmarks in each slice (because
> what is "visibly salient" differs across images).  But, multiple
> pair-wise registrations is fine - it's what we do now.
>
> I will wrap the  chosen method in a Java plugin to handle the details
> of loading images, guiding the user in locating matching sets of
> landmarks, and packaging up the results (as needed by the remainder
> of my pipeline).  Typically, I write the resulting set of registered
> images to a folder as an Image Sequence, with specific filenames
> required by the application.  In principle, the ideal solution would
> be something that could be dropped in in place of:
>
>                         //  Ask User to define matching lines
>                          ...
>                         // images have matching lineROIs
>                         IJ.run("Align Image by line ROI",
>                                "source=[" +
>                                sourceLabel +
>                                "] target=[" +
>                                targetLabel +
>                                "] scale rotate");
>
> I would prefer to avoid a control panel popping up - but if this is
> necessary, I can deal with it.
>
> My Photoshop-loving customer currently uses "PuppetWarp".
>
> --
> Kenneth Sloan
> [hidden email]
> Vision is the art of seeing what is invisible to others.
>
> --
> ImageJ mailing list:
> https://urldefense.com/v3/__http://imagej.nih.gov/ij/list.html__;!!Eh6p8Q!XJxA65Y0pUYUzJD_ojyZ-P6M_rHWyGcm9FhuCGdQqhKtZ1-dqCN0ZEp_N5oroCN1sr4f$
>  

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Kenneth Sloan-2]

Will do.

First question: how is Transform>Landmark Correspondences related to Registration>Moving Least Squares?

--
Kenneth Sloan
[hidden email]
Vision is the art of seeing what is invisible to others.

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Saalfeld, Stephan]

Different implementation of the MLS method, faster and more robust
rendering, interpolation, least squares models.

On Fri, 2020-11-27 at 22:04 -0600, Kenneth Sloan wrote:

> Will do.
>
> First question: how is Transform>Landmark Correspondences related to
> Registration>Moving Least Squares?
>
> --
> Kenneth Sloan
> [hidden email]
> Vision is the art of seeing what is invisible to others.
>
> --
> ImageJ mailing list:
> https://urldefense.com/v3/__http://imagej.nih.gov/ij/list.html__;!!Eh6p8Q!V6bThNF00uGSni3mwp79f-XLIDSNJ6QDAyICxGCuf30Zcvs0mLP0YqXgHuSAcfubuwZ9$
>  

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html