Any known curve fitting oddness in Java VM > 1.60.33?

[Jeff Hardin]

Hi folks,
We've notice that the curve fitting (or something related to curve fitting) in my FRAP Profiler plugin is broken in Java VMs beyond 1.6.0.33 on either OS X or WinXP (haven't tried Win7, but guessing there as well). The plugin works great on a Mac with VMA 1.6.0.33 or lower. You can get it here:

http://worms.zoology.wisc.edu/research/4d/4d.html#frap

Were there any changes to floating point operations or anything else that might have generally broken ImageJ curve fitting routines in Java 1.6 VM updates that anyone is aware of?  I will try troubleshooting further, but am hoping someone may have noticed this in their plugin and found a fix.

Cheers,
Jeff
----------------------------------------
Jeff Hardin
Professor and Chair, Department of Zoology
Director, Biology Core Curriculum
University of Wisconsin
1117 W. Johnson St.
Madison, WI 53706
voice: [phone]
fax: [phone]
email: [hidden email]
web page: http://worms.zoology.wisc.edu

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Jeff Hardin]

Hi all,
A follow up here. What is triggering the failure of the curve fit are the following conditions:
(1) Use  "*User defined*" as the algorithm.
(2) Use y = a*(1-exp(-b*x)) as the function [this constrains the curve to pass through the origin, rather than having the y intercept added as in the ImageJ built-in curve fit]
(3) If one just runs it without the "Show settings" checkbox checked, ImageJ appears to populate a and b with zeroes.
(4) One then gets a straight line passing the through origin and the last point of the data set, rather than an exponential recovery.
(5) This also happens if one uses "Show settings" but leaves a and b as zeroes.

One can avoid this by either (a) putting something in for a a & b like -1 for a and 1 for b in the Settings dialogue for the initial parameters; or (b) by populating the param list (a double[]) when calling CurveFitter without showing the Settings dialogue. This is all unnecessary under Java VMs 1.6.033 or prior.

For our purposes, we can just populate the params list in this way and also add an option to manually set them via the "Show settings" dialogue, but this does seem to be something about how the newer JMVs are handling the math calls within CurveFitter, which worries me a bit. We'll side by side testing to see of the fits are identical, which would suggest convergence of the fitting algorithm is what is failing .

Does anyone have any creative ideas about this?

Cheers,
Jeff

 for data sets whose first point is the origin that one must provide initial parameters in the CurveFitter via checking the "Show Setting" checkbox and then supplying numbers

On Sep 23, 2013, at 11:00 PM, IMAGEJ automatic digest system <[hidden email]> wrote:

> Date:    Mon, 23 Sep 2013 14:07:25 -0500
> From:    Jeff Hardin <[hidden email]>
> Subject: Any known curve fitting oddness in Java VM > 1.60.33?
>
> Hi folks,
> We've notice that the curve fitting (or something related to curve fitting) in my FRAP Profiler plugin is broken in Java VMs beyond 1.6.0.33 on either OS X or WinXP (haven't tried Win7, but guessing there as well). The plugin works great on a Mac with VMA 1.6.0.33 or lower. You can get it here:
>
> http://worms.zoology.wisc.edu/research/4d/4d.html#frap
>
> Were there any changes to floating point operations or anything else that might have generally broken ImageJ curve fitting routines in Java 1.6 VM updates that anyone is aware of?  I will try troubleshooting further, but am hoping someone may have noticed this in their plugin and found a fix.
>
> Cheers,
> Jeff

----------------------------------------
Jeff Hardin
Professor and Chair, Department of Zoology
Director, Biology Core Curriculum
University of Wisconsin
1117 W. Johnson St.
Madison, WI 53706
voice: [phone]
fax: [phone]
email: [hidden email]
web page: http://worms.zoology.wisc.edu


--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Jeff,

yes, I can reproduce the problem of fitting y = a*(1-exp(-b*x)) with some data, e.g. these ones:
0 0
1 7
2 8.00
3 12
4 15
5 14

I don't think it is a problem of the Java VM, I get exactly the same result with Java 1.5_09. The ImageJ CurveFitter starts with random numbers, and I don't think that anything has changed in the Java Random Number Generator.  It may easily happen, however, that a very small change of the data can cause a different result - the Simplex algorithm (like many minimization algorithms) behaves a bit like a chaotic system.

The ImageJ version does make a difference; I have done a few changes to the curve fitting in ImageJ 1.46f.  So far, all other fits that I tried got better with my changes, but it seems that it got worse for your function.  I don't understand why - I'll try to find out what goes wrong.

It would be easy to add a built-in 'exponential recovery (no offset)' that is essentially guaranteed to give the correct result for any set of reasonable data, because it would be a one-parameter fit (the 'a' parameter can be eliminated in the fit procedure).  Would that help?


Michael
________________________________________________________________
On Sep 24, 2013, at 14:51, Jeff Hardin wrote:

> Hi all,
> A follow up here. What is triggering the failure of the curve fit are the following conditions:
> (1) Use  "*User defined*" as the algorithm.
> (2) Use y = a*(1-exp(-b*x)) as the function [this constrains the curve to pass through the origin, rather than having the y intercept added as in the ImageJ built-in curve fit]
> (3) If one just runs it without the "Show settings" checkbox checked, ImageJ appears to populate a and b with zeroes.
> (4) One then gets a straight line passing the through origin and the last point of the data set, rather than an exponential recovery.
> (5) This also happens if one uses "Show settings" but leaves a and b as zeroes.
>
> One can avoid this by either (a) putting something in for a a & b like -1 for a and 1 for b in the Settings dialogue for the initial parameters; or (b) by populating the param list (a double[]) when calling CurveFitter without showing the Settings dialogue. This is all unnecessary under Java VMs 1.6.033 or prior.
>
> For our purposes, we can just populate the params list in this way and also add an option to manually set them via the "Show settings" dialogue, but this does seem to be something about how the newer JMVs are handling the math calls within CurveFitter, which worries me a bit. We'll side by side testing to see of the fits are identical, which would suggest convergence of the fitting algorithm is what is failing .
>
> Does anyone have any creative ideas about this?
>
> Cheers,
> Jeff
>
> for data sets whose first point is the origin that one must provide initial parameters in the CurveFitter via checking the "Show Setting" checkbox and then supplying numbers
>
> On Sep 23, 2013, at 11:00 PM, IMAGEJ automatic digest system <[hidden email]> wrote:
>
>> Date:    Mon, 23 Sep 2013 14:07:25 -0500
>> From:    Jeff Hardin <[hidden email]>
>> Subject: Any known curve fitting oddness in Java VM > 1.60.33?
>>
>> Hi folks,
>> We've notice that the curve fitting (or something related to curve fitting) in my FRAP Profiler plugin is broken in Java VMs beyond 1.6.0.33 on either OS X or WinXP (haven't tried Win7, but guessing there as well). The plugin works great on a Mac with VMA 1.6.0.33 or lower. You can get it here:
>>
>> http://worms.zoology.wisc.edu/research/4d/4d.html#frap
>>
>> Were there any changes to floating point operations or anything else that might have generally broken ImageJ curve fitting routines in Java 1.6 VM updates that anyone is aware of?  I will try troubleshooting further, but am hoping someone may have noticed this in their plugin and found a fix.
>>
>> Cheers,
>> Jeff
>
> ----------------------------------------
> Jeff Hardin
> Professor and Chair, Department of Zoology
> Director, Biology Core Curriculum
> University of Wisconsin
> 1117 W. Johnson St.
> Madison, WI 53706
> voice: [phone]
> fax: [phone]
> email: [hidden email]
> web page: http://worms.zoology.wisc.edu
>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html