interpolation/extrapolation of array values

[Rainer M. Engel]

Hey there..

I know a little bit about CurveFitting functions, but I'm not sure how
to interpolate and/or extrapolate missing values of an array, where I
want to keep all given values and only fill gaps.

#,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19
OK, the example above is very simple :)

Another example, where # marks an entry/gap that needs to be filled.
#,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8

What would be a good starting point for this?

Best regards,
Rainer

--
###################################
Rainer M. Engel
Pichelsdorfer Str. 143
13595 Berlin
###################################

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Rainer,

you can simply add an 'x' axis and then omit the data that you don't have. E.g. for the 2nd data set you get

2,7.4
3,6.3
4,6.1
5,5.8
7,5.7
8,5.9
9,5.8
13,5.6
14,5.7
15,5.5
17,5.6
18,5.8

Then use the ImageJ Curve Fitter (Analyze>Tools>Curve Fitting) and try which type of function provides a reasonable fit. For this set, a 2nd-order to 4th-order polynomial seems to be appropriate.  Look at the graphs, and with some knowledge of the typical noise and typical shape of your data sets you should decide.
If you know what function to use, you can write a macro for it
  http://rsb.info.nih.gov/ij/developer/macro/functions.html#Fit

Of course, extrapolation of such noisy data is very difficult. With these data, you will get very different values for the x=0 and x=1 points depending on which type of function you choose. Polynomial extrapolation usually tends to overshoot when extrapolating, especially if the degree of the polynomial is rather high, so use the lowest degree that can reasonably fit your data.

If you want to use high-order polynomials (more than 4th or 5th order), the polynomial fitting procedure tends to become numerically unstable. In that case, it is better to use x values symmetric around zero, e.g. -9 to +9 instead of 0 to 18, then you can even use 8th order (but this makes no sense with the current data).

Michael
________________________________________________________________
On Jul 29, 2015, at 10:17, Rainer M. Engel wrote:

> Hey there..
>
> I know a little bit about CurveFitting functions, but I'm not sure how
> to interpolate and/or extrapolate missing values of an array, where I
> want to keep all given values and only fill gaps.
>
> #,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19
> OK, the example above is very simple :)
>
> Another example, where # marks an entry/gap that needs to be filled.
> #,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8
>
> What would be a good starting point for this?
>
> Best regards,
> Rainer

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Herbie]

Rainer.

as your previous question, this again concerns 1D-signal processing not
image processing ...

The main question is:
        What do you expect from the desired processing?
...because there are infinite ways of performing the very task.

In general, curve fitting requires a model and it must be reasonably
founded. If you don't have an idea of the type of function, things
become more or less dubious.

In order to get an idea of the problems, you may have a look at the
chapter "Interpolation and Extrapolation" of "Numerical Recipes".

If you want a curve to exactly pass through existing points you may have
a look at spline interpolation, but spline extrapolation is generally
not convincing.

Either you search for corresponding algorithms and code yourself or for
one of the many software packages for 1D data processing. This may be
Origin, Kaleidagraph, etc. or perhaps even Excel.

Here are the rounded number I get from your example:
7.4, 6.75, 6.3, 6.1, 5.8, 5.63, 5.7, 5.9, 5.8, 5.64, 5.54, 5.51, 5.6,
5.7, 5.5, 5.46, 5.6, 5.8;
(no extrapolation)

HTH

Herbie

:::::::::::::::::::::::::::::::::::::::::::::
Am 29.07.15 um 10:17 schrieb Rainer M. Engel:

> Hey there..
>
> I know a little bit about CurveFitting functions, but I'm not sure how
> to interpolate and/or extrapolate missing values of an array, where I
> want to keep all given values and only fill gaps.
>
> #,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19
> OK, the example above is very simple :)
>
> Another example, where # marks an entry/gap that needs to be filled.
> #,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8
>
> What would be a good starting point for this?
>
> Best regards,
> Rainer
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Rainer M. Engel]

Dear Herbie and Michael,

thank you for your tips and explanation. I wanted to know a way to do
extrapolation and your support was very helpful.

I'll have to see how it works on real projects.

Below is some macro code with functions to ease this kind of task. At
least it did it for me..

Best regards,
Rainer



// COPY - PASTEFIX -----------------------------------------------------
print("\\Clear");
requires("1.49u");
print("Valid values..");
a1 =
newArray("#","#",7.4,"#",6.3,6.1,5.8,"#",5.7,5.9,5.8,"#","#","#",5.6,5.7,5.5,"#",5.6,5.8);

x = numberValidSlots(a1);
y = stripEmptySlots(a1);

xR = Array.getSequence(a1.length);

for (i=0; i<x.length; i++) {
        print(x[i]+" / "+y[i]);
}

print("");
print("Filled gaps.. (inter-/extrapolation)");

a2 = getCurveFittingRange(12, x, y, xR);
a3 = mixCurvesFillGaps(a1, a2);

for (i=0; i<a1.length; i++){
        print(i+" | "+a1[i]+" / "+a3[i]);
}


//
F-U-N-C-T-I-O-N-S---------------------------------------------------------------------
function mixCurvesFillGaps(srcArray, fitArray) {
        string = "";
        for (i=0; i<srcArray.length; i++) {
                if (srcArray[i] != "#") {
                        string = string+srcArray[i]+"|";
                } else {
                        string = string+fitArray[i]+"|";
                }
        }
        return split(string,"|");
}

function numberValidSlots(array) {
        string = "";
        for (i=0; i<array.length; i++) {
                if (array[i] != "#") {
                        string = string+i+"|";
                }
        }
        return split(string,"|");
}

function stripEmptySlots(array) {
        string = "";
        for (i=0; i<array.length; i++) {
                if (array[i] != "#") {
                        string = string+array[i]+"|";
                }
        }
        return split(string,"|");
}

function getCurveFittingRange(equation, xpoints, ypoints, xRange) {
        Fit.doFit(equation, xpoints, ypoints);
        cfr = "";
        for (i=0; i<xRange.length; i++){
                cfr = cfr+parseFloat(Fit.f(xRange[i]))+"|";
        }
        return split(cfr, "|");
}
// END - PASTEFIX -----------------------------------------------------




Am 29.07.2015 um 11:16 schrieb Herbie:

> Rainer.
>
> as your previous question, this again concerns 1D-signal processing not
> image processing ...
>
> The main question is:
>     What do you expect from the desired processing?
> ...because there are infinite ways of performing the very task.
>
> In general, curve fitting requires a model and it must be reasonably
> founded. If you don't have an idea of the type of function, things
> become more or less dubious.
>
> In order to get an idea of the problems, you may have a look at the
> chapter "Interpolation and Extrapolation" of "Numerical Recipes".
>
> If you want a curve to exactly pass through existing points you may have
> a look at spline interpolation, but spline extrapolation is generally
> not convincing.
>
> Either you search for corresponding algorithms and code yourself or for
> one of the many software packages for 1D data processing. This may be
> Origin, Kaleidagraph, etc. or perhaps even Excel.
>
> Here are the rounded number I get from your example:
> 7.4, 6.75, 6.3, 6.1, 5.8, 5.63, 5.7, 5.9, 5.8, 5.64, 5.54, 5.51, 5.6,
> 5.7, 5.5, 5.46, 5.6, 5.8;
> (no extrapolation)
>
> HTH
>
> Herbie
>
> :::::::::::::::::::::::::::::::::::::::::::::
> Am 29.07.15 um 10:17 schrieb Rainer M. Engel:
>> Hey there..
>>
>> I know a little bit about CurveFitting functions, but I'm not sure how
>> to interpolate and/or extrapolate missing values of an array, where I
>> want to keep all given values and only fill gaps.
>>
>> #,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19
>> OK, the example above is very simple :)
>>
>> Another example, where # marks an entry/gap that needs to be filled.
>> #,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8
>>
>> What would be a good starting point for this?
>>
>> Best regards,
>> Rainer
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Herbie]

Rainer,

be careful with extrapolation.

Michael has pointed out that extrapolation with polynomial fits isn't
easy and mostly prone to considerable errors.

Without any knowledge about the signal generating process, i.e. the
mathematics behind it, extrapolation by curve fitting is a pure guess.

I'm not sure whether the first parameter you use in
        Fit.doFit(equation, xpoints, ypoints);
is suitable: "12 -> Gaussian"
Do you have a clue that your data should follow a Gaussian?
If not, see above...

HTH

Herbie

:::::::::::::::::::::::::::::::::::::::::::::
Am 29.07.15 um 13:58 schrieb Rainer M. Engel:

> Dear Herbie and Michael,
>
> thank you for your tips and explanation. I wanted to know a way to do
> extrapolation and your support was very helpful.
>
> I'll have to see how it works on real projects.
>
> Below is some macro code with functions to ease this kind of task. At
> least it did it for me..
>
> Best regards,
> Rainer
>
>
>
> // COPY - PASTEFIX -----------------------------------------------------
> print("\\Clear");
> requires("1.49u");
> print("Valid values..");
> a1 =
> newArray("#","#",7.4,"#",6.3,6.1,5.8,"#",5.7,5.9,5.8,"#","#","#",5.6,5.7,5.5,"#",5.6,5.8);
>
> x = numberValidSlots(a1);
> y = stripEmptySlots(a1);
>
> xR = Array.getSequence(a1.length);
>
> for (i=0; i<x.length; i++) {
> print(x[i]+" / "+y[i]);
> }
>
> print("");
> print("Filled gaps.. (inter-/extrapolation)");
>
> a2 = getCurveFittingRange(12, x, y, xR);
> a3 = mixCurvesFillGaps(a1, a2);
>
> for (i=0; i<a1.length; i++){
> print(i+" | "+a1[i]+" / "+a3[i]);
> }
>
>
> //
> F-U-N-C-T-I-O-N-S---------------------------------------------------------------------
> function mixCurvesFillGaps(srcArray, fitArray) {
> string = "";
> for (i=0; i<srcArray.length; i++) {
> if (srcArray[i] != "#") {
> string = string+srcArray[i]+"|";
> } else {
> string = string+fitArray[i]+"|";
> }
> }
> return split(string,"|");
> }
>
> function numberValidSlots(array) {
> string = "";
> for (i=0; i<array.length; i++) {
> if (array[i] != "#") {
> string = string+i+"|";
> }
> }
> return split(string,"|");
> }
>
> function stripEmptySlots(array) {
> string = "";
> for (i=0; i<array.length; i++) {
> if (array[i] != "#") {
> string = string+array[i]+"|";
> }
> }
> return split(string,"|");
> }
>
> function getCurveFittingRange(equation, xpoints, ypoints, xRange) {
> Fit.doFit(equation, xpoints, ypoints);
> cfr = "";
> for (i=0; i<xRange.length; i++){
> cfr = cfr+parseFloat(Fit.f(xRange[i]))+"|";
> }
> return split(cfr, "|");
> }
> // END - PASTEFIX -----------------------------------------------------
>
>
>
>
> Am 29.07.2015 um 11:16 schrieb Herbie:
>> Rainer.
>>
>> as your previous question, this again concerns 1D-signal processing not
>> image processing ...
>>
>> The main question is:
>>      What do you expect from the desired processing?
>> ...because there are infinite ways of performing the very task.
>>
>> In general, curve fitting requires a model and it must be reasonably
>> founded. If you don't have an idea of the type of function, things
>> become more or less dubious.
>>
>> In order to get an idea of the problems, you may have a look at the
>> chapter "Interpolation and Extrapolation" of "Numerical Recipes".
>>
>> If you want a curve to exactly pass through existing points you may have
>> a look at spline interpolation, but spline extrapolation is generally
>> not convincing.
>>
>> Either you search for corresponding algorithms and code yourself or for
>> one of the many software packages for 1D data processing. This may be
>> Origin, Kaleidagraph, etc. or perhaps even Excel.
>>
>> Here are the rounded number I get from your example:
>> 7.4, 6.75, 6.3, 6.1, 5.8, 5.63, 5.7, 5.9, 5.8, 5.64, 5.54, 5.51, 5.6,
>> 5.7, 5.5, 5.46, 5.6, 5.8;
>> (no extrapolation)
>>
>> HTH
>>
>> Herbie
>>
>> :::::::::::::::::::::::::::::::::::::::::::::
>> Am 29.07.15 um 10:17 schrieb Rainer M. Engel:
>>> Hey there..
>>>
>>> I know a little bit about CurveFitting functions, but I'm not sure how
>>> to interpolate and/or extrapolate missing values of an array, where I
>>> want to keep all given values and only fill gaps.
>>>
>>> #,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19
>>> OK, the example above is very simple :)
>>>
>>> Another example, where # marks an entry/gap that needs to be filled.
>>> #,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8
>>>
>>> What would be a good starting point for this?
>>>
>>> Best regards,
>>> Rainer
>>>
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Rainer, Herbie,

for the current data set there is almost no difference between "Exponential with Offset" and "Gaussian" (which actually is a Gaussian with offset). That's because the maximum of the Gaussian is far from the range of the actual data, at large negative x. So, if you have some kind of exponentially decaying function, in the current case both fit functions will do. For other data sets, such as essentially the same data with different noise, there will be a significant difference, because the fit might find a suitable Gaussian with its maximum much closer to x=0.

So, if you just expect a kind of exponential decrease with offset or a Gaussian with its maximum always at very negative x (far from the data points supplied), better use "Exponential with Offset".

BTW, the 'equation' of Fit.doFit also accepts String arguments like "Gaussian"; this would make the code a bit more readable. Otherwise, I like your macro, obviously you know the macro language really well!

Michael
________________________________________________________________
On Jul 29, 2015, at 15:40, Herbie wrote:

> Rainer,
>
> be careful with extrapolation.
>
> Michael has pointed out that extrapolation with polynomial fits isn't easy and mostly prone to considerable errors.
>
> Without any knowledge about the signal generating process, i.e. the mathematics behind it, extrapolation by curve fitting is a pure guess.
>
> I'm not sure whether the first parameter you use in
> Fit.doFit(equation, xpoints, ypoints);
> is suitable: "12 -> Gaussian"
> Do you have a clue that your data should follow a Gaussian?
> If not, see above...
>
> HTH
>
> Herbie
>
> :::::::::::::::::::::::::::::::::::::::::::::
> Am 29.07.15 um 13:58 schrieb Rainer M. Engel:
>> Dear Herbie and Michael,
>>
>> thank you for your tips and explanation. I wanted to know a way to do
>> extrapolation and your support was very helpful.
>>
>> I'll have to see how it works on real projects.
>>
>> Below is some macro code with functions to ease this kind of task. At
>> least it did it for me..
>>
>> Best regards,
>> Rainer
>>
>>
>>
>> // COPY - PASTEFIX -----------------------------------------------------
>> print("\\Clear");
>> requires("1.49u");
>> print("Valid values..");
>> a1 =
>> newArray("#","#",7.4,"#",6.3,6.1,5.8,"#",5.7,5.9,5.8,"#","#","#",5.6,5.7,5.5,"#",5.6,5.8);
>>
>> x = numberValidSlots(a1);
>> y = stripEmptySlots(a1);
>>
>> xR = Array.getSequence(a1.length);
>>
>> for (i=0; i<x.length; i++) {
>> print(x[i]+" / "+y[i]);
>> }
>>
>> print("");
>> print("Filled gaps.. (inter-/extrapolation)");
>>
>> a2 = getCurveFittingRange(12, x, y, xR);
>> a3 = mixCurvesFillGaps(a1, a2);
>>
>> for (i=0; i<a1.length; i++){
>> print(i+" | "+a1[i]+" / "+a3[i]);
>> }
>>
>>
>> //
>> F-U-N-C-T-I-O-N-S---------------------------------------------------------------------
>> function mixCurvesFillGaps(srcArray, fitArray) {
>> string = "";
>> for (i=0; i<srcArray.length; i++) {
>> if (srcArray[i] != "#") {
>> string = string+srcArray[i]+"|";
>> } else {
>> string = string+fitArray[i]+"|";
>> }
>> }
>> return split(string,"|");
>> }
>>
>> function numberValidSlots(array) {
>> string = "";
>> for (i=0; i<array.length; i++) {
>> if (array[i] != "#") {
>> string = string+i+"|";
>> }
>> }
>> return split(string,"|");
>> }
>>
>> function stripEmptySlots(array) {
>> string = "";
>> for (i=0; i<array.length; i++) {
>> if (array[i] != "#") {
>> string = string+array[i]+"|";
>> }
>> }
>> return split(string,"|");
>> }
>>
>> function getCurveFittingRange(equation, xpoints, ypoints, xRange) {
>> Fit.doFit(equation, xpoints, ypoints);
>> cfr = "";
>> for (i=0; i<xRange.length; i++){
>> cfr = cfr+parseFloat(Fit.f(xRange[i]))+"|";
>> }
>> return split(cfr, "|");
>> }
>> // END - PASTEFIX -----------------------------------------------------
>>
>>
>>
>>
>> Am 29.07.2015 um 11:16 schrieb Herbie:
>>> Rainer.
>>>
>>> as your previous question, this again concerns 1D-signal processing not
>>> image processing ...
>>>
>>> The main question is:
>>>    What do you expect from the desired processing?
>>> ...because there are infinite ways of performing the very task.
>>>
>>> In general, curve fitting requires a model and it must be reasonably
>>> founded. If you don't have an idea of the type of function, things
>>> become more or less dubious.
>>>
>>> In order to get an idea of the problems, you may have a look at the
>>> chapter "Interpolation and Extrapolation" of "Numerical Recipes".
>>>
>>> If you want a curve to exactly pass through existing points you may have
>>> a look at spline interpolation, but spline extrapolation is generally
>>> not convincing.
>>>
>>> Either you search for corresponding algorithms and code yourself or for
>>> one of the many software packages for 1D data processing. This may be
>>> Origin, Kaleidagraph, etc. or perhaps even Excel.
>>>
>>> Here are the rounded number I get from your example:
>>> 7.4, 6.75, 6.3, 6.1, 5.8, 5.63, 5.7, 5.9, 5.8, 5.64, 5.54, 5.51, 5.6,
>>> 5.7, 5.5, 5.46, 5.6, 5.8;
>>> (no extrapolation)
>>>
>>> HTH
>>>
>>> Herbie
>>>
>>> :::::::::::::::::::::::::::::::::::::::::::::
>>> Am 29.07.15 um 10:17 schrieb Rainer M. Engel:
>>>> Hey there..
>>>>
>>>> I know a little bit about CurveFitting functions, but I'm not sure how
>>>> to interpolate and/or extrapolate missing values of an array, where I
>>>> want to keep all given values and only fill gaps.
>>>>
>>>> #,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19
>>>> OK, the example above is very simple :)
>>>>
>>>> Another example, where # marks an entry/gap that needs to be filled.
>>>> #,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8
>>>>
>>>> What would be a good starting point for this?
>>>>
>>>> Best regards,
>>>> Rainer
>>>>
>>>
>>> --
>>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Herbie]

Good evening Michael and Rainer,

I'm not quite sure whether I can bring all this together...

Rainer originally wrote:
"I want to keep all given values and only fill gaps"

 From my point of view this means that the fit has to pass through the
existing values which neither an exponential nor a Gaussian does, at
least for the provided sequence of numbers.

Michael now speaks of noise but this is a different story and implies
that a fitting curve needs not pass through the existing values but
should minimize a mean distance measure.

Now, what is the actual task?

Somehow puzzled

Herbie

::::::::::::::::::::::::::::::::::::::::::::
Am 29.07.15 um 16:22 schrieb Michael Schmid:

> Hi Rainer, Herbie,
>
> for the current data set there is almost no difference between
> "Exponential with Offset" and "Gaussian" (which actually is a
> Gaussian with offset). That's because the maximum of the Gaussian is
> far from the range of the actual data, at large negative x. So, if
> you have some kind of exponentially decaying function, in the current
> case both fit functions will do. For other data sets, such as
> essentially the same data with different noise, there will be a
> significant difference, because the fit might find a suitable
> Gaussian with its maximum much closer to x=0.
>
> So, if you just expect a kind of exponential decrease with offset or
> a Gaussian with its maximum always at very negative x (far from the
> data points supplied), better use "Exponential with Offset".
>
> BTW, the 'equation' of Fit.doFit also accepts String arguments like
> "Gaussian"; this would make the code a bit more readable. Otherwise,
> I like your macro, obviously you know the macro language really
> well!
>
> Michael
> ________________________________________________________________ On
> Jul 29, 2015, at 15:40, Herbie wrote:
>
>> Rainer,
>>
>> be careful with extrapolation.
>>
>> Michael has pointed out that extrapolation with polynomial fits
>> isn't easy and mostly prone to considerable errors.
>>
>> Without any knowledge about the signal generating process, i.e. the
>> mathematics behind it, extrapolation by curve fitting is a pure
>> guess.
>>
>> I'm not sure whether the first parameter you use in
>> Fit.doFit(equation, xpoints, ypoints); is suitable: "12 ->
>> Gaussian" Do you have a clue that your data should follow a
>> Gaussian? If not, see above...
>>
>> HTH
>>
>> Herbie
>>
>> ::::::::::::::::::::::::::::::::::::::::::::: Am 29.07.15 um 13:58
>> schrieb Rainer M. Engel:
>>> Dear Herbie and Michael,
>>>
>>> thank you for your tips and explanation. I wanted to know a way
>>> to do extrapolation and your support was very helpful.
>>>
>>> I'll have to see how it works on real projects.
>>>
>>> Below is some macro code with functions to ease this kind of
>>> task. At least it did it for me..
>>>
>>> Best regards, Rainer
>>>
>>>
>>>
>>> // COPY - PASTEFIX
>>> -----------------------------------------------------
>>> print("\\Clear"); requires("1.49u"); print("Valid values.."); a1
>>> =
>>> newArray("#","#",7.4,"#",6.3,6.1,5.8,"#",5.7,5.9,5.8,"#","#","#",5.6,5.7,5.5,"#",5.6,5.8);
>>>
>>>
>>>

x = numberValidSlots(a1);

>>> y = stripEmptySlots(a1);
>>>
>>> xR = Array.getSequence(a1.length);
>>>
>>> for (i=0; i<x.length; i++) { print(x[i]+" / "+y[i]); }
>>>
>>> print(""); print("Filled gaps.. (inter-/extrapolation)");
>>>
>>> a2 = getCurveFittingRange(12, x, y, xR); a3 =
>>> mixCurvesFillGaps(a1, a2);
>>>
>>> for (i=0; i<a1.length; i++){ print(i+" | "+a1[i]+" / "+a3[i]); }
>>>
>>>
>>> //
>>> F-U-N-C-T-I-O-N-S---------------------------------------------------------------------
>>>
>>>

function mixCurvesFillGaps(srcArray, fitArray) {

>>> string = ""; for (i=0; i<srcArray.length; i++) { if (srcArray[i]
>>> != "#") { string = string+srcArray[i]+"|"; } else { string =
>>> string+fitArray[i]+"|"; } } return split(string,"|"); }
>>>
>>> function numberValidSlots(array) { string = ""; for (i=0;
>>> i<array.length; i++) { if (array[i] != "#") { string =
>>> string+i+"|"; } } return split(string,"|"); }
>>>
>>> function stripEmptySlots(array) { string = ""; for (i=0;
>>> i<array.length; i++) { if (array[i] != "#") { string =
>>> string+array[i]+"|"; } } return split(string,"|"); }
>>>
>>> function getCurveFittingRange(equation, xpoints, ypoints, xRange)
>>> { Fit.doFit(equation, xpoints, ypoints); cfr = ""; for (i=0;
>>> i<xRange.length; i++){ cfr =
>>> cfr+parseFloat(Fit.f(xRange[i]))+"|"; } return split(cfr, "|");
>>> } // END - PASTEFIX
>>> -----------------------------------------------------
>>>
>>>
>>>
>>>
>>> Am 29.07.2015 um 11:16 schrieb Herbie:
>>>> Rainer.
>>>>
>>>> as your previous question, this again concerns 1D-signal
>>>> processing not image processing ...
>>>>
>>>> The main question is: What do you expect from the desired
>>>> processing? ...because there are infinite ways of performing
>>>> the very task.
>>>>
>>>> In general, curve fitting requires a model and it must be
>>>> reasonably founded. If you don't have an idea of the type of
>>>> function, things become more or less dubious.
>>>>
>>>> In order to get an idea of the problems, you may have a look at
>>>> the chapter "Interpolation and Extrapolation" of "Numerical
>>>> Recipes".
>>>>
>>>> If you want a curve to exactly pass through existing points you
>>>> may have a look at spline interpolation, but spline
>>>> extrapolation is generally not convincing.
>>>>
>>>> Either you search for corresponding algorithms and code
>>>> yourself or for one of the many software packages for 1D data
>>>> processing. This may be Origin, Kaleidagraph, etc. or perhaps
>>>> even Excel.
>>>>
>>>> Here are the rounded number I get from your example: 7.4, 6.75,
>>>> 6.3, 6.1, 5.8, 5.63, 5.7, 5.9, 5.8, 5.64, 5.54, 5.51, 5.6, 5.7,
>>>> 5.5, 5.46, 5.6, 5.8; (no extrapolation)
>>>>
>>>> HTH
>>>>
>>>> Herbie
>>>>
>>>> ::::::::::::::::::::::::::::::::::::::::::::: Am 29.07.15 um
>>>> 10:17 schrieb Rainer M. Engel:
>>>>> Hey there..
>>>>>
>>>>> I know a little bit about CurveFitting functions, but I'm not
>>>>> sure how to interpolate and/or extrapolate missing values of
>>>>> an array, where I want to keep all given values and only fill
>>>>> gaps.
>>>>>
>>>>> #,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19 OK, the
>>>>> example above is very simple :)
>>>>>
>>>>> Another example, where # marks an entry/gap that needs to be
>>>>> filled.
>>>>> #,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8
>>>>>
>>>>>
>>>>>

What would be a good starting point for this?

>>>>>
>>>>> Best regards, Rainer
>>>>>
>>>>
>>>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>>
>>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>>
>>
>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Rainer M. Engel]

Hey Herbie and Michael,

I want to auto crop an overscan, where the active image itself is
on a dark canvas and slightly moving with neat noise/dirt around it.
It's possible to estimate the top and bottom image positions mostly
stable with some rules and a height constraint between both positions,
but it seems mandatory to fill missing measures to work on with this
data later on (next step).

So, I still have to see how the different equations work in practice.
For me, a linear interpolation for these gaps should/would/could work as
well, but in the first place I hoped to get something to extrapolate
more suitable than I could have done it.

Your background is much more scientific and mathematic than mine I
strongly suppose. So feel free to get lost on one of my
detours/short-cuts, but I do consider your support for me to be very
kind. Thank you again.

Best regards,
Rainer



Am 29.07.2015 um 17:11 schrieb Herbie:

> Good evening Michael and Rainer,
>
> I'm not quite sure whether I can bring all this together...
>
> Rainer originally wrote:
> "I want to keep all given values and only fill gaps"
>
> From my point of view this means that the fit has to pass through the
> existing values which neither an exponential nor a Gaussian does, at
> least for the provided sequence of numbers.
>
> Michael now speaks of noise but this is a different story and implies
> that a fitting curve needs not pass through the existing values but
> should minimize a mean distance measure.
>
> Now, what is the actual task?
>
> Somehow puzzled
>
> Herbie
>
> ::::::::::::::::::::::::::::::::::::::::::::
> Am 29.07.15 um 16:22 schrieb Michael Schmid:
>> Hi Rainer, Herbie,
>>
>> for the current data set there is almost no difference between
>> "Exponential with Offset" and "Gaussian" (which actually is a
>> Gaussian with offset). That's because the maximum of the Gaussian is
>> far from the range of the actual data, at large negative x. So, if
>> you have some kind of exponentially decaying function, in the current
>> case both fit functions will do. For other data sets, such as
>> essentially the same data with different noise, there will be a
>> significant difference, because the fit might find a suitable
>> Gaussian with its maximum much closer to x=0.
>>
>> So, if you just expect a kind of exponential decrease with offset or
>> a Gaussian with its maximum always at very negative x (far from the
>> data points supplied), better use "Exponential with Offset".
>>
>> BTW, the 'equation' of Fit.doFit also accepts String arguments like
>> "Gaussian"; this would make the code a bit more readable. Otherwise,
>> I like your macro, obviously you know the macro language really
>> well!
>>
>> Michael
>> ________________________________________________________________ On
>> Jul 29, 2015, at 15:40, Herbie wrote:
>>
>>> Rainer,
>>>
>>> be careful with extrapolation.
>>>
>>> Michael has pointed out that extrapolation with polynomial fits
>>> isn't easy and mostly prone to considerable errors.
>>>
>>> Without any knowledge about the signal generating process, i.e. the
>>> mathematics behind it, extrapolation by curve fitting is a pure
>>> guess.
>>>
>>> I'm not sure whether the first parameter you use in
>>> Fit.doFit(equation, xpoints, ypoints); is suitable: "12 ->
>>> Gaussian" Do you have a clue that your data should follow a
>>> Gaussian? If not, see above...
>>>
>>> HTH
>>>
>>> Herbie
>>>
>>> ::::::::::::::::::::::::::::::::::::::::::::: Am 29.07.15 um 13:58
>>> schrieb Rainer M. Engel:
>>>> Dear Herbie and Michael,
>>>>
>>>> thank you for your tips and explanation. I wanted to know a way
>>>> to do extrapolation and your support was very helpful.
>>>>
>>>> I'll have to see how it works on real projects.
>>>>
>>>> Below is some macro code with functions to ease this kind of
>>>> task. At least it did it for me..
>>>>
>>>> Best regards, Rainer
>>>>
>>>>
>>>>
>>>> // COPY - PASTEFIX
>>>> -----------------------------------------------------
>>>> print("\\Clear"); requires("1.49u"); print("Valid values.."); a1
>>>> =
>>>> newArray("#","#",7.4,"#",6.3,6.1,5.8,"#",5.7,5.9,5.8,"#","#","#",5.6,5.7,5.5,"#",5.6,5.8);
>>>>
>>>>
>>>>
>>>>
> x    = numberValidSlots(a1);
>>>> y    = stripEmptySlots(a1);
>>>>
>>>> xR    = Array.getSequence(a1.length);
>>>>
>>>> for (i=0; i<x.length; i++) { print(x[i]+" / "+y[i]); }
>>>>
>>>> print(""); print("Filled gaps.. (inter-/extrapolation)");
>>>>
>>>> a2    = getCurveFittingRange(12, x, y, xR); a3    =
>>>> mixCurvesFillGaps(a1, a2);
>>>>
>>>> for (i=0; i<a1.length; i++){ print(i+" | "+a1[i]+" / "+a3[i]); }
>>>>
>>>>
>>>> //
>>>> F-U-N-C-T-I-O-N-S---------------------------------------------------------------------
>>>>
>>>>
>>>>
> function mixCurvesFillGaps(srcArray, fitArray) {
>>>> string    = ""; for (i=0; i<srcArray.length; i++) { if (srcArray[i]
>>>> != "#") { string    = string+srcArray[i]+"|"; } else { string    =
>>>> string+fitArray[i]+"|"; } } return split(string,"|"); }
>>>>
>>>> function numberValidSlots(array) { string    = ""; for (i=0;
>>>> i<array.length; i++) { if (array[i] != "#") { string    =
>>>> string+i+"|"; } } return split(string,"|"); }
>>>>
>>>> function stripEmptySlots(array) { string    = ""; for (i=0;
>>>> i<array.length; i++) { if (array[i] != "#") { string    =
>>>> string+array[i]+"|"; } } return split(string,"|"); }
>>>>
>>>> function getCurveFittingRange(equation, xpoints, ypoints, xRange)
>>>> { Fit.doFit(equation, xpoints, ypoints); cfr    = ""; for (i=0;
>>>> i<xRange.length; i++){ cfr    =
>>>> cfr+parseFloat(Fit.f(xRange[i]))+"|"; } return split(cfr, "|");
>>>> } // END - PASTEFIX
>>>> -----------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>> Am 29.07.2015 um 11:16 schrieb Herbie:
>>>>> Rainer.
>>>>>
>>>>> as your previous question, this again concerns 1D-signal
>>>>> processing not image processing ...
>>>>>
>>>>> The main question is: What do you expect from the desired
>>>>> processing? ...because there are infinite ways of performing
>>>>> the very task.
>>>>>
>>>>> In general, curve fitting requires a model and it must be
>>>>> reasonably founded. If you don't have an idea of the type of
>>>>> function, things become more or less dubious.
>>>>>
>>>>> In order to get an idea of the problems, you may have a look at
>>>>> the chapter "Interpolation and Extrapolation" of "Numerical
>>>>> Recipes".
>>>>>
>>>>> If you want a curve to exactly pass through existing points you
>>>>> may have a look at spline interpolation, but spline
>>>>> extrapolation is generally not convincing.
>>>>>
>>>>> Either you search for corresponding algorithms and code
>>>>> yourself or for one of the many software packages for 1D data
>>>>> processing. This may be Origin, Kaleidagraph, etc. or perhaps
>>>>> even Excel.
>>>>>
>>>>> Here are the rounded number I get from your example: 7.4, 6.75,
>>>>> 6.3, 6.1, 5.8, 5.63, 5.7, 5.9, 5.8, 5.64, 5.54, 5.51, 5.6, 5.7,
>>>>> 5.5, 5.46, 5.6, 5.8; (no extrapolation)
>>>>>
>>>>> HTH
>>>>>
>>>>> Herbie
>>>>>
>>>>> ::::::::::::::::::::::::::::::::::::::::::::: Am 29.07.15 um
>>>>> 10:17 schrieb Rainer M. Engel:
>>>>>> Hey there..
>>>>>>
>>>>>> I know a little bit about CurveFitting functions, but I'm not
>>>>>> sure how to interpolate and/or extrapolate missing values of
>>>>>> an array, where I want to keep all given values and only fill
>>>>>> gaps.
>>>>>>
>>>>>> #,#,#,#,4,5,6,7,8,#,10,11,12,13,#,#,#,17,18,19 OK, the
>>>>>> example above is very simple :)
>>>>>>
>>>>>> Another example, where # marks an entry/gap that needs to be
>>>>>> filled.
>>>>>> #,#,7.4,#,6.3,6.1,5.8,#,5.7,5.9,5.8,#,#,#,5.6,5.7,5.5,#,5.6,5.8
>>>>>>
>>>>>>
>>>>>>
> What would be a good starting point for this?
>>>>>>
>>>>>> Best regards, Rainer
>>>>>>
>>>>>
>>>>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>>>
>>>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>>>
>>>
>>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
>

--
###################################
Rainer M. Engel - Managing Director
endime | ENGEL DIGITAL MEDIA

Pichelsdorfer Str. 143
13595 Berlin

Ust-StNr.:DE254256305
BLZ: 20010020
Kto: 971873201

web: www.endime.de
mail: [hidden email]
fon.:-Berlin [phone]
mobile: [phone]
skype: tomdooley4711
###################################
not only a game http://cx.endime.de
###################################

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html