Hi Knut,
ImageJ has no image data type with double precision (approx. 14
digit) accuracy, only float (approx. 7 digits) - at least for
standard microscopy and light imaging techniques, there are no image
sensors that would require more than float accuracy. Thus, also
functions like profile plot use float accuracy.
The Results table has full double precision accuracy.
Michael
________________________________________________________________
On 19 Mar 2010, at 12:53, Knut Kvaal wrote:
> Hello
> I have a confusing problem when writing a plugin. When I put
> numbers into the results table and plot as well, the list in plot
> dialog shows numbers with different precision than the results.
> I have in this example set decimal places to 9 and enabled
> Scientific Notation.
> Here is what I get:
> Results table 2. Column and List from Plot 3.rd column
> 1 1.695265623E0 1.695265651E0
> 2 1.741630384E0 1.741630435E0
> 3 1.996989389E0 1.996989369E0
> 4 1.756611630E0 1.756611586E0
> 5 1.594811512E0 1.594811559E0
> 6 1.329282159E0 1.329282165E0
> 7 1.114680459E0 1.114680409E0
> 8 9.434604256E-1 9.434604049E-1
> 9 9.000309046E-1 9.000309110E-1
> 10 7.652984144E-1 7.652984262E-1
>
> Do anyone have any explanation? The correct values should be the 2.
> Column as I have checked by direct print.
> I have no statements to set the precision an let ImageJ default rule.
>
> Best regards
> Knut Kvaal
>
> _____________________________
> Knut Kvaal, Associate Professor
> Department of Mathematical Sciences and Technology
> P.O.Box 5003
> Norwegian University of Life Sciences (UMB) N - 1432 Ås NORWAY
>
> Tel: +47 64965394 (Norway)
> Cell: +47 90920332 (Norway)
>
> Personal Homepage:
http://arken.umb.no/~kkvaal
Locked
