1 Topic by k6sti (edited by k6sti 2011-04-05 03:00:30)

Topic: Roundoff Error

Two graphs below compare the same data using linear interpolation and 1D cubic spline interpolation (2D looks the same as 1D). Above X = 7000, the trace bobbles up and down for 1D. Another image shows that the Y data values increase monotonically. A final curve shows linear interpolation of 100 points generated by an external cubic spline interpolator. It has no bobbles.

Because the bobble does not increase when magnifying the Y scale, I suspect it may be a roundoff error at the pixel level. Note that it occurs at X intervals of 1000, which is the data resolution in the bobble region.

On other curves with dense data, the problem manifests itself as little dots on top of the traces.

Brian

Post's attachments

t0.png, 5.26 kb, 640 x 360
t0.png 5.26 kb, 564 downloads since 2011-04-04 

t1.png, 4.7 kb, 640 x 360
t1.png 4.7 kb, 574 downloads since 2011-04-04 

t2.png, 4.75 kb, 640 x 360
t2.png 4.75 kb, 582 downloads since 2011-04-04 

test2.png, 8.07 kb, 253 x 465
test2.png 8.07 kb, 574 downloads since 2011-04-04 

Re: Roundoff Error

This may shed some light on the problem.

Brian

Post's attachments

1d.png, 4.2 kb, 400 x 400
1d.png 4.2 kb, 576 downloads since 2011-04-05 

data.gif, 2.76 kb, 228 x 219
data.gif 2.76 kb, 531 downloads since 2011-04-05 

lin.png, 3.9 kb, 400 x 400
lin.png 3.9 kb, 568 downloads since 2011-04-05 

3 Reply by k6sti (edited by k6sti 2011-04-06 14:19:18)

Re: Roundoff Error

Here is an example of the dots the internal 1D spline routine introduces (2D is similar). I saved these images at screen resolution and then magnified them by a factor of two.

Brian

Post's attachments

Attachment icon comp.gif 13.24 kb, 371 downloads since 2011-04-06 

Re: Roundoff Error

As you have guessed it is a rounding problem. The spline oscillates a little. It just happens to sometimes round a pixel up and sometimes down. I have now changed the rounding in the latest beta version so it rounds to the nearest pixel. This seems to have fixed the problem.

Re: Roundoff Error

Ivan, I tried downloading it twice. Both times I got the error shown.

Brian

Post's attachments

msg.gif, 14.13 kb, 609 x 372
msg.gif 14.13 kb, 545 downloads since 2011-04-07 

Re: Roundoff Error

It looks like something went wrong uploading the file. I have uploaded it again.

Re: Roundoff Error

No problem downloading now, Ivan. The bobbles and dots are gone and the curves look nice! Thanks for the fix.

Brian