Floating point numbers and inaccurate arithmetic

Floating point arithmetic is… at its worst when it is expected to model the interactions of quantities expressed as decimal strings that are expected to be exact.

Floating point – Wikipedia, the free encyclopedia

I’m running into this problem at work, where adding float values together results in slightly incorrect arithmetic. For example, for a given situation, I’m expecting the value 40.41, but the float value is stored as 40.40995. To work around this, I’ve had to round information to two decimal places after finishing sets of math operations. Kludge!

We will need to think about using decimal values instead of floats due to float’s arithmetic issues.

Blogged with Flock

Tags:

Advertisements

One Response to Floating point numbers and inaccurate arithmetic

  1. […] Posted by geeknotes under Programming  I read Chris Vance’s blog entry about floating point numbers and inaccurate arithmetic.  It got me thinking about precision and significant digits, and a simple problem illustrates […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: