WeBWorK Problems

Article on Floating Point Calculations

Article on Floating Point Calculations

by Jason Aubrey -
Number of replies: 0
One of the recurring topics in the forums is rounding of floating point calculations. I just recently saw a post in slashdot referencing this article:


It points out things like: According to Google 599,999,999,999,999 – 599,999,999,999,998 = 1

And tells this story:

The case of a Patriot defense system failing to take down a Scud missile attack: 'The calculation of where to look for confirmation of an incoming missile requires knowledge of the system time, which is stored as the number of 0.1-second ticks since the system was started up. Unfortunately, 0.1 seconds cannot be expressed accurately as a binary number, ...
...and 28 people paid with their lives.'"

Good teaching examples, and much more accessible than The Standard Reference (What Every Computer Scientist Should Know about Floating Point Arithmetic)