Home Original page

high precision mathematics

Karl M. Syring syring at email.com
Wed Feb 20 01:45:34 EST 2002 
"Carl Banks" <idot at vt.edu> schrieb>
<snip>
> Flight simulation: The earth's radius is about 20 million feet (about
> 7 million meters for my SI using friends), and the simulator must
> resolve, at a minimum, distances of about 0.1 feet.  That's a
> difference of 9 decimal places.  We then need several more places for
> numerical robustness.
<snip>

You surely are joking, Mr. Banks?
This is exactly the kind of misuse of floating point that was referred to in
a previous posting. Every computational geometry software would use a
rational data type here, although you might get away with simple point data
binning in this case.

Karl M. Syring
More information about the Python-list mailing list