Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!cs.utexas.edu!usc!snorkelwacker!mit-eddie!bu.edu!mirror!frog!cpoint!jjmhome!m2c!umvlsi!dime!dime.cs.umass.edu!moss From: moss@cs.umass.edu (Eliot Moss) Newsgroups: comp.unix.wizards Subject: Re: Floating Point Expectations Message-ID: Date: 1 Jun 90 18:40:00 GMT References: <411@yonder.UUCP> <1990May31.001618.24274@ico.isc.com> <1011@s6.Morgan.COM> Sender: news@dime.cs.umass.edu Reply-To: moss@cs.umass.edu Organization: Dept of Comp and Info Sci, Univ of Mass (Amherst) Lines: 17 In-reply-to: amull@Morgan.COM's message of 31 May 90 17:22:47 GMT Just a minor followup to the (45.0 / 100.0) == .45 question: While the integers 45.0 and 100.0 are undoubtedly represented exactly, .45 is a repeating form in binary (just as, say, 1/7 is in decimal notation). One *would* expect (50.0 / 100.0) == .50 since .50 *does* have an exact representation in binary. In general, when writing floating point algorithms one must demand equality only within a tolerance; choosing that tolerance is a complex issue, and one can have relative and absolute tolerances. Hope this adds a little to the original posters understanding .... Eliot -- J. Eliot B. Moss, Assistant Professor Department of Computer and Information Science Lederle Graduate Research Center University of Massachusetts Amherst, MA 01003 (413) 545-4206; Moss@cs.umass.edu