Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.2 9/18/84; site watmath.UUCP Path: utzoo!watmath!wjafyfe From: wjafyfe@watmath.UUCP (Andy Fyfe) Newsgroups: net.arch,net.decus,net.micro Subject: Re: DEC PRO350 Floating Point Message-ID: <11961@watmath.UUCP> Date: Thu, 14-Mar-85 23:46:29 EST Article-I.D.: watmath.11961 Posted: Thu Mar 14 23:46:29 1985 Date-Received: Fri, 15-Mar-85 05:15:37 EST References: <2810@ncsu.UUCP> <3100001@rna.UUCP> Reply-To: wjafyfe@watmath.UUCP (Andy Fyfe) Organization: U of Waterloo, Ontario Lines: 9 Xref: watmath net.arch:983 net.decus:93 net.micro:9699 Summary: There are a number of DEC PRO 350s around the campus here. A number of them are used by the applied math department for various numerical things (with many of the programs written in basic). We found out that the basic square root function is hopelessly inaccurate -- no more than about 3 significant digits. Raising to the power .5 is much better. Is this the fault of basic, or the floating point hardware? --Andy Fyfe ...!{decvax, allegra, ihnp4, et. al}!watmath!wjafyfe wjafyfe@waterloo.csnet