Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!newstop!exodus!exodus-bb!khb From: khb@chiba.Eng.Sun.COM (Keith Bierman fpgroup) Newsgroups: comp.sys.hp Subject: Re: cc optimizer dies on a few perl.4.0.beta files Message-ID: Date: 26 Mar 91 21:13:08 GMT References: <7370347@hpfcso.FC.HP.COM> Sender: news@exodus.Eng.Sun.COM Organization: Sun MegaSystems Lines: 98 In-reply-to: mike@hpfcso.FC.HP.COM's message of 26 Mar 91 16:37:02 GMT ....optimizer is unsafe.... I've commented on this before. In general, in my experience application codes are at fault far more often than optimizer errors. This same topic has come up in comp.benchmarks. Since Robert posted it publically, I see no reason not to repost it here. Sometimes it is no ones "bug" it is just an interesting interaction ... typically machines with wide accumulators (ala the 8087 and 68881) user code and register allocators can blend to provide really hard to track down bugs .... which in some sense are not bugs. The allocator did a good job. The user had no way to know (or control) how big the temps are, etc. I usually argue against such wide registers for just such reasons.... From: metzger@convex.com (Robert Metzger) Newsgroups: comp.benchmarks Subject: Re: benchmarks and interprocedural optimization Date: 23 Mar 91 18:21:54 GMT Organization: Convex Computer Corporation, Richardson, Tx. Nntp-Posting-Host: bach.convex.com In article <1991Mar21.213144.14636@nas.nasa.gov> eugene@wilbur.nas.nasa.gov (Eugene N. Miya) writes: >In article <1991Mar21.184428.8226@convex.com> metzger@convex.com >(Robert Metzger) writes: >>One other note: interprocedural optimization will render many industry >>standard benchmarks meaningless. > >Just getting back...... >You should be careful about this. > >The problem is how companies like yours go about testing fixes and >optimizations. Standard test suites exist for language syntax conformance, >deviance, but no such standard set of tests exists for language optimizations. >There needs to be some testing consistency for all this fancy software, >otherwise, how do we have any assurance said features are working. > >That's one reason why I just regard this as another form of testing, >and nothing special. Of course, no company wants embarassing test results. I appreciate your kind concern, but I'm not sure what your're warning me about :-) I'll elaborate on my point: 1) Compilers that perform interprocedural optimization render meaningless the results of simple-minded benchmarks. For example, one of the first things a well-meaning technical-sales-support person at CONVEX did when he got a crack at the Application Compiler was to run whetstone through it. Sure enough, since whetstone attempts to measure call overhead and the APC attempts to minimize call overhead, the results made it look like CONVEX had released a new generation of hardware. We didn't release those numbers to the public, but there's nothing that prevents an unsophisticated buyer from doing the same thing and being misled. (Of course if he's buying million dollar computers based on the results of such stupid benchmarks, maybe he deserves what he gets.) 2) Compilers that perform interprocedural optimization cannot be fooled by subroutine calls into not performing optimizations. This is a common trick in some widely used benchmarks. Well, an IPO compiler will certainly perform thorough IP side-effect analysis, and it will know exactly what is going on inside the procedure. It may automatically perform procedure inlining as well. Procedure calls no longer prevent compilers from moving assignments and uses, or eliminating them altogether, where procedure compilers could not do so. Procedure calls are no longer firewalls. 3) Compilers that perform interprocedural optimization perform a lot more compile-time evaluation of your code. Basically, if you don't read your all data from a data file at runtime, the compiler could potentially evaluate your expressions at compile time and replace your application with a bunch of PRINT statements. I will be the first to say that testing interprocedural optimizers is very challenging. In some senses, a 100,000 line application is ONE test case. Which means you have to run hundreds of applications and millions of lines of code through the compiler before you trust it. On the other hand, our experience with an interprocedural compiler is that the compiler find a dozens or hundreds of bugs in every application for every one bug the application finds in the compiler. Logic errors, abuse of language features, and numerically unstable algorithms are all exposed by this new technology. From my perspective, 99.99% of all scientific/ engineering FORTRAN applications are shot through with numerous errors, and I'm far more concerned about the reliability of the applications than the compilers at this point. -- Robert Metzger CONVEX Computer Corp. Richardson, Texas Generic Disclaimer: I write software, not CONVEX Corporate Policies. "The only legitimate function of government is to protect its citizens from harm or property loss by violence, stealth, or fraud. All else is tyranny." -- ---------------------------------------------------------------- Keith H. Bierman kbierman@Eng.Sun.COM | khb@chiba.Eng.Sun.COM SMI 2550 Garcia 12-33 | (415 336 2648) Mountain View, CA 94043