Path: utzoo!attcan!uunet!seismo!sundc!pitstop!sun!amdcad!ames!mailrus!tut.cis.ohio-state.edu!bloom-beacon!spdcc!ima!compilers-sender
From: linus!mbunix!eachus@harvard.edu (Eachus)
Newsgroups: comp.compilers
Subject: Re: Validation Suite Report
Message-ID: <2584@ima.ima.isc.com>
Date: 30 Aug 88 14:28:45 GMT
Sender: compilers-sender@ima.ima.isc.com
Reply-To: linus!mbunix!eachus@harvard.edu (Eachus)
Organization: The MITRE Corporation, Bedford, Mass.
Lines: 53
Approved: compilers@ima.UUCP
Posted-Date: Tue, 30 Aug 88 10:28:45 EDT
In-Reply-To: <2332@ima.ima.isc.com>

>If you can add any information to this posting, please let me know.

>=============================== ADA ===============================        
        
>ACVC - Ada Compiler Validation

     This is the official validation test suite.  Currently about 4000
files, my guess  is about 20000  actual tests. (Some  tests must be in
multiple files, but many of the  test programs contain multiple tests.
A real fun experience -- if you are into S&M.

>ACEC - Suite to test Ada performance.  

     This is  being developed by  the Boeing Military Airplane Company
under government contract. First release is scheduled to occur any day
now.  It will be   distributed by the   Data and  Analysis  Center for
Software (DACS) at Griffiss AFB, Rome NY.
         
>The University of Michigan has a suite of Ada tests and benchmarks.

     These are useful, but for the most part require intrumentation to
Get accurate timings.  A  better set of  benchmarks is the  ACM SIGAda
Performance Issues Working Group (PIWG) test suite, which is the suite
most vendors use when publishing perfomance figures.

>Don't forget the DoD Ada validation suite!  It's supposed to be one
>of the most strenuous around, and is applied to all Ada compilers
>before they are allowed to use the name (since Ada is a registered
>trademark of the DoD, they can do that).

     There is no question that the ACVC is  the most strenuous around,
and the standards  are   MUCH stricter  than for   other  validations.
Before the validation you submit a list of tests which you think don't
apply to  your  validation and why. (Most  of   these are  usually for
floating point precision beyond what the compiler supports.) With this
you provide a listing from running all  of the tests,  and a few other
things. (This is the infamous "box of paper".)

     If the validation agency  approves your submission, they come  on
site to verify the results  themselves.  Failing  any test means  that
you fail  the validation.  No  second  chances, no  fixing bugs on the
spot.  If you pass you get to do it all again next year.

     Incidently,  Ada is no longer  a registered trademark of the DoD.
The trademark was intentionally  allowed to lapse  last year as having
accomplished its purpose.

					Robert I. Eachus
--
Send compilers articles to ima!compilers or, in a pinch, to Levine@YALE.EDU
Plausible paths are { decvax | harvard | yale | bbn}!ima
Please send responses to the originator of the message -- I cannot forward
mail accidentally sent back to compilers.  Meta-mail to ima!compilers-request