Path: utzoo!utgpu!water!watmath!clyde!rutgers!uwvax!dogie!uwmcsd1!ig!agate!ucbvax!MITVMA.MIT.EDU!KASTEN
From: KASTEN@MITVMA.MIT.EDU (Frank Kastenholz)
Newsgroups: comp.protocols.tcp-ip
Subject: Verification Suite
Message-ID: <8802131548.AA22277@ucbvax.Berkeley.EDU>
Date: 13 Feb 88 15:03:21 GMT
Sender: daemon@ucbvax.BERKELEY.EDU
Organization: The Internet
Lines: 37











Hal Peterson (hrp%windsor.CRAY.COM@uc.msc.umn.edu) at Cray wrote

>I should stress, though, that a verification suite is only part of a
>comprehensive testing strategy, and that bake-offs can be extremely
>valuable for finding bugs.  A verification suite is software, and like
>all software it is imperfect.  It is entirely possible that two
>implementations both pass the suite but don't interoperate.  Bake-offs
>can catch that, and when they do they have found as many as FOUR bugs:
>
>1.  a bug in one of the implementations.
>2.  a bug in the other implementations; after all, they could both be
>    at fault.
>3.  a bug in the specification.  It may be unclear or inconsistent or
>    incomplete and so have misled the implementors.
>4.  a bug in the verification suite.  It should have caught the
>    problem, and next time it will, since it's a simple matter to add
>    a test to a well-designed suite.  The suite gets better as time
>    goes on.

His last two points bring two simple(?) questions to mind -

How do the Specification Bugs get resolved?

Who Verifies the Verification Suite???? (Qui Ipsos Custode Custodiet)

Frank Kastenholz
Atex Inc.