Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!ukma!husc6!cmcl2!ccnysci!alexis
From: alexis@ccnysci.UUCP (Alexis Rosen)
Newsgroups: comp.sys.mac
Subject: Re: Need a beta-tester?
Keywords: test,shareware,fun,mac,freeware,help
Message-ID: <1850@ccnysci.UUCP>
Date: 3 May 89 09:13:42 GMT
References: <3456@udccvax1.acs.udel.EDU> <6972@bsu-cs.bsu.edu>
Reply-To: alexis@ccnysci.UUCP (Alexis Rosen)
Organization: City College of New York
Lines: 52


PRE-DISCLAIMER:
I don't mean to criticize the poster of the origianl message specifically.
I don't know him and have no reason to think he's irresponsible in any way.

That said, I'd like to bitch about something near and dear to my heart: The
quality of QA in this business. All too often, we see companies releasing
software that's full of bugs- sometimes dangerous bugs capable of destroying
files, or even whole disks. This attitude of "let the buyers be beta" can
only be encouraged by sloppy or non-existent testing on the part of the
"official" beta testers.

The idea that you can just use some software casually and maybe report back
the bugs occasionally is all too common. It may be fine for small PD or
shareware programs distributed on the net (probably all the original poster
had in mind, I'm sure) but it's totally inadequate for real-live commercial
programs whose binaries can range up to a megabyte.

Unfortunately, that's exactly what you see much of the time. I know people
who are beta-ing new versions of many of the big-name commercial products,
and most of them (not all) completely fail to live up to their side of the
bargain. Only outside testing can kill that last layer of bugs, and when it's
poorly done there's little the manufacturer can do about it.

Just to give you an example, I have tested all the versions of FoxBase since
before the initial release. I know that they had over 250 Beta testers for
release 1.0, and they conscientiously sent us weekly updates (sometimes even
more frequently) by Fed Ex red label. They retained about ten sites for the
Beta of 1.1, because they never heard once from the other 240. The few of us
who did proper testing spent hours on the phone each week with them. The
nearly bug-free release of Fox 1.0 can be attributed to great programming by
the Fox team, but also to the people on the outside who tried things the
program was never designed to do (but usually succeeded anyway... what a great
piece of work...).  Some months (when I did testing under contract) my phone
bills for long-distance could go up to several hundred dollars.

There is also another side to this, which is the manufacturer's willingness
to acknowledge and address bugs, both during beta and after release. I've
beaten up Acius enough in the past that I won't tell the whole story over,
but they are a classic example of a company that totally ignored bug reports
during Beta. TOPS is another such example.

The moral of the story is that when you talk about Beta testing, it is
NOT primarily a way to get free software. IT's the ONLY way to get a big
complex product truly and well-debugged. (Ask MicroSoft :-)

Oh well. Off the soapbox.

---
Alexis Rosen
alexis@ccnysci.{uucp,bitnet}
alexis@rascal.ics.utexas.edu  (last resort)