Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!wuarchive!decwrl!henry.jpl.nasa.gov!elroy.jpl.nasa.gov!gryphon!jspear From: jspear@gryphon.COM (Jon Spear) Newsgroups: comp.sys.mac.programmer Subject: Re: Testing Tools... Summary: Let us know... Message-ID: <23788@gryphon.COM> Date: 25 Dec 89 07:37:46 GMT References: <2bbd2b567@caladan.UUCP> Reply-To: jspear@gryphon.COM (Jon Spear) Organization: Trailing Edge Technology, Redondo Beach, CA Lines: 62 In article <2bbd2b567@caladan.UUCP> atreides@caladan.UUCP (root/postmaster/etc) writes: >There is always a lot of discussion about a variety of development >environments (and rightly so), but my interests lie more with testing >tools, and I have never seen a discussion on this topic. > [...] >I'm especially interested in tools for: > > - Automation > - Code Coverage > - Code Complexity Measurement [note: I'd intended to mail this, but it bounced...] It is indeed hard to accept the Mac as a platform for developing highly reliable software when there seems to be a dearth of testing and code analysis tools available (not to mention the lack of a protected-mode operating system or a verified kernel or the popularity of languages like C that almost encourage unsafe coding [donning flame-retardant undies]). An automated test driver for general Mac applications is an interesting problem. The closest I know of is the monkey DA that posts random mouse and keyboard events, ... what more could you want? You can get a little mileage out of scripting/macro facilities like Tempo II, but that isn't quite a complete facility for maintaining a database of regression tests, running some subset, collecting and analyzing results, and displaying a summary report. Is this a problem with the Mac environment(s), or is it a general problem for systems that use a graphical user interface? The only test automation packages like this that I'm aware of are for Unix or VMS programs that are either batch-oriented or use simple character-oriented display user interfaces. Test coverage seems like an easier problem, but again, I've seen nothing for the Mac. The closest things I can think of are the performance analyzers that come with a couple environments. You know - the thingies that instrument your code to call a time-stamper subroutine that collects timing data on a per-module basis. [I haven't actually used these on the Mac -- do they instrument the code or sample the PC at some interrupt interval, or both?] It seems like it wouldn't be too hard to read that bucket file and identify which modules were called and which were not -- very crude coverage indicator, but better than nothing... Complexity metrics are also straightforward to port to the Mac, assuming you are using a language with standard syntax. But metrics are of less obvious value, other than curiosity, or maybe I've missed something. Do any complexity metrics exist that are reasonably simple to automate that have been shown to correlate well with the true maintainability or understandability of a piece of code? That are not easily fooled? That provide specific feedback on how to improve the code? I know Halstead's, McCabe's, and other metrics I've seen don't seem very useful in this respect. Please let us know what you find. -Jon -- ----- [The following address is soon to evaporate. New address sought.] Jon L Spear: jspear@gryphon.COM !gryphon!jspear gryphon!jspear@elroy.jpl.nasa.gov "With computers we can make billions of mistakes every second!"