Xref: utzoo comp.lang.lisp:1641 comp.cog-eng:1003
Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!unmvax!pprg.unm.edu!hc!lll-winken!uunet!mcvax!ukc!icdoc!syma!aarons
From: aarons@syma.sussex.ac.uk (Aaron Sloman)
Newsgroups: comp.lang.lisp,comp.cog-eng
Subject: Re: Debug techniques. What's your favorite?
Summary: Rapid exploration and extendable debugging tools help
Keywords: debugging, abstraction, levels of description
Message-ID: <877@syma.sussex.ac.uk>
Date: 8 Apr 89 21:44:07 GMT
References: <1499@wasatch.UUCP>
Organization: School of Cognitive & Computing Sciences, Sussex Univ. UK
Lines: 123

u-dmfloy%ug.utah.edu@wasatch.UUCP (Daniel M Floyd)

> Reply-To: u-dmfloy%ug.utah.edu.UUCP@wasatch.UUCP (Daniel M Floyd)
> Organization: University of Utah CS Dept
 writes:
> I am writing a debuger for a Common LISP software package.
    .....
> if I should happen to have extra time, I could build the
> best lisp debugger in the world ... assuming I knew what
> a 'best lisp debugger' was. :-))

> Also, if you have a wish (or wish list) now would be a good time
> for me to hear about that too.

I'll also be interested to see what people think they need: we would
like to improve the debugging aids in our development system too!

The question asked is potentially very deep. Here are just a few
semi-random reflections on the problem.

One way to think about this is to ask what is needed to help less able
programmers perform more like the best ones. This takes us straight into
problems of cognitive science/cognitive engineering, usually ignored
by computer scientists and language designers.

I believe that the ability to track down program bugs and design flaws
(relatively) quickly is one of the things that distinguish the very best
software developers from the mediocre.

How do they do it? If we had good answers to that, we might be able to
begin to design tools to help lesser mortals improve their productivity.

One important characteristic that some people have to a greater extent
than others is the ability to discern when the level of description
given in a bug-report is spurious: it describes conditions and behaviour
that distract from the real problem. Put another way, the expert faced
with a bug knows how to find the right way to describe the bug. This is
important because if you get the description wrong you can get the fix
wrong - and e.g. just cope with a special case of a general problem,
or simultaneously fix your bug and introduce new ones.

The inexpert programmer varies conditions and observes behaviour at a
low level of detail to explore the problem. The expert varies them at a
higher level of abstraction, ruling out larger search spaces at a time.
Eventually the right description is found, and identifying the relevant
portion of the source code is easy.

The fix may or may not be easy. Again, the expert will have access to
knowledge about many ways of altering programs to change their
behaviour, and will be very good at homing in rapidly on the right kind
of change (even if the fixing isn't right first time, it may be in the
right ball-park, requiring only small subsequent alterations to get the
final fix). The less expert programmer will know about fewer ways of
transforming programs and will have less accurate, less general, or less
complete descriptions of what can be achieved by such transformations.

So the main debugging aids required will include tools to help one
narrow down the right description of the bug, and tools to help one
narrow down the right way to change a program (or a design, or
specification) to produce the desired result.

What would such tools be like? It's clear that very sophisticated
pattern recognition and associative memory capapbilities will be
required. But prior to this is needed a good language (or set of
languages) for describing different kinds of bugs, different kinds of
program alterations, and different kinds of effects of such alterations.
What else? I think that's a hard research problem.

In the meantime, a minimum requirement is the ability rapidly to explore
a variety of different inputs to a program and a variety of different
modifications, and to record the results conveniently. In that way we can
collect evidence as to what the problem is.

This rules out systems where every little change requires a very slow
edit -> compile -> link -> run -> edit cycle.

AI languages and development environments typically provide these rapid
exploration abilities, though they vary in the ease with which they
allow switching between editing, running, examining datastructures,
recording and comparing output, turning different kinds of tracing on
and off, changing the tracer to display new kinds of information,
creating new temporary procedures or modifying old ones, to bring out
normally hidden aspects of what a program has done, finding
documentation on the facilities used, etc. etc.

For example if you use a low level language like lisp or Pop-11 to
implement something like prolog or a production system, you should have
tools that make it easy to extend the debugging and tracing tools to
operate at a level that is suitable for the new higher level language.

More generally, user programs will have their own level at which they
need debugging aids: so ideally it should be easy for the user to
implement them.

What kinds of tools make this possible? I'd include macro facilities and
more generally the ability to extend the syntax of the programming
language to meet the requirements of the problem. Similarly the syntax
of the instructions to print out helpful information or interactively
modify behaviour during debugging may need to be tailored to the problem
domain: if you are debugging a music synthesising program you want to be
able to give debugging instructions using a syntax appropriate for
music, not just procedure calls in the implementation language.

Of course, for each such new level of abstraction and each application
domain, designing good debugging aids is itself a hard problem requiring
considerable ingenuity and creativity.

Oh, and there are also the trivial bugs for which it is convenient to be
able to do things like single step through a program, examining
variables and the control flow. Is it possible that providing that sort
of facility stops people thinking about the really hard problems?

Aaron Sloman,
School of Cognitive and Computing Sciences,
Univ of Sussex, Brighton, BN1 9QN, England
    ARPANET : aarons%uk.ac.sussex.cogs@nss.cs.ucl.ac.uk
              aarons%uk.ac.sussex.cogs%nss.cs.ucl.ac.uk@relay.cs.net
    JANET     aarons@cogs.sussex.ac.uk
    BITNET:   aarons%uk.ac.sussex.cogs@uk.ac
        or    aarons%uk.ac.sussex.cogs%ukacrl.bitnet@cunyvm.cuny.edu

    UUCP:     ...mcvax!ukc!cogs!aarons
            or aarons@cogs.uucp