Path: utzoo!attcan!uunet!mcsun!ukc!warwick!nott-cs!piaggio!anw From: anw@maths.nott.ac.uk (Dr A. N. Walker) Newsgroups: comp.edu Subject: Re: Automatic checking the students' answers Message-ID: <1990Oct26.163410.29399@maths.nott.ac.uk> Date: 26 Oct 90 16:34:10 GMT References: <1990Oct19.103605.5035@maths.nott.ac.uk> Reply-To: anw@maths.nott.ac.uk (Dr A. N. Walker) Organization: Maths Dept., Nott'm Univ., UK. Lines: 56 In article juh@cs.hut.fi (Juha Hyv|nen) writes: >In numerical analysis, the (unambiguous) answer to a problem cannot be >known if you do not know any method to solve it. In our case, the >"answers" are trivial (e.g., a sorted array). We are not interested in >the answer. We are interested in the method (e.g. quicksort). That >makes it more difficult because the representation of the *method* >must be parsable (and as natural for the student as possible). Then you must ask a different question. If you want your students to know about sorting, get them to write a sorting program, and test it. Your automatic marker will put it through a collection of tests, time it, check the output for correctness, and mark accordingly. If you want them to know *very* *specifically* about quicksort, then (a) you can get them to write their answers in (eg) Pascal, and parse the results; (b) you can ask them for the state of the array after so-many partitions using such-and-such pivots. In my NA system, some of the questions were of this type; for example, one might ask what approximation to a root was obtained by three cycles of Newton-Raphson starting from x = 1. >In our case (of practical test), the results are irrelevant. We want >the computer to see the method. The syntax of representing the method >is the key issue. Computer scientists, of all people, should know about syntax of representing methods! What else is a programming language for? >Intelligently dealing with partly correct answers is perhaps the >hardest requirement to meet. OK. For example, my NA system looked to see if the student's solution differed from the correct one in various standard ways -- wrong sign, out by a power of ten, out by an integer factor, out by using degrees instead of radians, etc. If you can guess at plausible incorrect answers, you can look for them. > The answer should therefore consist of >distinct steps. If the result of one intermediate step is wrong, it >should be taken as a basis of the next step and not the correct result >(of the step). This is sometimes, but not always, possible. For example, if a sort step is followed by a merge step in which the input is assumed to be sorted, then a sort failure is potentially disastrous. There is (perhaps) no way of properly assessing the merge unless the sort is correct. >(Could your equations be solved in a single step?) No. (Can they ever?) -- Andy Walker, Maths Dept., Nott'm Univ., UK. anw@maths.nott.ac.uk