Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!dimacs.rutgers.edu!seismo!uunet!mcsun!ukc!keele!nott-cs!piaggio!anw From: anw@maths.nott.ac.uk (Dr A. N. Walker) Newsgroups: comp.edu Subject: Re: Automatic checking the students' answers Message-ID: <1990Oct19.103605.5035@maths.nott.ac.uk> Date: 19 Oct 90 10:36:05 GMT References: Reply-To: anw@maths.nott.ac.uk (Dr A. N. Walker) Organization: Maths Dept., Nott'm Univ., UK. Lines: 63 In article eibo@rzsun3.informatik.uni-hamburg.de (Eibo Thieme) writes: >juh@cs.hut.fi (Juha Hyv|nen) writes: [automated question/mark/answer server description deleted] > 1. The set of questions askable is confined to the area > of reproducing memorized facts and application of > memorized rules. (Anybody out there in AI-land daring > to oppose ? :-) I'm not in AI-land, but this is not true. The difficulty lies in making the answers easily parsable. I have been using such systems since 1970 to set and mark problems in Numerical Analysis. The questions were askable because they conformed to a simple template in which certain elements could be randomised. What facts the students had memorised, or what rules they used was in no way a factor in the system, which worked because the computer was at least able to assess the answers. [Initially, the computer was used primarily to generate a "random" question sheet (for the student) and the corresponding correct answers (for me); once the computer power became available, the whole system became interactive, and the student responses were assessed by the computer. As the responses were always numbers or (occasionally) a menu selection, the computer either understood the response or could reject it as "ungrammatical". All interactions were logged, and students could include comments/queries/complaints in the log for me to deal with.] > 2. Knowledge is *always* embedded, being part of a person > acting in the world. To pay attention to this *central* > aspect of knowledge it requires direct interaction, > which in this context means aural examinations. I wanted to find out whether the students could *do* NA. The course *also* (naturally) included theoretical knowledge, assessed in the traditional ways, but *practical* knowledge can often, with imagination, be assessed mechanically. > 3. People tend to forget the nature of learning, believing > it being nothing more than is required. [I'm sorry, but I don't understand this point.] > 4. Much energy will be spent to conform system requirements, > ingenious ideas will be considered false. Again, people > will see the method, not the contents. No. In a practical test, the method and the ingenuity are irrelevant. The computer sees the results. If I ask you to solve an equation, or to perform a quadrature, and you get the [unambiguous!] right answer in a reasonable time, fine, no matter how you do it. If the answer is wrong, it's no mitigation that you used a very ingenious method. [NA is not, of course, the only area of knowledge susceptible to automated assessment. One to watch out for, on the horizon, is the driving test.] -- Andy Walker, Maths Dept., Nott'm Univ., UK. anw@maths.nott.ac.uk