Path: utzoo!utgpu!news-server.csri.toronto.edu!bonnie.concordia.ca!thunder.mcrcim.mcgill.edu!snorkelwacker.mit.edu!paperboy!hsdndev!wuarchive!zaphod.mps.ohio-state.edu!magnus.ircc.ohio-state.edu!tut.cis.ohio-state.edu!ucbvax!CS.BUFFALO.EDU!neal From: neal@CS.BUFFALO.EDU (Jeannette Neal) Newsgroups: comp.ai Subject: Call for Participation Message-ID: <9102182045.AA03896@sybil.cs.Buffalo.EDU> Date: 18 Feb 91 20:45:17 GMT Sender: daemon@ucbvax.BERKELEY.EDU Lines: 80 CALL FOR PARTICIPATION Workshop on Evaluation of Natural Language Processing Systems 18 June 1991 University of California Berkeley, CA There has been increased concern with the evaluation of natural language processing (NLP) systems over the past few years. The evaluation of NLP systems is essential in order to measure the capabilities of individual systems, to measure technical progress and growth in the field, and to provide a basis for selecting NLP systems to best fit the communication requirements of application domain systems. This 1991 Workshop is a follow on to the workshop on evaluation held in December of 1988 at the Wayne Hotel in Wayne, PA. Technical report RADC-TR-89-302 on the previous workshop is available from Rome Laboratory. Important issues for any evaluation effort and relevant to this workshop include identification of the items or capabilities to be evaluated, choosing between "black box" and "glass box" approaches, definition of evaluation criteria, development of methods or procedures for evaluation, determination of evaluation metrics, and determination of the type of output to be produced by the evaluation procedures. The areas of NLP relevant for this workshop include syntactic analysis, semantic analyisis, pragmatic analysis, lexical processing, morphology, sharable knowledge bases and ontologies, speech understanding, and trainable systems. The purpose of this workshop is to provide a forum for computational linguists to report on and discuss current efforts and activities, research progress, new approaches, problems and issues; to promote scientific interchange on important evaluation issues; and to generate recommendations and directions for future investigations in the evaluation area. Workshop attendance will be by invitation, limited to 45 people. The workshop will be held June 18th at the University of California, Berkeley Campus, in association with the 29th Annual Meeting of the Association for Computational Linguistics. SUBMISSIONS: Interested participants should submit a 3-5 page abstract of their presentation and a brief description of their research activities. Persons desiring to attend the workshop, but not make a presentation, should send only a brief description of their research activities. All persons should include name, mailing address, phone number, and electronic mail address. Submission may be transmitted via electronic mail, U.S. Postal Service, or FAX. If hardcopy is submitted, please include six copies (including the original). Send submissions to: Jeannette G. Neal, Ph.D. Calspan Corporation P.O. Box 400, Buffalo, NY 14225 (716) 631-6844 FAX: (716) 631-6722 neal@cs.buffalo.edu SCHEDULE: March 1, 1991 Submissions due (changed from February 1) April 1, 1991 Notification of acceptance/invitation ORGANIZATION AND PROGRAM COMMITTEE: Jeannette G. Neal, Calspan Corporation (Committee Chair) Tim Finin, Unisys Center for Advanced Information Technology Ralph Grishman, New York University Christine Montgomery, Language Systems, Inc. Sharon Walter, Rome Laboratory SUPPORT for this workshop is provided by Rome Laboratory. For Technical Report RADC-TR-89-302, contact: Sharon Walter Rome Laboratory Griffiss AFB, New York 13441-5700 USA E-mail: walter@aivax.radc.af.mil