Path: utzoo!mnetor!uunet!husc6!bloom-beacon!athena.mit.edu!yba From: yba@athena.mit.edu (Mark H Levine) Newsgroups: comp.ivideodisc Subject: Visual Courseware and the X Window System platform (long) Message-ID: <2299@bloom-beacon.MIT.EDU> Date: 16 Jan 88 07:40:41 GMT Sender: daemon@bloom-beacon.MIT.EDU Reply-To: yba@athena.mit.edu (Mark H Levine) Organization: Massachusetts Institute of Technology Lines: 178 Keywords: X This is the second of two postings describing work at MIT in interactive video disc related areas. It is a copy of the article recently published in the first issue of the X User's Group Newsletter. I hope the slant toward the X User audience does not make it less useful. Visual Courseware and the X Window System In the excitement surrounding X today, it is easy to forget that four years ago its development was partly motivated by Project Athena's need to support teaching. While most of the applications on the Athena system use only text and graphics, ten development projects also use video. This article presents a brief introduction to Athena's effort to integrate X with video. Video at the workstation allows a student to actually see and hear native speakers on the streets of Paris while studying French; to watch motion video of bearings operating under conditions the student selects; to browse through an interactive visual database of the human brain; and to interact with the material through the many possibilities of the computer, rather than just to view a video tape pas- sively -- the student controls the language tape through a natural (foreign) language recognizer. Video as a network resource allows the sharing of still and moving picture libraries, computer enhanced images, closed circuit televi- sion and video recordings of all types. At Project Athena, the Visual Courseware Group (VCG) supports the integration of video material and optical media into our workstation environment. Specifically, our research and development efforts target the extension of the X Window System to handle both motion and still video as the contents of a window and network retrieval of pictures and other visual materials. In general, VCG develops tools for the preparation and use of video and image data as part of an integrated display system that also includes both computer text and graphics. To date we have successfully integrated video with X Version 10 Release 4 (X10), and we are now adding this capability to X Version 11 (X11). VCG uses a "visual workstation" specially configured to deliver these materials; this station includes a 1280x1024x8 RGB color display, three-button mouse, twelve-inch optical disk players, stereo audio, and access to both the MIT cable TV system and the Athena network. The visual workstation may be based on either the DEC Microvax-II or IBM RT PC, and depends heavily on the Model 1280 videographics processor from Parallax Graphics, Inc. The user of a visual workstation can see both moving and still video as the contents of a window. Video windows can be moved, resized, and iconified; they behave for the most part like any other X window. Still frames can be scaled and distorted by the resize operation, and both motion and still pictures can be clipped to a window region. Graphics can be drawn over video images. Under X10, the programmer's interface to video exten- sions is primarily contained in a set of four routines which exploit a "hole" in the Xlib's error checking to communicate with the X server for the Parallax 1280; they provide the basic operations: XStartVideo(w,srcpix,sx,sy,prio) Windoww; intsrcpix, sx, sy, prio; { if (!nullbm) nullbm = XStoreBitmap(1,1,&sx); XPixFill(w,0,0,2048|sx,2048|sy,0,nullbm,1,1); XPixFill(w,0,0,2048|prio,2048,srcpix,nullbm,0,1); XSync(FALSE); } XStartVideo () will start flashing live video into existing X window "w" -- it allows the user to clear the background to the pattern in "srcpix" and to choose the upper left corner of the full size video source frame which becomes the upper left corner of the displayed window with "sx" and "sy". "prio" is a number indicating the relative priority of graphics versus video. Similarly, XStillVideo() puts up a still frame, and XScaleVideo () will shrink or expand a still frame as it is displayed. Pseudo-color values which will be drawn over video regions may also be set with a call to XSetVideoColor (). For use with X11, we plan to implement a more complete video extension library. The actual implementation of X10 on the videographics processor was a joint effort by Robert L. Goodwin of Paral- lax and the author. The first version was finished early in 1987 and has been a base for VCG applications work at MIT since then. An X11 port begun last summer is still under development. Application work currently in progress includes: 1. A Display Handler to control presentation of visual material based on plain text instructions (author- ing support). 2. Tools for making menus and other (X Toolkit) objects at a higher level than that provided by X. 3. Glossary support: tools to look up words presented on the screen in a glossary and to present the explanations found (for example, this will be used in the subtitles of language films and to explain medical jargon). 4. General text browsing and note-taking tools which will connect to the glossary tools. 5. Image processing software to display parts of large images or assemble large images from several parts or frames (this will support enlargements of com- plex images and the assembly of composites from several sources). 6. "Tagging" software to associate "objects" with parts of a picture. 7. A video-editing suite for use in retrieving and compiling materials (preparation of reports and lectures based on library materials, for example, or the production of a video disk). 8. A timeline editor: for synchronizing time-dependent elements in a multi-media display. For example, text subtitles for a motion video, perhaps with alternate sound tracks. 9. An image delivery service. This would, for instance, provide retrieval of video and data stored on optical disks from a central location over the network to the video workstation (using the baseband network for control and the broadband network for transmission). With such tools, we intend to demonstrate the impor- tance of video materials and optical media as a part of both educational and workstation technologies, and to simplify authoring of applications which use these technologies. I would like to personally thank Ken Irvin of MITRE, Frank Janes of NASA, Lester Ludwig of BellCoRe, and all at Parallax for their encouragement and support of the X11 effort. Further information about Athena may be obtained and visits may be arranged through contacting Dr. Kenneth A. Goldman, E40-358F MIT, Cambridge, MA, 02139 [e-mail: democenter@athena.mit.edu; telephone: (617) 253-0194]. Further information about Parallax products can be obtained from Marty Picco, Parallax Graphics Inc., 2500 Condensa Street, Santa Clara, CA, 95051 [e-mail: picco%parallax.uucp@sun.com; telephone: (408) 727-2220]. The author is solely responsible for the accuracy of the information given; no statement should be construed as necessarily representing the position of any of the organi- zations mentioned. (The author is I).