Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!think.com!zaphod.mps.ohio-state.edu!mips!apple!amdahl!fai!bruceb
From: bruceb@fai.UUCP (Bruce Bailey)
Newsgroups: comp.graphics.visualization
Subject: Stereo User Interface Issues
Message-ID: <3066@fai.UUCP>
Date: 21 May 91 20:43:45 GMT
Reply-To: bruceb@fai.fai.com (Bruce Bailey)
Distribution: na
Organization: Fujitsu America, Inc
Lines: 63


Hi.  I've been developing a c++ GUI toolkit that supports both mono and
stereographic windows on a high-end SGI platform.  I'm interested in sharing
ideas, experiences, technical references, etc from interested people out there
in net-land.

In my experience, commercial GUI toolkits (regardless of the language/software
design paradigm) fall down in a stereo environment.  The reason is that (assuming
a StereoGraphics Corp. hardware scenario) the user-interface entities need to
appear simultaneously in both the bottom and top halves (left and right eye views)
of the screen.  To make matters worse, these 2 corresponding UI entities must 
appear synchronized in each view (imagine a pulldown menu in 2 different views,
where buttons are simultaneously highlighted and unhighlighted as the cursor moves
up and down). Another similar nightmare occurs with the cursor, since my
(and probably most other) graphics hardware cannot support a dual-view cursor setup.

Basically, I've been handling the above problems by creating classes
for windows, user-interface entities, display list, and so on.  When a mono
window becomes stereo it takes over the entire screen (as per the hardware 
requirements).  The only thing that needs to worry about stereo/mono display
is the window object itself, which resolves stereo/mono display issues.  The
cursor, menus, display list, and so on are all passes to the window for display.
I've addressed the cursor problem by implementing a software cursor, which
is just another object derived from the UI class.

Because I didn't want to write a full blown window system supporting both
stereo and mono modes, I decided to go with sort of a poor-man's windowing
environment instead.  My windows support multiple viewports, where each
viewport has its own display list, transform stack, and so on.  Then, when the
window becomes stereo (full screen), whichever viewport the cursor is in
becomes "active", i.e., user commands are applied to the stuff in that
viewport.  Viewports cannot overlap.

Currently, all my user interface objects are 2D and displayed at the plane
of the screen.  I'm toying with the possibility of going to 3D UI objects,
which raises fascinating possibilities.  For example, a 3D cursor controlled
by a 3 axis input device, where the cursor is displayed in stereo.  If the
user pops up a menu, the menu could pop up out in front of the screen!
Or, if the cursor is deep in the space of the display device and the user 
invoked a pulldown menu, the menu would come up at the same Z-depth as the 
cursor.  

This raises many design issues. Should UI objects in 3-space be z-buffered
against stuff in the display list (conflicting cues may exist otherwise, e.g.,
stereo cues versus z-depth occlusion)?  If z-buffering is used, the user would
then be forced to move the UI object to an unobstructed area of the viewport/
screen.  Or, suppose a decision is made to pop all UI menus out in front of 
the screen to avoid the occlusion/stereo cue conflict.  With a pulldown menu
the cursor would have to be popped out to the same z depth as the menu.  Would
this drive users crazy?

Please respond with any ideas/experiences/questions you have.  I'll summarize 
comments for possible future posting (if there is sufficient interest).

Thanks in advance!

Bruce Bailey (Dr. OW)

Computational Research Dept.
Fujitsu America, Inc.


bruceb@fai.com