Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!zaphod.mps.ohio-state.edu!rpi!uupsi!TALOS!jerry
From: jerry@talos.npri.com (Jerry Gitomer)
Newsgroups: comp.admin.policy
Subject: Re: User Satisfaction ?
Message-ID: <2299@talos.npri.com>
Date: 23 May 91 16:54:44 GMT
References: <1991May21.152727.27423@eagle.lerc.nasa.gov>
Organization: NPRI, Alexandria VA
Lines: 62

uudot@ariel.lerc.nasa.gov (Dorothy Carney) writes:



:Our upper management, which is not very computer literate, wants to
:receive quarterly reports (oral: 15 minutes!) which are "METRICs" of
:customer satisfaction.  Our customers are hundreds of researchers and
:engineers, as well as secretaries and office staff.

:Do any of you have metrics for user satisfaction?  

	I don't happen to have any metrics handy, but I have had similar
	assignments in the past.  Based on my experience I believe that
	"User satisfaction" is based on perception rather than reality.
	The first thing you have to do is get your users to tell you 
	what they want -- in quantifiable terms.  For example, if they 
	tell you that terminal response time is to long ask them "How 
	long should it be?".  Sure, you will have a lot of data to weed
	through (some of it ridiculous), but you will then be able to
	establish some quantifiable goals and compare your performance
	to the goals.

:Somehow, telling upper managers about the mean time between failures
:or the integrity of disk data doesn't do it. 

	You're right.  As someone else pointed out it is more important
	not to fail at the wrong time than not to fail at all.  Also both
	managers and users expect the system to maintain the integrity of
	the data.  (If your system doesn't you have some real problems
	that you better take care of NOW).
	
:Neither does surveying
:a random sampling of users with insipid questions like "On a scale
:of 1 to 10 ...".

	Yes, but this isn't a bad idea for an initial survey.  Take 
	the 1 to 10s and formulate questions with meaningful choices.
	For example on the initial survey you might ask "Please rate
	terminal response time on a scale of one to ten"  If the bulk 
	of your users respond in the range of 8 to 10 you can report
	to management that terminal response time is satisfactory and
	go on to something else.  If the bulk of your responses are 7
	or lower you might come back with a question on the second
	survey which asks the users to pick an acceptable terminal 
	response time from a number of choices such as: under 5 seconds,
	6 to 10 seconds, etc.  (If they all ask for under 5 seconds and
	your system can't support better than 10 seconds -- tell top
	management that the users can't be satisfied unless you obtain
	more equipment).  In any event this will give you a set of goals
	to measure against.

:We do have a Help Desk which tracks telephone requests for help ...
:but focusing the metrics on problem reports would be negative, and we
:want to be positive.

	How about the percentage of your users who don't call the Help
	Desk?  If they don't need help they at least know how to use
	the system to get their jobs done.  
-- 
Jerry Gitomer at National Political Resources Inc, Alexandria, VA USA
I am apolitical, have no resources, and speak only for myself.
Ma Bell (703)683-9090  (UUCP:  ...uunet!uupsi!npri6!jerry )