Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!att!linac!pacific.mps.ohio-state.edu!zaphod.mps.ohio-state.edu!swrinde!cs.utexas.edu!asuvax!ncar!noao!arizona!amethyst!math.arizona.edu!weg
From: weg@convx1.ccit.arizona.edu (Eythan Weg)
Newsgroups: comp.lang.apl
Subject: Re: Statistical Functions in J
Message-ID: <WEG.91May21104318@convx1.convx1.ccit.arizona.edu>
Date: 21 May 91 17:43:18 GMT
References: <1991May12.145907.19563@yrloc.ipsa.reuter.COM> <356@tslwat.UUCP>
	<1991May21.042804.21102@yrloc.ipsa.reuter.COM>
Sender: news@amethyst.math.arizona.edu
Organization: University of Arizona, Economics Dept.
Lines: 45
In-reply-to: hui@yrloc.ipsa.reuter.COM's message of 21 May 91 04:28:04 GMT

In article <1991May21.042804.21102@yrloc.ipsa.reuter.COM> hui@yrloc.ipsa.reuter.COM (Roger Hui) writes:


   Lou Kates writes:

   > Nevertheless, I believe the key  concept here is not expectation,
   > probability or  measure  but regression and projection. From this
   > viewpoint the old APL's domino  operator (or regression operator)
   > had it correct and the above suggestions are a step backwards.

   > The mean    of  a  vector  V   is the  regression coefficient  of
   > projecting  V onto a vector  of all ones. The space orthogonal to
   > this vector of ones  is the deviation space and the length of the
   > projection of  V onto  this  deviation  space   divided   by  the
   > dimensionality of the deviation  space (which is  the length of V
   > minus one) is the standard deviation.

   I am puzzled.  How would a belief in regression and projection
   instead of expectation as the key concept materially affect
   the design of the primitives m., n., and s.?

   There are many statistical functions worthy of inclusion in the
   language.  We are adding the new functions mean, normalization, 
   and cross product matrix; and assigning the cognate weighted mean, 
   weighted normalization, and weighted cross product to the dyads 
   of these functions seems both consistent and useful.  These new 
   functions do not preclude adding other statistical functions in 
   the future, nor do they affect the domino function or any other 
   existing function.  Therefore, I would not characterize them 
   as a backward step.

   We would welcome suggestions for other statistical functions.

I do not want to see J as a statistical package.  But I would agree
that the mean and its cousins are basic enough to be included in
some form.   Regression is after all some derived concept for which
expectation is needed to set up the appropriate model.

But since we are invited to suggest I will voice my desire to have
generalized means etc that are some robust versions of these, were you
obtain the regular concepts as defaults.  A generalization of this to
general robust regression is welcome but I am not sure it can be done
in the spirit of J.

Eythan