Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!yale!cmcl2!adm!smoke!gwyn
From: gwyn@smoke.brl.mil (Doug Gwyn)
Newsgroups: comp.lang.c
Subject: Re: RMS deviation between two sets of cartesian coords
Message-ID: <14666@smoke.brl.mil>
Date: 6 Dec 90 19:34:58 GMT
References: <14660@smoke.brl.mil> <27625@cs.yale.edu>
Organization: U.S. Army Ballistic Research Laboratory, APG, MD.
Lines: 18

In article <27625@cs.yale.edu> jim@doctor.chem.yale.edu (James F. Blake) writes:
-From article <14660@smoke.brl.mil>, by gwyn@smoke.brl.mil (Doug Gwyn):
-> In article <27599@cs.yale.edu> jim@doctor.chem.yale.edu (James F. Blake) writes:
->-     I am looking for code to compute the RMS deviation between two
->-sets of cartesian coordinates.
-> Sounds to me like the sums of the squares of the (vector) differences
-> between corresponding coordinates would suffice --
-     That was the approach I took first.  It turns out that their is a
-significant amount of "round off" error inherent in the least-squares
-equations when solved in this manner.  If I computed the RMS deviations
-between structures A -> B and B -> A, I would see as much as a 5%
-difference in answers.

It's hard to see how you could get 5% fuzz in a variance computation
if you use the direct definition (sums of squares of deviations).
The textbook rewrite as mean-of-squares minus square-of-mean is well
known to produce bogus answers, however, particularly when the variance
is relatively small.