Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!husc6!m2c!jjmhome!cloud9!cme
From: cme@cloud9.Stratus.COM (Carl Ellison)
Newsgroups: comp.graphics
Subject: Re: Matrix Operations on Normals
Summary: a solution
Message-ID: <3530@cloud9.Stratus.COM>
Date: 4 Feb 89 18:53:14 GMT
References: <2392@carthage.cs.swarthmore.edu>
Distribution: na
Organization: Stratus Computer, Inc., Marlboro, MA
Lines: 27

If you have either a plane equation or just a normal (4 or 3 components
respectively; homogeneous coordinates or raw 3D) which you want to
rotate via some matrix, M, (4x4 or 3x3, respectively) the answer is
found in math or physics books on tensors.

Between a vertex, v, and a normal, n, one is covariant and the other is
contravariant.  (I'm not sure, but I think the normal is covariant...
That doesn't matter so much for what follows.)

The solution comes from the fact that the dot product v.n is an invariant.
That is, if you rotate coordinate systems, the scalar result of v.n
should remain unchanged ... or v'.n' (each in the new system) should
= v.n

Assuming v is a row vector so that v' = vM, the normal would be a column
vector and n' = inverse(M)n.

v'.n' = v M inverse(M) n = v.n

It's been ages since I worked this out for matrices with both scaling and
skew.  I'll leave that to fellow netters.  I'm away from graphics devices
at the moment.

Enjoy,

--Carl Ellison                      UUCP::  cme@cloud9.Stratus.COM
SNail:: Stratus Computer; 55 Fairbanks Blvd.; Marlborough MA 01752
Disclaimer::  (of course)