Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!snorkelwacker!bloom-beacon!atrp.mit.edu!ashok
From: ashok@atrp.mit.edu (Ashok C. Popat)
Newsgroups: comp.dsp
Subject: Re: FFT vs ARMA
Message-ID: <1989Nov26.194904.1376@athena.mit.edu>
Date: 26 Nov 89 19:49:04 GMT
References: <5619@videovax.tv.tek.com> <10208@cadnetix.COM> <1989Nov22.170850.21777@athena.mit.edu> <3589@quanta.eng.ohio-state.edu>
Sender: news@athena.mit.edu (News system)
Reply-To: ashok@atrp.mit.edu (Ashok C. Popat)
Organization: MIT
Lines: 60

In article <3589@quanta.eng.ohio-state.edu> rob@kaa.eng.ohio-state.edu (Rob Carriere) writes:
>In article <1989Nov22.170850.21777@athena.mit.edu>, ashok@atrp.mit.edu (Ashok
>C. Popat) writes: 
>> In applications, you don't always have a good apriori formal model.
>> Unless you have a formal model that's *useful* for your application,
>> parametric estimation is worthless.
>> 
>> Suppose I gave you some data (say 10^6 samples) and told you that the
>> source was ergodic, but nothing else.  How would you estimate the
>> spectrum?  If you used an ARMA model, how would you decide what the
>> order of the model should be?  Wouldn't you have much more confidence
>> in an averaged-periodogram (i.e., DFT-based) estimate?  I would.
>
>Nor necessarily.  DFT is quite good at some things, not at others.  If you
>give me recorded data that I can play with for a while, I would probably run
>FFT, several different periodograms, ARMA or Prony models of several orders
>and whatever alse the data made me feel like.  After doing all that, I'd feel
>reasonably confident I could tell you something about your data.

     Sounds reasonable on the surface --- try a few well known techniques,
     then sort of mentally average the results to conclude something about
     the data.  The problem is that there is absolutely no justification
     for trying some of the techniques.  What you want is a consistent,
     unbiased estimate of the spectrum of an (unknown) ergodic random
     process, given a bunch of samples.  Averaging periodograms (e.g.,
     Welch's method) gives you a consistent, asymptotically unbiased
     estimate.  What a parametric technique gives you depends strongly on
     the assumed model (which isn't given as part of the problem).

>If averaged periodograms showed different behavior in different segments of
>the data, that means you also want to look at parametric models over subsets
>of the data.

     Nope, ergodicity implies stationarity.  You'd have to attribute the
     behavior to chance.

>In short, if I knew that little no ONE technique would make me happy.  And

     Any consistent, unbiased, and efficient estimate should make you
     happy.  An estimate based on unfounded assumptions should not.

>finally, the fact that the DFT is non-parametric does not mean that you aren't
>making assumptions about the data (in fact, you're assuming periodicity --
>something that doesn't always make sense either)

     You are making assumptions, but periodicity isn't one of them.
     Remember, DFT-based spectral estimation *doesn't* mean simply
     computing the DFT of the data.  In fact, it is well known that a
     single periodogram (the magnitude squared of the DFT) is a
     particularly shitty spectral estimate, since it is biased, and since
     its variance doesn't diminish with the amount of data you use (see
     Oppenheim and Schafer, Chapt.  11).  DFT-based spectral estimation
     usually involves some sort of averaging of short, modified
     periodograms.  Now, an argument can be made that this type of
     estimation also assumes something about the data, but the concept is
     subtle.  I suggest Ronald Christiansen's "Entropy Minimax Sourcebook"
     for philosophical/technical discussions of problems in statistical
     inference.

     Ashok Chhabedia Popat  MIT Rm 36-665  (617) 253-7302