Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!casbah.acns.nwu.edu!ils.nwu.edu!sandell
From: sandell@ils.nwu.edu (Greg Sandell)
Newsgroups: comp.music
Subject: Timbre Perception and Orchestration
Message-ID: <2118@anaxagoras.ils.nwu.edu>
Date: 15 Jun 91 18:56:48 GMT
Sender: news@ils.nwu.edu
Reply-To: sandell@ils.nwu.edu (Greg Sandell)
Distribution: world
Organization: The Institute for the Learning Sciences
Lines: 171

Vance Maverick writes:

> To reverse the decay of comp.music into rec.music.synth.backup, Greg
> Sandell proposes chasing all the synth people away, and then says such
> researchers as he have no time to contribute.  I think the latter is
> the real problem -- the only way to influence the tone of the newsgroup
> is by positive contributions. 

Bravo, Vance.  I'll make a contribution too.  Get ready, though, it's
150 or so lines long...

For the last three years I have been doing research on the perception of 
musical timbre, with an eye towards learning something about orchestration.
I am working on a Ph.D. in Music Theory at Northwestern University.
Orchestration is not considered normal territory for a music theorist,
but I don't think that this is a very good state of affairs for the
field.  The current interests which dominate music theory, still mostly
pitch set theory and Schenker theory, further the belief that the key
to unlocking a piece's meaning is in discovering some tightly organized
network of pitch relations.  As a result works that are masterpieces in
part due to their orchestration yet which exhibit pitch structures resistant
to both pitch set theory or Schenker theory are in effect snubbed by music
theorists.  And this is in stark contrast to the interests of composers,
who are quick to recognize effective exploitation of the timbral 
domain and at times exhibit a language for discussing qualities of
sounds that theorists are not privy to.

So what can a theorist do to bring orchestration into the fold of
music theory?  Remember, theorists are creatures who shun vague,
qualitative descriptions of personal listening experiences (although
lately narrative analyses have come in fashion) and seek to create and
use tools that categorize and quantify the elements which make up
a piece of music.  My choice was to focus on that aspect of orchestration
pertaining to choosing combinations of instruments for concurrent
presentation.  This is one of the great mysteries of orchestration which 
fascinates many musicians, and which is clearly not merely a pragmatic
issue of `instrumentation.'  Next, I chose to restrict the field  of
interest to largely homophonic combinations:  melodies in unison other
semi-fixed intervals, or vertical sonorities.  (This is purely practical;
research has to start somewhere.)  But before can categorizing or
quantifying of this domain can begin, we need to know how such combinations
are evaluated in orchestration practice.

Many orchestration manuals, especially the ones by Rimsky-Korsakov,
Rogers and Piston, spend a great deal of time instructing the student
how to choose timbres that "blend."  With few exceptions, the use of the
term suggests a fusion phenomenon:  blended combinations are those
in which the timbral line of demarcation cannot be distinguished, e.g.,
a cello and bass clarinet which merges into some hybrid, single timbral
quality.  The other end of this spectrum are combinations which clearly
separate into distinct timbres (tuba and piccolo, to name an extreme
example).  Other than the obvious factors that separate timbres (gross
differences in attack time, intonation, etc.), what are the acoustics
underlying the phenomenon of blend?  Although orchestration manuals offer
plenty of prescriptions (do's and don'ts) and examples ("Ravel did it,
so it must be good"), they offer no metric for evaluating any particular
instance.  High speed analysis and visualization of timbre by computer
is here and it's cheap; Rimsky-Korsakov's and Joseph Schillinger's dream
of a scientific basis for orchestration is within reach, or at least,
the ability to investigate its possibility is.  (I might add that Piston
and one or two others deplored the idea of a systematization of
orchestration, but there will always be conservatives.)  

The ideal orchestration manual would offer the student not only isolated cases,
but empower him or her with (to borrow a concept from my current
employer, Roger Schank) case-based reasoning.  Students could
generalize from the acoustical properties which lead to the good blend 
between cello and bass clarinet and apply the principle
to other combinations:  for example, perhaps low-register violin and
English horn blend for analogous reasons.  While blend is certainly not
the only quality one evaluates in concurrent timbres, I decided to
focus on blend and investigate it in a series of perceptual experiments
using musical listeners.

I ran three experiments using sysnthesized musical instrument tones
(same tones as in John Grey's 1975 dissertation).  Tones were presented in
concurrent pairs, and listeners rated how well they "blended".  Poor
blend was defined as the case where individual timbres could be
clearly heard (say, piccolo and tuba) and good blend was defined
as the case where the timbres fused to form a single sonic impression
(say, French horn and trumpet).  A single trial in one of my 
experiments consisted of two instruments playing one short note each, 
in rhythmic unison.  

I explored instruments in unison and in minor thirds, and timbral
modifications such as "artificially bright", "increased inharmonicity",
and unequal intensity levels to observe the effect on "blend."
The main findings have to do with the centroid (level of brightness/
darkness) and the length of the attack portion.  (Examples of dark timbres 
are:  bassoon, French horn; bright timbres are oboe, clarinet in clarion 
register; instruments of moderate brightness include flute and trumpet.)
For unisons, dark tones blend best of all, and the blend steadily worsens as 
a function of the increasing brightness of one or both of the tones in a 
concurrent pair.  This was true in the case of artificially darkened or
brightened tones:  for example, you can improve the blend of oboe
and clarinet by artificially darkening the oboe (like turning up
the "bass knob" on a stereo).  Analogous effects to darkening and
brightening were found with respect to length of attack time:  the
longer the attack time of one or both of the tones, the worse
the blend.  However, the effect for attack time is not as strong as
the effect for centroid.  For non-unison intervals (the minor third),
an additional factor for centroid played a role:  instruments that
were close in centroid (similar degrees of bright/darkness) blended
well.  So, although the presence of low centroids strongly influenced
the blend of the pair, pairs not including an instrument with a low
centroid blended fairly well nonetheless if they were similarly
bright.  Still other factors appeared to involve mechanisms of
auditory stream segregation:  instruments with hightly correlated
amplitude envelopes (also, centroid envelopes, the change in centroid
over the duration of the tone) tended to blend well, while non-correlated
envelopes led to greater separation.

(Note: Unfortunately this medium does not allow me to present all the data
which led me to come to these conclusions; this requires a number of
high-resolution graphs, tables of correlation coefficients, and alot of 
space devoted to explaining exactly how each analysis was done.  But the 
statistical methods employed in coming to these conclusions 
included regression, correlation, t-tests, and Multidimensional 
Scaling, and the measures of statistical significance were 
those used in standard psychological experimentation.  Eventually 
the dissertation will be available through standard channels
(NU's dissertations are carried in University Microfilms) or directly from
me, for those who want to follow up on the details.  The dissertation
will be completed in December 1991.)

John Grey's landmark 1975 dissertation in timbre perception ("An
Exploration of Musical Timbre", Stanford University) proposed a
timbre space of three dimensions, pertaining to (and I am somewhat
generalizing here) centroid, attack and harmonic synchrony.  These
dimensions emerged from an statistical analysis of timbral 
similarity judgments from a group of musical listeners.  The results of 
the present experiments, although involving a different task,
largely corroborate Grey's timbre space.  A "blend space" can
be constructed for each instrument with centroid, attack duration and
envelope correlation as three of the primary dimensions.

The experiments I ran were obviously limited in musical breadth, so
my findings only suggest a beginning for further research in timbre
perception and orchestration.  First of all, one would also want
to investigate blend using melodies rather than isolated note-
pairs, and other intervals should be investigated.  Furthermore,
other ways of evaluating concurrent timbres could be explored.  For
one thing, the relative salience of two timbres in a pair is an
important factor:  loud flute and soft trumpet is very different than
soft flute and loud trumpet.  Part of how we may evaluate those combinations
depends on which of the two qualities dominate the sum perception of
the sound (e.g., flute in the former, trumpet in the latter).  Next,
it is expected that masking among harmonics or masking due to noisy
aspects of instruments (air streams and bow scrapes) play an important
role in evaluating combinations.  One frequent observation in orchestration
manuals is that the flute somehow "softens" the effect of other 
instrumental combinations (for example, Rimsky Korsakov says the flute
can soften the harsh combination of oboes and clarinets; see p. 78 of
his orchestration manual).  The way in which some timbres seem to 
modify others is a point of interest to many musicians, and experiments
to explicate these effects would make an interesting investigation.
Finally, one can investigate the acoustic dissonance of a pair of
timbres based on Plomp and Levelt's measures of "roughness" (in fact,
I haven't eliminated this as a possibility in my dissertation).

Greg Sandell  (sandell@ils.nwu.edu)
Northwestern Computer Music
Northwestern University, Evanston, IL

p.s. I call my research area "Concurrent Timbre."  Eliot ("Music Mediates
Mind[tm]") Handelman, if you're reading this, I could use some advice
on how I can patent this phrase and license it for profit... :-)

-- 
Greg Sandell
sandell@ils.nwu.edu