Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!gem.mps.ohio-state.edu!ginosko!uunet!portal!cup.portal.com!mmm
From: mmm@cup.portal.com (Mark Robert Thorson)
Newsgroups: comp.arch
Subject: Re: flexible caches
Message-ID: <22151@cup.portal.com>
Date: 14 Sep 89 18:57:16 GMT
References: <224@qusunr.queensu.CA>
Organization: The Portal System (TM)
Lines: 40

A very smart cache controller might gather statistics on the usage of
particular memory locations, which might be used to decide which memory
locations to cache.

However there are many possible cases in which two programs seem to make
the same use of some memory locations, but one program would be very
sensitive to the caching of these locations, and the other would not.

In these cases, an even smarter cache might make use of a neural network
to recognize which program should get caching of these memory locations.
The training signal for this network could come from some indicator of
the speed of execution of the program, for example number of instructions
executed during a sample period.  Or, the compiler might insert code to 
provide each program with a heartbeat visible to the cache.

The data gathered on caching effectiveness might be part of the module
loaded when the program is run.  And of course, it would be written back
to disk when the program terminates.  A smart compiler might prepare an
initial set of predicted dynamic characteristics for the module, and the
neural network could optimize these characteristics with data gathered
from the actual use of the program.

The caching effectiveness of particular memory locations might even change on
on the fly.  For example, a program might have some distinct operating modes,
like restructuring a tree and refreshing a display.  This is like having two
separate programs knit into one.  A neural network which learned and forgot
quickly might make determinations like "Hey!  Caching address 0177566 doesn't
work anymore.  Try caching 0000400!"  Another possibility is that a register
would be updated by the program to indicate what mode it's in.  The cache
could then select different dynamic characteristics for each mode.  Programs
which do not provide this information might have their different modes
recognized by correlating dynamic behavior with references to particular
areas in their instruction space (e.g. when the REFRESH_WINDOW subroutine
is running, don't bother to cache anything but the display list).

The nice thing about this application for neural networks is that nothing
croaks if the neural network occasionally makes a mistake.  If the net
mistakenly decides to cache one memory location in preference to another,
no big deal.  The system just runs a little slower until the net realizes
it has made a less than optimal decision, and corrects itself.