Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.1 6/24/83; site uvm-cs.UUCP
Path: utzoo!watmath!clyde!burl!ulysses!allegra!mit-eddie!genrad!decvax!dartvax!uvm-gen!uvm-cs!hartley
From: hartley@uvm-cs.UUCP (Stephen J. Hartley)
Newsgroups: net.text
Subject: ditroff problem?
Message-ID: <351@uvm-cs.UUCP>
Date: Tue, 12-Feb-85 20:36:25 EST
Article-I.D.: uvm-cs.351
Posted: Tue Feb 12 20:36:25 1985
Date-Received: Fri, 15-Feb-85 04:37:47 EST
Organization: University of Vermont
Lines: 58


  I don't know if this is a bug in deqn, ditroff, or the DEC LN01 postprocessor
that we have (courtesy of the Unievrsity of Wisconsin).  The following ditroff
input produces some bad subscripts.  HOWEVER if one changes from 12 point to
10 point (remove .ps 12, .sz 12, .nr pp 12, gsize 12), the problem goes away.
In the line beginning "Thus, if I=...", the subscripts get closer and closer
to the variables as we go towards the right (in 12 point version).  By the time
we get to "a sub M", the "M" overlaps the "a".  Any ideas?  Thanks.

---------------------(cut here and remove leading blank)------------------------
 .\"				ditroff -me
 .ps 12
 .sz 12
 .nr pp 12
 .po 1.0i
 .ll 6.5i
 .na
 .ls 1
 .EQ
 delim @@
 gsize 12
 .EN
 .hy 1
 .pp
 A multimodule Markov Decision Process (3MDP) is a specially
 structured MDP with the following characteristics.
 The system is composed of M modules, each of which has the dynamic structure
 of an MDF, that is each module has its own state and action
 spaces and transition probabilities.
 The state of the entire system is therefore a vector
 @( s sub 1 , s sub 2 , ... , s sub M )@ where @s sub m@ is the state
 of module @m@, and likewise any action taken is a vector
 @( a sub 1 , a sub 2 , ... , a sub M )@ where @a sub m@ describes
 the action to be performed for module @m@.
 The probability of module @m@ making a transition from its own state @i@
 to @j@ when action @a@ is applied locally is given by @P sub ij sup m (a)@.
 We assume that the modules make transitions independently.
 .pp
 Thus, if @I = ( i sub 1 , ... , i sub M )@, @J = ( j sub 1 , ... , j sub M )@,
 and @A = ( a sub 1 , ... , a sub M )@, we see
 .EQ
 P sub IJ (A) = prod from m=1 to M P sub {{i sub m} {j sub m}} sup m ( a sub m ) .
 .EN
 Although the module dynamics are independent, applying a control to an
 individual module causes costs to be incurred by the overall system
 that depend on the states of the other modules.
 Hence the cost structure can only be described globally.
 Because of this dependence, a global optimal control rule cannot
 be found by simply applying the optimal controls for each module.
 It is necessary to solve the accompanying large scale MDP to find
 this optimum.
 Unfortunately, such a large scale problem is far too costly to
 solve exactly.
--------------------------------------------------------------------------------
-- 
"If that's true, then I'm the Pope!"		Stephen J. Hartley
USENET:	decvax!dartvax!uvm-gen!uvm-cs!hartley	The University of Vermont
CSNET:	hartley%uvm@csnet-relay			(802) 656-3330