Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!rutgers!tut.cis.ohio-state.edu!pt.cs.cmu.edu!cive.ri.cmu.edu!hmp
From: hmp@cive.ri.cmu.edu (Henning Pangels)
Newsgroups: comp.realtime
Subject: Re: Shared memory with multiple processors running VxWorks
Summary: Another way of using shared memory in vxWorks
Keywords: VxWorks/multiple processors/ shared memory
Message-ID: <6143@pt.cs.cmu.edu>
Date: 14 Sep 89 14:57:21 GMT
References: <4252@ncar.ucar.edu>
Organization: Carnegie-Mellon University, CS/RI
Lines: 45

In article <4252@ncar.ucar.edu>, vanandel@stout.ucar.edu (Joe Van Andel) writes:
> I am working on a real-time radar signal processing system that will
> use multiple 68020 processors running VxWorks on a VME backplane.
> Because of the large amount of data we are processing
> (400Kbytes/second), I can't afford to use TCP/IP or RPC interprocess
> communication methods.  As I read the manual, I conclude that VxWorks
> doesn't provide facilities for managing shared memory or semaphores
> between tasks executing on different processors.  I very much like
> VxWorks, but I feel it needs more multi-processor support.
> 

	Rather than mucking around with the sysLib routines, I modified the
Makefile and usrConfig.c used to build VxWorks. In the Makefile, I define
USER_CFLAGS = -DRESERVE_MEM=0x100000, which is appended to the regular
CFLAGS. Then, in usrConfig.c, I change the kernel initialization to read

kernelInit (TRAP_KERNEL, usrRoot, ROOT_STACK_SIZE,
            FREE_RAM_ADRS, sysMemTop () - RESERVE_MEM,
            ISR_STACK_SIZE, INT_LOCK_LEVEL);

Of course, you only want to do this for the processor board on which
the shared memory actually resides - on all other processors in the system,
your application code will have to know where you've mapped your memory
spaces in order to correctly share the memory which you've reserved above.
To coordinate access to the shared memory region, I use the vxTas() routine,
(which is all that is called by sysBusTas()). We have implemented a
rudimentary "backplane pipe" using this mechanism, which uses mailbox- or
backplane interrupts (depending on the processor board used). Even the very
first un-optimized experimental version is almost 10 times faster than
going through the overhead of TCP/IP sockets - as usual, it's possible
to trade off some portability and generality in favor of performance.

To anyone from WRS who might be listening: I agree with comments made
by others that some mechanism like this should be made part of the vxWorks
package.

As an aside: Be careful about mapping several processor's memory spaces
contiguously -- some versions of the sysMemTop() routines work by probing
for live memory, so if there's no memory gap between boards, one processor
might actually claim another's memory for itself.

-- 
Henning Pangels                                   Field Robotics Center
ARPAnet/Internet: hmp@cive.ri.cmu.edu             Robotics Institute
(412) 268-6557                                    Carnegie-Mellon University