Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!uwm.edu!zaphod.mps.ohio-state.edu!usc!ucsd!ucbvax!trouble.cs.nps.navy.mil!zyda
From: zyda@trouble.cs.nps.navy.mil (michael zyda)
Newsgroups: comp.sys.sgi
Subject: Help Requested on Arenas
Message-ID: <9007051819.AA09079@trouble.cs.nps.navy.mil>
Date: 5 Jul 90 18:19:45 GMT
Sender: daemon@ucbvax.BERKELEY.EDU
Organization: The Internet
Lines: 284

Help Needed with Using Arenas for Sharing Data Between Processes

     Arenas can be used to share data between processes under IRIX.
The arena is a pool of shared memory (really a memory mapped file).
When one process allocates (through usmalloc) memory from the arena,
that piece of memory is unavailable to other processes for allocation
(through usmalloc) though that piece of memory is "shared" (meaning
accessible) by the other processes.

     In order for the other processes to have access to that allocated
shared memory, the other processes must somehow obtain a pointer
to that allocated memory. How is this done safely, i.e. what
system routine do I call in process 1 to find a pointer to
the first byte allocated by usmalloc in process 2?

     I have thought of a few ways of doing this, most unsatisfactory.
I am working from the SGI Manual entitled "Parallel Programming
on the IRIS-4D" (Version 1.0). This manual has a sample program for every
chapter BUT Chapter 7, "Sharing Data through Arenas".

     (Way 1) Use System V shared memory routines (shmget, shmat)
to pass the pointers. If I have to use shmget/shmat, why bother
with arenas? shmat gets me the pointer I want.

     (Way 2) Have process 1 usmalloc all of the bytes in the arena.
When it does this, process 1 has a ptr to these bytes. Then have
process 1 return the bytes with usfree, keeping the ptr (very
unsafe). Then process 2 does a usmalloc for all of the bytes
in the arena, gets the pointer and then returns the bytes
with usfree. Now both processes have pointers to the bytes
in the arena and can read/write data there. As long as I
never again usmalloc this arena AND no other system service
does, I am ok (maybe) but feel pretty terrible about
this solution. Perhaps I can get this pointer from the
arena structure? Please show me or tell me which routine
to call to get this.

     Notes: I am concerned with independent programs that need
to share data. I am NOT talking about threads that have been
"sproc"ed or "m_fork"ed, as the sharing of the virtual address
space in such cases is the default.

     Michael Zyda
     zyda@trouble.cs.nps.navy.mil

     The following two programs illustrate Way 2.

/*

   this is file share1.c

   This program is an example of shared memory for two unrelated
   processes. This program sets up an arena and allocates some
   bytes from that arena.

   Once the arena is successfully set up and the shared memory
   has been obtained, this program then looks for a message
   in the shared memory. If this program sees a message, it
   prints out the message and zeroes the message. The zeroing
   of the message indicates to the other process that it can then 
   generate another message.

   Either program can be spawned off first.
   The programs establish communication with each other
   by trying to allocate SIZEIWANT bytes from the arena.
   If this is the first program to try and allocate that
   size buffer, then this program will get a ptr to that
   many bytes from the arena. The program will then free
   the bytes BUT keep the ptr (very dangerous).
   SIZEIWANT is carefully crafted to be greater than
   half the size of the defined arena.

   The second program to attempt to get SIZEIWANT bytes
   from the arena will fail (and loop) until the first
   program returns the bytes. When the second program
   get the bytes, it also then returns the bytes BUT
   AGAIN keeps a ptr to those bytes.

   The two programs can then use the ptrs to reference
   memory shared between them.
   This is unsafe in that if either program does another
   usmalloc() to that region, then that program will
   possibly stomp all over the data being written by
   the other processor.

   One cure to this is to then create another arena and
   properly allocate out of it, passing ptrs through maybe
   the first arena.

   Yes, the arena mechanism does define shared data but
   no mechanism provides for passing the ptrs to the
   dynamically allocated shared memory...

*/


#include <ulocks.h>   /* Includes required by the arena services */
#include <malloc.h>
#include <stdio.h>


#define MAXARENASIZE 4096   /* Max arena size for this program */

#define SIZEIWANT 3000      /* Number of bytes I want for shared memory. */


main()
{

   usptr_t *arenaptr;   /* ptr to the arena */

   volatile char *buf;  /* ptr to the start of the shared bytes */

   long i;   /* loop temp */


   /* Define the size of each arena subsequently allocated. */
   if(usconfig(CONF_INITSIZE, MAXARENASIZE) == -1)
   {
      printf("usconfig: cannot define the size to allocate the arena!\n");
      exit(1);
   }

   /* Create an arena */
   arenaptr = usinit("myarena.file");

   if(arenaptr == (usptr_t *)NULL)
   {
      printf("usinit: cannot create the arena!\n");
      exit(1);
   }

   /* Allocate a piece of shared memory for communication.
      We wait here until we can allocate the memory. We are
      going to either be first, and get the memory right
      away or we are going to wait until the other process
      frees the memory back to the arena.
   */
   while((buf = usmalloc(SIZEIWANT, arenaptr)) == (char *)NULL)
   {
      /* We do nothing in this loop but wait... */

   }

   /* Now free the buffer for reallocation */
   /* We are going to keep using this ptr though!!! */
   usfree(buf, arenaptr);

   /* zero the first byte of buf as a signal to the other process */
   *buf = '\0';

   /* loop forever...*/
   for(i=0; i < 100000000; i=i+1)
   {

     /* whenever the first byte is non-zero, print out a message */
     while(*buf != '\0')
     {
        printf("Other process says:%s\n",buf);
        *buf = '\0';
     }

   }   /* end for i */

   /* Leave the arena file, but here is where we could delete it */

}

/*

   this is file share2.c

   This program is an example of shared memory for two unrelated
   processes. This program sets up an arena and allocates some
   bytes from that arena.

   Once the arena is successfully set up and the shared memory
   has been obtained, this program then waits for the buffer
   in shared memory to be clear of the message. Once the
   buffer is clear, this program writes a message there
   and then again waits for the buffer to be clear.

   Either program can be spawned off first.
   The programs establish communication with each other
   by trying to allocate SIZEIWANT bytes from the arena.
   If this is the first program to try and allocate that
   size buffer, then this program will get a ptr to that
   many bytes from the arena. The program will then free
   the bytes BUT keep the ptr (very dangerous).
   SIZEIWANT is carefully crafted to be greater than
   half the size of the defined arena.

   The second program to attempt to get SIZEIWANT bytes
   from the arena will fail (and loop) until the first
   program returns the bytes. When the second program
   get the bytes, it also then returns the bytes BUT
   AGAIN keeps a ptr to those bytes.

   The two programs can then use the ptrs to reference
   memory shared between them.
   This is unsafe in that if either program does another
   usmalloc() to that region, then that program will
   possibly stomp all over the data being written by
   the other processor.

   One cure to this is to then create another arena and
   properly allocate out of it, passing ptrs through maybe
   the first arena.

   Yes, the arena mechanism does define shared data but
   no mechanism provides for passing the ptrs to the
   dynamically allocated shared memory...


*/


#include <ulocks.h>   /* Includes required by the arena services */
#include <malloc.h>
#include <stdio.h>


#define MAXARENASIZE 4096   /* Max arena size for this program */

#define SIZEIWANT 3000      /* Number of bytes I want for shared memory. */


main()
{

   usptr_t *arenaptr;   /* ptr to the arena */

   volatile char *buf;  /* ptr to the start of the shared bytes */

   long i;   /* loop temp */


   /* Define the size of each arena subsequently allocated. */
   if(usconfig(CONF_INITSIZE, MAXARENASIZE) == -1)
   {
      printf("usconfig: cannot define the size to allocate the arena!\n");
      exit(1);
   }

   /* Create an arena */
   arenaptr = usinit("myarena.file");

   if(arenaptr == (usptr_t *)NULL)
   {
      printf("usinit: cannot create the arena!\n");
      exit(1);
   }

   /* Allocate a piece of shared memory for communication.
      We wait here until we can allocate the memory. We are
      going to either be first, and get the memory right
      away or we are going to wait until the other process
      frees the memory back to the arena.
   */
   while((buf = usmalloc(SIZEIWANT, arenaptr)) == (char *)NULL)
   {
      /* We do nothing in this loop but wait... */

   }

   /* Now free the buffer for reallocation */
   /* We are going to keep using this ptr though!!! */
   usfree(buf, arenaptr);

   /* loop forever...*/
   for(i=0; i < 100000000; i=i+1)
   {

     /* whenever the first byte is zero, put the message into the buffer */
     while(*buf == '\0')
     {
         strcpy(buf, "Message from share2: Hello share1\n");
     }

   }   /* end for i */

   /* Leave the arena file, but here is where we could delete it */

}