Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!ukma!xanth!lll-winken!vette!brooks
From: brooks@vette.llnl.gov (Eugene Brooks)
Newsgroups: comp.arch
Subject: Re: Multi-Processor Serializability
Keywords: data ordering, coherence, shared memory multiprocessing
Message-ID: <19875@lll-winken.LLNL.GOV>
Date: 4 Feb 89 18:57:54 GMT
References: <3492@cloud9.Stratus.COM> <19635@lll-winken.LLNL.GOV> <7650@polyslo.CalPoly.EDU> <1120@l.cc.purdue.edu>
Sender: usenet@lll-winken.LLNL.GOV
Reply-To: brooks@maddog.llnl.gov.UUCP (Eugene Brooks)
Organization: Lawrence Livermore National Laboratory
Lines: 21

In article <1120@l.cc.purdue.edu> cik@l.cc.purdue.edu (Herman Rubin) notes
that failure of the hardware to enforce the the ordering of side effects to
memory locations can be a problem.  We in fact ran into this problem with the
Cerberus multiprocessor simulator, accesses to shared memory are fully
pipelined through a packet switched network.  Because conflicts in the
network could cause the actual stores to the memory banks to occur out of
order the set of a flag which is guarding access to data being written could
beat the actual writes to the data.  In Cerberus, we dealt with this problem
by requiring return reciepts for all memory accesses.  A counter in each
processor kept track of the number of pending memory operations issued by
the processor, and the instruction "sync" stalled further instruction
issue until all pending memory operations were complete.  The user explicitly
used "sync" in his algorithm before a flag was set indicating written data
was ready.  By seperating the "sync" operation as a seperate instruction
one could fully pipeline the writes of an aggregate data object.  We actually
found that the use of "sync" was required in this sort of data transaction
for a customized Gauss elimination algorithm, without the "sync" the set of
a flag would indeed beat the writes of data it was protecting in rare cases
and caused incorrect execution of the algorthm.

brooks@maddog.llnl.gov or brooks@maddog.uucp