Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!cos!fetter
From: fetter@cos.com (Bob Fetter)
Newsgroups: comp.arch
Subject: Re: fork and preallocation (was Re: Paging page tables)
Message-ID: <33663@cos.com>
Date: 24 Jul 90 22:23:25 GMT
References: <58330@bbn.BBN.COM> <5928@titcce.cc.titech.ac.jp> <2845@awdprime.UUCP>
Reply-To: fetter@cos.UUCP (Bob Fetter)
Organization: Corporation for Open Systems, McLean, VA
Lines: 107

In article <2845@awdprime.UUCP> tif@doorstop.austin.ibm.com (Paul Chamberlain) writes:
>In article <5928@titcce.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
>>In article <58330@bbn.BBN.COM>
>>	lkaplan@BBN.COM (Larry Kaplan) writes:
>>>>[ deeply nested crud that we're all tired of reading]
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Not me.  I've found this discussion to be interesting, and it has
given me several reasons and occasions for me to stop and rethink what
I though I already knew--in the process (re-)learning some things.
Hey, its the real *fun* of this business, at least for me.

>
>Why not agree to disagree?
>
>Some people don't want their programs to run unless they are absolutely
>guaranteed that it will finish.  The rest of us like things to run as
>often as possible even when troubled with the occasional failure due to
>lack of resources.

  Ah, but from the system OS platform standpoint, an occasional
failure, esp. one due to lack of resources, is pretty important.  So
far as conservation of swapspace goes, it (if the system runs with a
fixed swapfile mechanism) is a finite resource, and should be
husbanded with appropriate care ('fixed', of course, at a given point
in time, not to say that swapfile space can't be grown/shrunk).
  From an application pgm standpoint, resource exhaustion/failure/restarting
can be a survivable sequence in many cases -- long-running, compute-bound,
memory-expensive, apps, of course, aren't good to let this happen to.

  If the OS platform dies/locks up end-user processes, due to resource
shortfalls or other reasons, not just one process suffers.  This
really isn't just a case of "not wanting a program to run unless
they're sure it WILL run", but wanting OTHER programs to run -- read:
user processes.  But, enough on this, I guess.

  As to vfork(), please correct me if I'm wrong, but I have the mental
model that vfork can be considered as a degenerate case of lightweight
processes (LWP - let me term this degererate case a "psuedo-LWP, or
pLWP).  I term it degenerate, in that

  1 - the main HWP is suspended until the exec*() of the pLWP, because
  2 - the stack is logically shared between between the two, and
  3 - that there can be only one pLWP paired to the main HWP at a time.

(One could say that the existance of the pLWP is bounded by the vfork()
call for its creation, and the exec() call terminating its existance
as a *pLWP*, transforming it into a full-blown HWP).

  I see and understand the argument that one doesn't want to create a
copy of the stack for this pLWP since, in the context of *this
particular case*, the end-goal is to exec() a new process with its own
resources, so why copy what one will toss away (using finite resources
in the process -- the infamous swapfile discussions to-date), etc.  To
effect this "savings", I can see why one then needs to suspend the
main HWP, so one doesn't get "dueling stack frames syndrome".  Having
one pLWP per HWP like this appears to be painful enough, hence the
limit 3 above.

  My understanding is that, using vfork(), all segments known to the
pLWP are exactly those known to the main HWP and there is no COW being
done.  As such, the pLWP is operationally the same as a "regular" LWP
in this regard, except, of course, that the pLWP and the main HWP
share the same stack segment ("regular" LWP have their own stacks).
It seems to me that it is this sharing of the stack segment which is
the issue (to use the same with its attendant risks, or perform COW
a-la fork() with ITS problems such as swapfile allocation, etc), along
with the "hidden" semantics of vfork() creating a pLWP when fork()
doesn't and the confusion this difference may cause.

  Standing back a bit (hopefully not too far back), I get the
impression that the overall issue just might be thought of as being
the conversion of a LWP to a HWP.  Now, it's not exactly that as of
this point in time as regards the vfork() call, in that "regular" LWP
have their own stack, whereas a pLWP is being a parasite on its main
HWP.  But...

  If/when actual direct kernal support for LWP is available, why not
just then treat vfork() as the creation of a LWP, suspending the main
HWP as done now and mapping all segment descriptors/references from
the LWP back to the HWP (including the stack) and, when the exec()
call occurs,
  1 - remap all segment descriptors for the LWP to new, appropriately
      sized segments for the exec()-ed process,
  2 - remove the suspension on the main HWP, putting in the appropriate
      return value from the vfork() call.

  This puts the LWP/task management into the system dispatcher/context
switcher, removing the existing runtime inter-process
implementation/multiplexing technique.

  I don't see (additional) usage of swapfile space in the above, which
appears to be one of the major points under discussion, during the
time between vfork() and exec() in the 'child'.  It appears to also
only allocate memory (and associated swapfile resource) as is required
by the new HWP, not carrying over the 'parent' HWP load for a transient
time only to release it effectively unused.

  After all, can't one consider that a LWP just a HWP which has one or
more of its segments bound to/shared with another HWP?  Or, am I
missing something here?  (Issues like files, signals, etc., would of
course percolate as done today.)  I seem to remember an article in
this chain (or a related one) which discussed segment management
system calls which could be implemented.  Am I on a similar train of
thought?

  -Bob-