Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!wuarchive!uwm.edu!bionet!raven.alaska.edu!hayes.ims.alaska.edu!floyd
From: floyd@ims.alaska.edu (Floyd Davidson)
Newsgroups: comp.sys.3b1
Subject: Re: swap space
Message-ID: <1991Jun14.122211.16417@ims.alaska.edu>
Date: 14 Jun 91 12:22:11 GMT
References: <1991Jun13.065207.10089@ucunix.san.uc.edu> <1991Jun13.122803.25362@ims.alaska.edu> <55913@rphroy.UUCP>
Organization: University of Alaska Institute of Marine Science
Lines: 101

In article <55913@rphroy.UUCP> tkacik@hobbes.cs.gmr.com (Tom Tkacik CS/50) writes:
>In article <1991Jun13.122803.25362@ims.alaska.edu>, floyd@ims.alaska.edu
>(Floyd Davidson) writes:
>|> 
>|> I don't know one way or the other on this.  Does it kill the new
>|> process or kill an old process?  I know if existing processes
>|> ask for more memory and swap space is full it will start killing off
>|> processes.  But I don't know what the algorithm is.
>
>When a process asks for more memory than there is virtual memory,
>(via malloc, or sbrk(2)), an error will be returned, and the process will not
>get any bigger.  It will not be killed by the kernel either, it will usually
>call exit by itself, or fail with a segmentation violation as it mindlessly
>tries to use the NULL pointer returned by malloc.
>(Of course, a proper program will try to clean up after itself and handle the
>lack of memory in a graceful fashion.:-)
>
>I believe that if there is still plenty of virtual memory but
>very little swap space left, if a program then calls malloc (or sbrk)
>the same error will be returned, and the program will not grow.  I do not think
>that a running program will be killed because there is not enough memory to
>grow into, it simply will not be allowed to grow anymore.
>
>If a new process tries to start running, and there is no more memory,
>the kernel will not let it run.  I do not think that it will start and then
>be killed by the kernel.  The kernel should never just start killing
>off processes when swap space runs out.  Individual processes will start to
>die off as they need more memory and cannot get it.


This is pretty easy to check out, so I did.  I took the memory
allocation check routine from hard-params.c in the gcc-1.40
distribution and made a little program that allocates all the
memory it can, tells how much it was, and sleeps for two minutes.

Then I ran it several times in the background.  Interesting.
The first three invocations were always repeatable:

At the start:
  -- Free memory: 2.368 Mb, Free swap 6.412 Mb

Allocate 2.481 Mb
  -- Free memory: 2.296 Mb, Free swap 3.828 Mb

Allocate 2.481 Mb
  -- Free memory: 2.236 Mb, Free swap 1.260 Mb

Allocate 1.021 Mb
  -- Free memory: 2.164 Mb, Free swap 0.228 Mb

At this point things work different every time.  Sometimes
it will allocate 20-40k of memory, sometimes it won't.

When it won't, a prompt that I assume comes from the kernel
but could be from the shell, says "Killed".

Any attempt at running ls results in "Killed".  However,
several smaller utilities like who and ps will run.

One one occasion I did get a message definitely from ksh
that said something like "ksh:  failed to fork()".  I'm not
sure exactly, but that was the essence of it.  The fork
failed get enough memory to start another process even.

Nothing else got killed, and there was nothing running that
would have tried mallocing more memory.  The free memory and
free swap space numbers came from sysinfo (updating every five
seconds).

It does appear that normally programs would in fact handle
a full swap device by whatever means they handle an out of
memory condition, because malloc definitely returns a NULL
when there is no swap space, even if there is sufficient
free memory.  But it also appears that the kernel will
kill a process attempting to start if it can't allocate swap
space for it.

Now as to whether the kernel has specifically a built in
per process limit, or whether it is just a case of running
out of virtual memory (at 4Mb max) due to the program size
and shared libs and the kernel adding up,  here is an easy
way to find out.  If a few people will run the hard-params
binary that comes with the gcc distribution and report 
exactly how much memory it says can be allocated, we can
tell.  That is the last thing hard-params reports, so just
let it run and read the last couple lines.

My kernel is a slightly modified 3.51m version (re-linked
with space for 10 mounted drives).  A standard 3.51m, 3.51a,
or 3.51 might show up slightly different if the limit is
merely how much virtual memory is available.  But it will
be the same if the kernel actually has a limit.  It might
also be the same if the actual difference in size between
various kernels is less than one page in memory too, though.
(Running size on my kernel shows 157,840 if thats worth
anything.)

Floyd
-- 
Floyd L. Davidson   | Alascom, Inc. pays me, |UA Fairbanks Institute of Marine
floyd@ims.alaska.edu| but not for opinions.  |Science suffers me as a guest.