Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!rutgers!bellcore!faline!ulysses!sfmag!sfsup!shap
From: shap@sfsup.UUCP (J.S.Shapiro)
Newsgroups: comp.arch,comp.unix.wizards,comp.os.misc
Subject: Re: Big Programs Hurt Performance
Message-ID: <2067@sfsup.UUCP>
Date: Mon, 21-Sep-87 09:25:00 EDT
Article-I.D.: sfsup.2067
Posted: Mon Sep 21 09:25:00 1987
Date-Received: Wed, 23-Sep-87 01:21:38 EDT
References: <1665@ncr-sd.SanDiego.NCR.COM) <8579@utzoo.UUCP) <6886@eddie.MIT.EDU) <2501@xanth.UUCP>
Organization: AT&T-IS, Summit N.J. USA
Lines: 77
Summary: Brief explanation of shared libraries
Xref: mnetor comp.arch:2283 comp.unix.wizards:4390 comp.os.misc:206

In article <2501@xanth.UUCP>, kyle@xanth.UUCP (Kyle Jones) writes:
> In <14888@topaz.rutgers.edu>, hedrick@topaz.rutgers.edu (Charles Hedrick) sez:
> 
> > What you really want is shared libraries.  That way, only one copy
> > of the code is shared by all programs that use it, but you can
> > change it.
> 
> Please explain more about shared libraries.

Okay, here goes. I have stayed out of this, but shared libraries I can talk
about intelligently. Basically a shared library is a piece of code which
is "shared" between two programs. A portion of the address space is
reserved in advance by *everyone* for each shared library (that is, the
shared library has a permanent reserved location in the virtual address
space). Then, whoever needs the functionality in the shared library simply
compiles as usual, linking in the shared version of the library instead of
the normal version. As a result, a marker is put in the binary indicating
which (if any) shared libraries need to be hauled in. If the marker is
there, exec() arranges for the shared library to get mapped into your
address space.

This is preferable to the NVRAM scheme because if you *don't* use the
feature you don't have to pay for RAM for it.

In order to provide simple upgrades, shared libraries often work through
a jump table (even though they don't strictly need to). The jump table size
is fixed, and usually larger than the number of externally visible
functions to allow for backwards compatible expansion. A consequence of the
jump table is that if you need to fix a broken function, you can just fix
it. So long as the jump table doesn't move the library is compatible with
the old one *without* any recompilation. In some implementations (depends
on your hardware), the jump table points to a stub routine which
backpatches the "real" address of your function into your code. This has
the advantage that you only incur the shared library overhead once per
function, but the disadvantage that you can no longer page in those pages
from the executable - they now have to go to the paging area.

Only one copy of the shared library text is kept in core for all users. It
is simply mapped into all of the appropriate virtual address spaces. On
System V, most of the system applications are compiled against it.
Read-only data can be shared too, but read-write data pages need to be
marked copy-on-write, or if your memory manager doesn't support that, need
to be duplicated. This is okay, because the data section is usually small.
The static shared library uses the process heap.

It is worth noting that a better scheme, though much more difficult, is to
generate "position independent code" so that you do not have to reserve the
address of the library. You can then use exactly the same tricks, but
generate your function calls via a jump table in the executable's data
space. When you 'attach' the shared library, you copy the table out of the
library into your own, adjusting the entries so that the function addresses
are correct. This scheme has the advantage that you don't have to reserve
large chunks of your address space. Unfortunately, position independent
code is quite difficult to do, which is why current UNIX compilers (to my
knowledge) don't do it. This scheme is referred to as "dynamic loading."

I am hardly an expert on any of this, and I may have gotten the details
wrong. I do know that shared libraries are provided in System V, though I
don't know which CCS release they started in. AT&T compilers don't generate
position independent code. 

Side observation: If your binary is 500K, shared libraries don't help much.
They just don't represent a significant portion of your code. If your
binary is really that big, you probably have a lot of rethinking to do, and
ultimately this rethought will be reflected in better performance, greater
flexibility, and lower maintainance cost. Depends, of course, on your
application, but try running size(1) on /unix (or /vmunix) and take a look
at the text size.

*** Disclaimer ***

The above is a personal exegesis, and should not be taken as representing
the views of AT&T.


Jonathan Shapiro
AT&T Information Systems