Path: utzoo!utgpu!news-server.csri.toronto.edu!bonnie.concordia.ca!uunet!auspex!guy
From: guy@auspex.auspex.com (Guy Harris)
Newsgroups: comp.unix.programmer
Subject: Re: shared libraries, when to use them
Keywords: shared libraries .so
Message-ID: <8631@auspex.auspex.com>
Date: 30 Jun 91 01:09:06 GMT
References: <1991Jun11.163544.20234@aio.jsc.nasa.gov> <8448@auspex.auspex.com> <1991Jun22.163706.1174@thunder.mcrcim.mcgill.edu>
Organization: Auspex Systems, Santa Clara
Lines: 98

>The main point is that the semantics are different.  A "library", in
>the past, is something from which "modules" are extracted and included
>in the program as necessary.  No more than necessary is included from
>the library.  But the thing so often termed a "shared library" has
>different semantics.  While for the most part the difference is not
>visible, it is in one case: when a routine in the user program has the
>same name as a routine in the library.  This is generally bad
>programming practice, when done deliberately - but when linking with a
>"shared library", there's often no notification that there's anything
>wrong; the program just does mysterious things, and worse, the symptoms
>go away or change when linked -Bstatic.

What are some cases where problems of that sort showed up?

The SunOS dynamic linking mechanism is intended to have that work the
same way with dynamically-linked libraries as with statically-linked
libraries (for better or for worse). 

I.e., if your program has its own memory-heap manager with "malloc()",
etc. entry points, when linked dynamically, routines from "libc.so.M.N"
that call "malloc()" should end up calling the ones in your program.

>(There are other differences; for example, the SunOS core-dump routine
>has a bug in that the core files it generates don't include anything
>from any dynamically linked shared objects.)

That's not a difference between "shared libraries" and "shared objects";
if any data or BSS in a "shared library" implementation wasn't in the
"data segment" part of the address space, it would have the same
problems as data and BSS in the current "shared object" implementation.

The problem is that the core-dump code only writes out the core dump
header, data segment, stack segment, and U area ("for those who care",
to quote the comment in the code; the idea is that most of the stuff you
want from the U area can be found in the core dump header instead, and
in a fashion less likely to change from release to release, as it's not
just a raw dump of the U area).

It manages to get some of the dynamic linker's data structures because
they're stuck in the stack segment, near the stack limit to keep them
out of the way of a growing stack.  The stack segment is sparsely
mapped, and the core dump code turns that part of the core dump into a
sparse file; that's why core dumps from dynamically-linked programs are
such large files - they have a large hole in them corresponding to the
hole in the stack segment.

(This assumes, of course, that the core dump wasn't from some
"ackerman(100, 100)" or something such as that. :-))

The proper fix is to beef up the core dump format a little more, to
handle sparse address spaces better, e.g. with a map between file
offsets and address-space offsets in the header.  This might also be
useful for getting rid of the sparse-file hack mentioned above.

The astute reader might note that some object-file formats have maps
such as those; SVR4's ELF is one such format, and ELF executables in
SVR4 should dump core files in ELF format, which should include more
than just data+stack.  (Dunno what exactly gets dumped; it's been a
while since I've seen that code.)  (And yes, at least in the code I saw,
COFF executables dumped core files in an SVR3ish core file format. 
Dunno why; maybe they wanted old debuggers to continue to work on old
binaries.)

>SunOS could be made to do shared libraries instead of shared objects,
>without all that much work.  All that would be necessary would be to
>have ld.so map the .a file instead and do the relevant relocations,
>which it has to be prepared to do anyway.

Yup, it's just a Simple Matter of Programming.  :-)

>Differences I see: (1) this would almost surely be slower than the
>current scheme;

Yup - more relocation to do, and because of 2), more copy-on-write
faults.

>(2) there's less memory sharing at run-time, because more of the
>mmap()ed memory would have to be modified by relocation;

You might end up with *no* sharing at run time - you get dynamic
linking, which is nice, but you don't end up sharing the code, which may
or may not make a difference.

>(3) only one library to maintain instead of two,

I'm told the SVR4 version of dynamic linking doesn't involve ".sa"
files, if that's what you're referring to; it's a SunOS 4.x-derived
"shared object" scheme, though.

>and the semantics are the same.

Which semantics?

>(In passing: anyone know why Sun doesn't document the format of the
>ld-generated data structures?

Documentation is always the stuff done last, and often falls through the
cracks....