Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!uwvax!dogie.macc.wisc.edu!csd4.milw.wisc.edu!lll-winken!ames!amdahl!rtech!hoptoad!tim
From: tim@hoptoad.uucp (Tim Maroney)
Newsgroups: comp.sys.mac.programmer
Subject: Re: System 7.0 Q & A (System heap protection)
Message-ID: <7387@hoptoad.uucp>
Date: 21 May 89 00:25:08 GMT
References: <30353@apple.Apple.COM> <4666@okstate.UUCP> <1787@internal.Apple.COM> <7266@hoptoad.uucp> <13472@dartvax.Dartmouth.EDU> <7321@hoptoad.uucp> <30935@apple.Apple.COM> <811@key.COM> <25141@coherent.com>
Reply-To: tim@hoptoad.UUCP (Tim Maroney)
Organization: Eclectic Software, San Francisco
Lines: 92

In article <25141@coherent.com> dplatt@coherent.com (Dave Platt) writes:

[Accurate and thoughtful comments on low memory globals deleted.]

>Consider what happens if your application loads a resource from
>the System file (say, a FONT), and the memory manager is forced to
>compact the system heap in order to free up a large-enough contiguous
>block.  CRUNCH!  The compaction may move, purge, or coalesce blocks all
>over the System heap in one swell foop.  All of a sudden, the entire
>heap is potentially dirty!

No, that's not the application writing to the system heap.  That's the
OS writing to the system heap.  OS code is always going to have the
theoretical potential to screw up this kind of thing; it's Apple's
responsibility to try to make the code so clean that it almost never
does.  And usually, they do pretty well at this.  I understand much
more of the OS is being written in high-level languages these days,
which is a further aid to reliability.

Routines like GetResource in the OS ought to run in supervisor mode and
have read-write system heap access.  It's applications that should have
read-only access to the system heap, running in user mode.  The
situation you describe only dirties the heap as much as the OS can do
(roughly epsilon), not as much as the application software can do
(roughly aleph-null).  And it works perfectly well with the application
in read-only mode.

>To make matters a bit worse, you can't simply protect the System heap
>against modification by application code running in user-state.  You
>must also protect it against modification by code in the OS itself
>that's performing tasks on behalf of the user!  For instance, there's
>really no difference between the user storing blindly into the middle of
>the heap, and the user calling FSRead to read several hundred bytes into
>a buffer that's allocated in the System heap... either of these events
>can trash the universe.

That's true.  Fortunately, there is a long tradition of kernel code
checking its buffers and sizes in memory protected environments.  In
UNIX implementations, this is often done using a special (and fairly
slow) block move routine that copies from kernel to user space.  It
checks each address written to make sure the user process it's working
on behalf of is allowed to write into that address.

>In order to prevent kernel-trashing-the-heap conditions, you must either
>add code to _every_ OS & toolbox routing that manipulates user-specified
>memory (checking the specified pointers for safety), or protect the
>System heap itself against modification _by_the_OS_, and then (when a
>store-violation occurs) decide whether the specified change to the heap
>is "valid".  Both of these approaches are expensive in time and/or in
>programming effort, and they're tricky to get right.

I don't think there are all that many places where user buffers need to
be checked in the Mac OS.  I think the former approach is clearly the
way to go; the latter might work, but it would be a mess.  Context
sensitive special page fault handlers -- yecch.

However, a compromise solution might work.  Memory permissions could be
raised for some system calls but not for others.  For instance,
BlockMove can only correctly be used to copy from the caller's space to
the caller's space, so it can run with its caller's protection.  Most
status calls that return something into a pointer variable may need to
read system space, but won't need to be able to write it, so they can
be set up without any expanded write protections.

There is complexity, but the problem is soluble given Apple's huge
resources in Mac programming expertise.  But they have to decide to do
it first!

>I suspect that you'd rapidly reach the point at which every application
>would have a completely separate copy of the System heap.  This isn't
>too terrible a problem if you have virtual memory and a fast enough
>disk, but it does eliminate one of the major advantages of the
>MultiFinder approach (only one copy of any particular System resource is
>in physical memory at any one time).

I don't see why there would need to be multiple system heaps.

One more note:  According to a reliable source, A/UX is only sort of
32-bit clean.  It changes into 24-bit mode before calling the ROMs,
which, claims of various people aside, are *not* now 32-bit clean.  The
source is an experienced A/UX hacker who has disassembled some of its
internals, and who will remain nameless for that reason.

This fact about the non-32-bit-clean ROMs is acknowledged twice in the
Developer Conference Q&A, albeit obliquely, and confirmed by sources
within Apple.  I deeply resent being flamed for pointing out the
truth, especially the documented truth.  Re-read the questions and
answers on 32-bit mode before flaming again.
-- 
Tim Maroney, Consultant, Eclectic Software, sun!hoptoad!tim
"Prisons are built with stones of Law, Brothels with bricks of Religion."
    - Blake, "The Marriage of Heaven and Hell"