Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!apple!motcsd!xdos!doug From: doug@xdos.UUCP (Doug Merritt) Newsgroups: comp.sys.amiga.tech Subject: (Fairly) Complete Amiga VM design (was Re: MEMF_PHYSICAL?) Summary: Rather than nitpick, I finally offer constructive design: Message-ID: <370@xdos.UUCP> Date: 2 Jun 89 16:40:47 GMT References: <8906010252.AA11810@jade.berkeley.edu> <367@xdos.UUCP> <7039@cbmvax.UUCP> Reply-To: doug@xdos.UUCP (Doug Merritt) Organization: Hunter Systems, Mountain View CA (Silicon Valley) Lines: 201 In article <7039@cbmvax.UUCP> jesup@cbmvax.UUCP (Randell Jesup) writes: > Single address space and protection are seperate issues. Another >thing to realize is that protection doesn't necessarily mean protection >from another process reading your memory. With one address space, you can >read lists and structures owned by different processes, which makes message >passing with pointers work much better. Agreed. I just realized that the problem is simply that I was speaking in general, whereas you and others are naturally thinking of the question "what *should* be implemented on the Amiga?" So let's talk about that specifically. Given the necessity of backward compatibility with existing Amiga software, *clearly* we need to have *some* shared pages between tasks (including with AmigaDOS) in order to have message passing. I never meant to disagree with that. But there's more... >> I wrote: >>in that particular piece of physical memory. This puts a severe restriction >>on virtual memory, since each time one of those tasks is run, the same >>physical ram must be freed again, which will often mean that whole [...] > > Wrong. The whole point of VM is that the addresses can be mapped. >A single address space merely means that all processes share a single >mapping of virtual->physical. Your hypothetical situation only arises >if processes can have different mappings. True. I was simply trying to point out that this issue means you can't have a *pure* single address space model, since if each task *does* have pages that have the same address, you either have to swap (if they're the same *physical* address, as required by the single address space model), or else you have to given them overlapping *virtual* pages, in which case you don't have a pure single address space model. Instead you have some pages in the same address space, and others outside of it. The fact that some pages at the same virtual address space have different contents inherently means that you don't have a simple flat single address space. Getting back to why we even care, yes, clearly you'd want to use virtual mapping rather than swapping into the same physical address. But this is no longer a pure single address space. > >>For real time programming, the performance advantage of shared address >>spaces makes them desirable, and hence the single address space model >>we have on the Amiga is a blessing. It is a mixed blessing nonetheless, >>since any malfunctioning program can bring down the whole machine. > > Protection (if implemented - various historical things make this >tough) would solve most of this, with a single address space. Yes and no. We can solve *some* of it. How much depends on particular details. Talking about simply "a single address space" with some pages protected would *not* solve "most of this"; more design is needed to get from "some" to "most". Thus my nitpicking. What, precisely, *is* needed? This: - Message passing pages must be in a universally shared portion of the address space because we don't know who will need to see the message and write to it. This implies that allocated memory must be in the global space. - Allocated memory should be tracked and freed. This can be accomplished to a large extent by initially giving each task private pages for allocation. When another task accesses one of those pages (in the global space but initially r/w only to the initial owner), AmigaDOS notes that page ownership is now shared by both programs and turns on r/w access for the second task. If that never happens (the normal case; consider big chunks of allocation like custom screens allocated by DPaint or games), then the memory is freed when the program exits. Already that's a big win. In addition, if *all* tasks with the capability to access an allocated page exit, again the page is freed. Another win. Finally, aging could be attempted, so that if the initial owner exits, and then secondary owners do no accesses at all over some period of time after the exit, it might be assumed that message passing had been done and the page is now stale and can be freed. The times when that's not what happened are troublesome, however, so either we use the patching/marking mechanism discussed below, or else forget it altogether. BTW the overhead of the above happens only once per page per task and hence should be acceptable. - Some programs use statically allocated messages. This implies that they must be forced to be in the global space. This is a pathological case, but could be dealt with via header information in the executable chunk. This info would have to be set by a sort of patch utility. The patch utility should be run automatically (after getting user permission from a requester) when the system detects a write fault. Thus the program would fail the first time it was run, but would get mapped into the global space on later runs. With proper requester phraseology, this would work even with novices. - Customization utilities that attempt to write to AmigaDOS data structures would trigger a similar mechanism that would set a permission bit on the executable file saying that it was privileged to write to these pages. AmigaDOS structures could be divided into two classes, those which are relatively safe to change, and those which might be part of virus action, so that the most dangerous types of modifications are limited to executables with very trusted privilege (i.e. two types of privileged permission bits). Ideally these utilities would even have detailed information in the new header to say *which* AmigaDOS pages it can write, (this utility modifies the device list, this one modifies memory allocation structures, etc) but it will be difficult to provide such fine grained information in all cases. The attempt is worthwhile, however. - task text pages are, by *default*, write protected and in a private address space. - text pages must in *some* cases be writable by, and/or in the same address space as, another task. This is necessary for things like load-and-go linkers and incrementally compiling interpreters (need writability), and for code sharing for e.g. shared libraries, multithreaded single text stream tasks, etc, which need same address space. - The two requirements above are mutually exclusive unless we have some means to distinguish the two. For compatability we cannot require programmatic changes (and recompilation), therefore it must again be an environmental distinction. We could in many cases assume that a new task should share writable text address space with its parent, assuming that if a parent and child want to destroy each other, it's none of our business (New York Mode :-) :-) :-) This is not good for CLIs, shells, and menu shells, so there should be a user-runable patch utility to mark these few executables as untouchable (their children get separate address spaces than the parents, once the child starts up). - What about demand paging? By default, CHIP memory should not be paged because hardware may access it without prior warning, and we can't put hardware on hold while we process an interrupt. Possible exceptions to this are e.g. screen memory which is not currently visible, and which has not been accessed by a task recently (perhaps the task is sleeping or busy elsewhere). We need to monitor when this changes, though, which means careful checking of the copper list. This significant overhead happens only on e.g. screen switching and so should not be noticeable. FAST memory is not subject to custom chip i/o, and therefore can be paged. Most reasonable forbid/permit accesses are to system memory, so that's not a problem. Others may surround message massaging, and it's ok to page these globally shared pages even inside forbid(). Then there are interrupt routines doing a forbid() for any reason (easily handled since all pages of an interrupt handler must be resident). Lastly are the pathological cases like s/w doing forbid for stupid reasons, in which case we can mark its executable again. - Finally, to get the miscellaneous cases not otherwise covered, the user needs to be able to mark executables as globally-shared and writable so that some other task can write them. In general all of these "patch" and "mark" actions should be usable by novices ("if the program fails with this message, take your chances and run this utility"), and also by experts (fine grained control over exactly what's going on). Given all the above to handle existing software, the discussion about introducing new AmigaDOS features (MEM_PHYSICAL/VIRTUAL/SHARED/PRIVATE/VOYEUR/ etc) then allows better control for newly written software. Semaphores become very important, too. >>Thus, despite the advantages of the Amiga approach, it also has some >>severe inherent design problems that lead to the machine crashing when >>some task goes bad. > > We cannot ever make it impossible, but we can cut way down on the >probability of it for MMU-based machines. There's nothing we can do for Agreed. My detailed proposal *still* won't prevent gurus in all cases. And I claim that some such detailed set of changes is necessary even to get *most* of the cases. > Virtual machines are an amusing idea, and can be useful for debugging >OS's (slowly) on fairly generic machines, but the killer is hardware access. Yes, but even slow virtual machines can be useful. Look at all the people willing to use MS-DOS software simulators, for instance. The hardware access issue is very real, but that's the way it goes...you get either danger or slowness (ideally a choice). Doug -- Doug Merritt {pyramid,apple}!xdos!doug Member, Crusaders for a Better Tomorrow Professional Wildeyed Visionary "Welcome to Mars; now go home!" (Seen on a bumper sticker off Phobos)