Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!umich!yale!mintaka!mit-eddie!uw-beaver!milton!blake!Tomobiki-Cho!mrc From: mrc@Tomobiki-Cho.CAC.Washington.EDU (Mark Crispin) Newsgroups: comp.arch Subject: Re: Multics & Memory mapped files Message-ID: <5684@blake.acs.washington.edu> Date: 10 Feb 90 01:07:01 GMT References: <8859@portia.Stanford.EDU> <20571@watdragon.waterloo.edu> <49956@sgi.sgi.com> <4791@helios.ee.lbl.gov> <2093@crdos1.crd.ge.COM> <1990Feb7.221800.804@utzoo.uucp> <2106@crdos1.crd.ge.COM> <5180@crdgw1.crd.ge.com> Sender: news@blake.acs.washington.edu Organization: Mendou Zaibatsu, Tomobiki-Cho, Butsumetsu-Shi Lines: 307 In article <5180@crdgw1.crd.ge.com> hammondr@sunroof.crd.ge.com (Richard A Hammond) writes: >Everybody seems to be missing the crucial fact about memory mapped files! > >They ONLY work for cases where the file size is < virtual address space!!! > >They are not a a generally useful solution which works over all possible >implementations. UNIX file I/O does work over all situations! As has already been pointed out, the case of a file that is larger than the virtual address space is relatively infrequent on all but very small machines such as PDP-11's. I don't think anyone considers a PDP-11 to be a modern platform. Furthermore, I hope that no one would implement file/memory mapping that only mapped the entire file in. You should be able to map n "pages" or "segments" from "page"/"segment" i of source to "page"/"segment" j of destination. It may be very common for n = size of file, i = 0, and j = free chunk of memory big enough to hold the file. However, this is not the only form of file/memory mapping you may want to do. I suppose it is also known that mapping in itself does not do any I/O unless you also specify "preloading". The overhead of mapping 1 page should be more or less the same as mapping 100,000. I'll acknowledge this requires some knowledge of page or segment sizes, but that can be done in a .h file so you can write portable code. A good way (which is what TOPS-20 used) is to have the filesystem allocation unit be the same as the memory allocation unit. As long as your code is careful to calculate the number of units based on the page size instead of assuming that it is 512 or 1024 or whatever, it's portable. For historical reference, here's a description of TOPS-20's memory mapping system call. Note that you could use either a file or a process for source or destination, as well as have a bi-directional file/memory map. No file/file mapping though. :-) :-) Disc I/O under TOPS-20 was simply byte copies to an operating system buffer that was PMAPed. Direct PMAP I/O was much faster than disc I/O! PMAP JSYS 56 Maps one or more complete pages from a file to a process (for input), from a process to a file (for output), or from one process to another process. Also unmaps pages from a process and deletes pages from a file. Each of the five uses of PMAP is described below. Case I: Mapping File Pages to a Process This use of the PMAP call does not actually transfer any data; it simply changes the contents of the process' page map. When changes are made to the page in the process, the changes will also be reflected in the page in the file, if write access has been specified for the file. ACCEPTS IN AC1: JFN of the file in the left half, and the page number in the file in the right half. This AC contains the source. AC2: process handle in the left half, and the page number in the process in the right half. This AC contains the destination. AC3: B0(PM%CNT) A count is in the right half of AC3. This count specifies the number of sequential pages to be mapped. If this bit is not set, one page is mapped. B2(PM%RD) Permit read access to the page. B3(PM%WR) Permit write access to the page. B4(PM%EX) Reserved for future use. The symbol PM%RWX can be used to set B2-4. B5(PM%PLD) Preload the page being mapped (move the page immediately instead of waiting until it is referenced). B9(PM%CPY) Create a private copy of the page when it is written into (copy-on-write). If the page is mapped between two processes (Case III below), both processes will receive a private copy of the page. | B10(PM%EPN) The right half of AC2 contains an | extended process page number. If the | section containing the page does not | exist, an illegal instruction trap is | generated. B11(PM%ABT) Unmap a page and throw its changed contents away. This bit is significant only when unmapping process pages that were mapped from a file (see case IV below) and OF%DUD is set in the OPENF. Normally, if a page is unmapped and has been changed since the last time the monitor updated the associated file page, the monitor will remove the page from the process and place it on a queue in order to update the file page. PM%ABT allows the page to be unmapped, but prevents the monitor from placing the page on the update queue. This feature is useful in the case of erroneous data written to a mapped page of a file open for simultaneous access. In this case, it is important that the erroneous page be discarded and not be used to update the file page. Another application is to allow processes in separate jobs to communicate by sharing a file page (and reading/writing the page) and avoid the overhead of the monitor periodically updating the page. B18-35 Number of pages to be mapped if (PM%RPT) B0(PM%CNT) is set. RETURNS +1: always This use of PMAP changes the map of the process such that addresses in the process page specified by the right half of AC2 actually refer to the file page specified by the right half of AC1. The present contents of the process page are removed. If the page in the file is currently nonexistent, it will be created when it is written (when the corresponding page in the process is written). If the process page is in a nonexistant section, an illegal instruction trap is generated. This use of PMAP is legal only if the file is opened for at least read access. The access bits specified in the PMAP call are ANDed with the access that was specified when the file was opened. However, copy-on-write is always granted, regardless of the file's access. The access granted is placed in the process' map. The file cannot be closed while any of its pages are mapped into any process. Thus, before the file is closed, pages must be unmapped from each process by a PMAP call with -1 in AC1 (see below). Case II Mapping Process Pages to a File This use of the PMAP call actually transfers data by moving the contents of the specified page in the process to the specified page in the file. The process' map for that page becomes empty. ACCEPTS IN AC1: process handle in the left half, and the page number within the process in the right half. This AC contains the source. AC2: JFN of the file in the left half, and the page number within the file in the right half. This AC contains the destination. AC3: access bits and repetition count. (Refer to Case I.) RETURNS +1: always The process page and the file page must be private pages. The ownership of the process page is transferred to the file page. The present contents of the page in the file is deleted. The access granted to the file page is determined by ANDing the access specified in the PMAP call with the access specified when the file was opened. This function does not update the file's byte size or the end-of-file pointer in the file's FDB. Failure to update these items in the FDB can prevent the reading of the file by sequential I/O calls such as BIN and BOUT. To update the file's FDB after using this PMAP function, do the following: 1. Use the CLOSF call with the CO%NRJ bit set to close the file but keep the JFN. 2. Use the CHFDB call to update the end-of-file pointer and, if necessary, the byte size in the file's FDB. 3. Use the RLJFN call to release the JFN. (Refer to Section 2.2.8 for the format of the FDB fields.) Case III Mapping One Process' Pages to Another Process This use of the PMAP call normally does not transfer any data; it simply changes the contents of the page maps of the processes. When changes are made to the page in one process, the changes will also be reflected in the corresponding page in the other process. ACCEPTS IN AC1: process handle in the left half, and the page number in the process in the right half. This AC contains the source. AC2: a second process handle in the left half, and page number in that process in the right half. This AC contains the destination. AC3: access bits and repetition count. (Refer to Case I.) RETURNS +1: always This use of PMAP changes the map of the destination process such that addresses in the page specified by the right half of AC2 actually refer to the page in the source process specified by the right half of AC1. The present contents of the destination page are deleted. The access granted to the destination page is determined by the access specified in the PMAP call. If the destination page is in a nonexistant section, the monitor generates an illegal instruction trap. Case IV Unmapping Pages In a Process As stated previously, a file cannot be closed if any of its pages are mapped in any process. ACCEPTS IN AC1: -1 AC2: process handle in the left half, and page number within the process in the right half AC3: B0(PM%CNT) Repeat count. Only the process page numbers are incremented. B18-35 Number of pages to remove from process This format of the PMAP call removes the pages indicated in AC2 from the process. A page that was locked with the PLOCK call may be unmapped. Doing so will unlock the process' page and return the now unlocked physical page to its previous state. Case V Deleting One or More Pages from a File Deletes one or more pages from a file on disk and does not affect the address space of any process. ACCEPTS IN AC1: -1 AC2: JFN of the file in the left half and page number within the file in the right half. AC3n: B0(PM%CNT) Indicates that the right half contains the number of pages to delete. B18-35 Number of pages to delete from file Illegal PMAP calls The PMAP call is illegal if: 1. Both AC1 and AC2 designate files. 2. Both AC1 and AC2 are 0. 3. The PMAP call designates a file with write-only access. 4. The PMAP call designates a file with append-only access. 5. The source and/or the destination designates an execute-only process and the process is not self (.FHSLF). Can cause several software interrupts on certain file conditions. Generates an illegal instruction interrupt on error conditions below. PMAP ERROR MNEMONICS: ARGX06: Invalid page number CFRKX3: Insufficient system resources DESX1: Invalid source/destination designator DESX3: JFN is not assigned DESX5: File is not open DESX7: Illegal use of parse-only JFN or output wildcard-designators FRKHX1: Invalid process handle FRKHX2: Illegal to manipulate a superior process FRKHX3: Invalid use of multiple process handle FRKHX7: Process page cannot exceed 777 FRKHX8: Illegal to manipulate an execute-only process IOX11: Quota exceeded IOX34: Disk full IOX35: Unable to allocate disk - structure damaged LNGFX1: Page table does not exist and file not open for write PMAPX1: Invalid access requested PMAPX2: Invalid use of PMAP PMAPX3: Illegal to move shared page into file PMAPX4: Illegal to move file page into process PMAPX5: Illegal to move special page into file PMAPX6: Disk quota exceeded PMAPX7: Illegal to map file on dismounted structure PMAPX8: Indirect page map loop detected _____ ____ ---+--- /-\ Mark Crispin Atheist & Proud _|_|_ _|_ || ___|__ / / 6158 Lariat Loop NE R90/6 pilot |_|_|_| /|\-++- |=====| / / Bainbridge Island, WA "Gaijin! Gaijin!" --|-- | |||| |_____| / \ USA 98110-2098 "Gaijin ha doko ka?" /|\ | |/\| _______ / \ +1 (206) 842-2385 "Niichan ha gaijin." / | \ | |__| / \ / \ mrc@CAC.Washington.EDU "Chigau. Gaijin ja nai. kisha no kisha ga kisha de kisha-shita Omae ha gaijin darou." sumomo mo momo, momo mo momo, momo ni mo iroiro aru "Iie, boku ha nihonjin." uraniwa ni wa niwa, niwa ni wa niwa niwatori ga iru "Souka. Yappari gaijin!"