Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!wuarchive!uunet!super!rminnich
From: rminnich@super.ORG (Ronald G Minnich)
Newsgroups: comp.arch
Subject: Re: Extremely Large Filesystems [was: Re: Extremely Fast Filesystems]
Message-ID: <31071@super.ORG>
Date: 9 Aug 90 13:40:27 GMT
References: <5539@darkstar.ucsc.edu> <13285@yunexus.YorkU.CA> <30728@super.ORG> <13667@cbmvax.commodore.com> <30979@super.ORG> <1990Aug8.195229.23544@jarvis.csri.toronto.edu>
Sender: news@super.ORG
Reply-To: rminnich@super.UUCP (Ronald G Minnich)
Organization: Supercomputing Research Center, Bowie, Md.
Lines: 36

In article <1990Aug8.195229.23544@jarvis.csri.toronto.edu> jonah@dgp.toronto.edu (Jeff Lee) writes:
>1) hire a station-wagon full of mag-tapes (or send a DAT by over-night
>courier) [6MB would saturate a T1 line (1.5Mbit/sec) for 9.1 hours.]
T1 is now slow.
6 GB would saturate an NREN for about 50 seconds. If you think I am saying
that mapping the 6GB NCAR file in is equivalent to copying the whole thing
over when your program starts up, I am not. That is just another form 
of FTP. 
>2) split between the data and program (e.g. with distributed shared memory)
Well, I am partial to that solution, not the least because hardware
implementations of such things have been done at least once, and I can't 
see using read and write calls to drive an NREN. 
>4) plan a quick holiday in Colorado (with Plan 9, your environemnt
>follows you automatically)
How many megabytes have to follow me? Sounds like the same problem to me.

>Only if you are planning to
>randomly access a *small* portion of this database does mapping all 6GB
>into your address space make sense.  
That depends to some extent on what you think your network is. 
For T1 i would agree with you. On a hypothetical 1Gb network I am no longer 
so sure. 
>I will agree though that most present operating systems will choke on
>the idea of a single 6GB random-access file -- or a 6GB virtual memory
>image.
More important, few architectures can accomodate it well. For instance, 
even 8Kb pages are silly in this case. But they are too big for most 
other cases. Does the small page/large page split deserve another look? 
I have seen one VM architecture recently in which the idea of 
paging is abandoned completely because it makes no sense in a large
memory environment. Note that VM was NOT abandoned on this machine.
ron
-- 
1987: We set standards, not Them. Your standard windowing system is NeUWS.
1989: We set standards, not Them. You can have X, but the UI is OpenLock.
1990: Why are you buying all those workstations from Them running Motif?