Path: utzoo!attcan!uunet!wuarchive!mailrus!umich!caen!wailea.engin.umich.edu!pha From: pha@caen.engin.umich.edu (Paul H. Anderson) Newsgroups: comp.arch Subject: Re: Extremely Fast Filesystems Message-ID: <1990Aug7.205747.14206@caen.engin.umich.edu> Date: 7 Aug 90 20:57:47 GMT References: <13667@cbmvax.commodore.com> <1990Aug7.190719.7907@caen.engin.umich.edu> Sender: news@caen.engin.umich.edu (CAEN Netnews) Organization: University of Michigan Engineering, Ann Arbor Lines: 46 In article khb@chiba.Eng.Sun.COM (Keith Bierman - SPD Advanced Languages) writes: > >Humm. In estimation problems there are lots of ways to skin cats. >Algorithms which have huge datasets, but "small" models do not require >huge "core" storage. > >In the satallite tracking biz, some experiements (like GPS baselines) >go on for years, and Tb of data could be necessary if one formed the >obvious > T > A A > >and proceeded to use elimination from there. > This is very true, but researchers benefit enourmously from interactive computation, where the type of one computation may depend on the outcome of the preceding ones. Ideally, students in classes would be able to investigate problems interacitely in a matter of minutes what it currently takes researchers many months to accomplish. The thing that prevents this from taking place currently is that the datasets are on magtapes, and therefore any computation using the entire set of data is forced to sequentially access a very slow medium. The current technique is for a researcher to identify the smallest possible subset that has all the information they think they need. This is coded up in a program that is run on a 3090 with lots of big fast expensive tape drives. Eventually, the researcher gets the information they asked for, and if they are lucky, it is actually what they need. The problem is that the process of exploration doesn't match this kind of turnaround at all, so as a result, highly competent social scientists sit around on their collective backsides, waiting for data to show up on their desks. Computers that can address this problem are needed now, independently of whether or not they actually exist. We don't have a real problem optimizing use of the hardware we have now, it is just that the available hardware has too little RAM, or has filesystems that are too slow. Paul Anderson University of Michigan