Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!zaphod.mps.ohio-state.edu!wuarchive!emory!hubcap!bcsaic!carroll From: bcsaic!carroll@cs.washington.edu (Jeff Carroll) Newsgroups: comp.parallel Subject: Re: Critical Issues in Distributed-Memory Multicomputing. Message-ID: <12401@hubcap.clemson.edu> Date: 27 Dec 90 14:33:15 GMT References: <12235@hubcap.clemson.edu> Sender: fpst@hubcap.clemson.edu Organization: Boeing Computer Services AI Center, Seattle Lines: 171 Approved: parallel@hubcap.clemson.edu In article <12235@hubcap.clemson.edu> eugene@wilbur.nas.nasa.gov (Eugene N. Miya) writes: >In article <12051@hubcap.clemson.edu> gdburns@osc.edu writes: >I think you have to ask if the usual engineer uses non-DMMCs with ease. >I assert not. Read back in the literature using "automatic programming" >as a keyword. It has gotten easier, but we have only learned with time >and experience. We have a deceptive situation: some programming >appears very easy learn, the basics: syntax of programming languages for >simple programs, etc. In my experience we now have two "generations" of practicing engineers - those who have learned how to use the computer as an engineering tool, and those who have not. (I hesitate to use the term "generation" because some belonging to the former group have been practicing since 1960, and some of the latter graduated from college in 1985.) The first group is quite adept in sitting down at the computer and coding a calculation that needs to be performed, in the personal language of choice (be it FORTRAN, BASIC, Pascal, C, or Forth.) It's certainly simpler than programming an SR-52 or an HP-41. We have some of these people who have learned to use hypercubes competently with relatively little trouble. Most of them have developed enough understanding of the computer in general that they can understand what goes on in a DMMP machine. The second group are the ones that are hard pressed to generate spaghetti FORTRAN that gets the right answer. They are completely hopeless when it comes to programming DMMPs. >>I am willing to take a shot at this. >>H/W is in reasonable shape. >A nice ATTEMPT at a state-of-the-art survey. >I don't quite agree. We have perfected packaging, but we have not >perfected communication or storage. We have a fetish for CPU speed >while ignoring balance. Amdahl deserves a lot of credit for his little >3 page NCC paper (1967). I think that it must be admitted that HW is at least way ahead of SW. The economics of the situation dictate that unless there are overwhelming arguments against it, we must use silicon which is available off-the-shelf as our building blocks, and adhere to industry standards wherever possible. The DMMP manufacturers have done a good job of using standards when it comes to storage and IO. What is needed now is an interprocessor interface standard - the transputer link is nearly perfect except that it is *way too slow*. It has all the other goodies - it's simple, it's cheap, and it's public domain. >That you ignored difficulties with programming languages tells me we have >a long way to go. A reviewer can tell the quality of a survey paper >with the %text on hardware versus the %software. This also goes for >market literature, personnel in a start up, and a few other artifacts. > >>from the vendors, in my opinion, because the customer tends to look at >>either the glamour of the h/w or the bottom line application. In the MIMD world, nobody with SOTA hardware has decent software. Thus none of the vendors face market pressure to improve their product. >>OS doesn't >>get enough attention from academia because, IMHO, the work that needs >>to be done is grunt level - essentially big D and very little r. >The OS people are contrainted by the limitations of hardware. End users >don't care as much about the OS. That indifference creates some subtle >problems (Elxsi, Denelcor, and a few other defunct companies can >tell you about this.) I had an interesting lunch discussion about >the concept of a "load average" on a multiprocessor. There are not enough systems programmers to go around in the industry at large, and even fewer (I guess) who can deal with MIMD boxes. Nonetheless the vendors push the hardware because that's ultimately where the performance is (you can always machine-code a sexy demo to take around to trade shows. This is not unique to parallel systems. I understand that that's what IBM did with the RS6000.) >>Algorithms are going well, I think, because of the number of people >>doing work. Getting the work productized and distributed is not going well. >>I have not done a survey but I'm sure you could find a >>workable algorithm for every solver somewhere in the literature. >I do not think algorithms are going well. I have done some survey. There are lots and lots of books containing papers about and algorithms for machines that don`t exist any more. What is lacking (as far as I know) is a broadly accepted taxonomy upon which parametrizable algorithms can be developed. Unfortunately there are only a handful of people around with the mental equipment to do this sort of thing (I certainly don't claim to be one of them). >>Education is the kicker. You have to learn how to program distributed >>memory. There are no true magic pills. >See you are re-inventing programming again. Hence the automatic programming >comment. This must have been what programming was like in 1946-1950. >Now, we replace the LOADs and STOREs with SENDs and RECEIVEs... >If the users are unwilling to explore new programming languages, >some models of parallel computation will be in trouble. Perfect >automatic parallelization only exists in the mind of the media, hopeful >users, and very few computer scientists... >We are so closed minded to other earlier approachs like LISP, APL, >and languages like VAL, SISAL, can't even get a chance for a reasonable >evaluation. I often wonder what happened to the flowchart. We are just now developing the technology (GUIs, OOP) which will enable us to make serious use of the flowchart as a programming tool. >>Back to OS, I think that we need to get out our hammer and saw and >>build a quality lifestyle on multicomputers. Quality to me means: >>1) a dynamic multi-tasking environment on every node >>2) a powerful, no-nonsense, save the pardigms for later, support everything >> message passing system on every node >>3) Unix file access in all its glory from every node >>4) lots of control (user is boss not servant) and monitoring >Naive Motherhood: somewhat. This is partially what led to CISC >architectures. It also created Unix and RISCs. This isn't a flame, but >I think we put too much functionality into some parallel systems... I think a generally useful DMMP system has to support node multitasking at least to the level of permitting the "virtual processor" concept. You shouldn't (within practical limits) have to size the problem to fit the machine. I'm not so concerned about file system access; you can always build a postprocessor to interpret the answer for you once it exists in some algorithmically convenient form. The stripped-down compute-node kernel is a good idea, as long as you can spawn multiple copies of it on a physical processor. Similarly the specialized file server node is a good idea as long as it has adequate connectivity to the rest of the machine. It should also support multitasking, since there are useful IO massaging processes that can be done on them rather than screwing up your load balance by putting them on "compute nodes". >>I think MIMDizer and F90 are logical next steps in multicomputer tool >>technology. I/O and comm libraries like CrOS and Cubix are also very good. >>Linda and Strand are, IMHO, appropriate for functional parallelism... >I am not so sure. (Sorry John 8^). I don't know how best to describe >what we have to do... >... User communities must be >willing to try new developments like OSes and languages, and architecture. Again, I expect to see the resurgence of the flowchart in the form of graphical programming aids. What good is that Iris on your desk if it can't help you program? If all you can do is display the results of your work, it might as well be a big-screen TV in the conference room. Once we have the right taxonomy and flowcharting concepts, we can start to merge the various parallel technologies that have developed (MIMD, SIMD, VLIW, dataflow, ...) into systems that can be architecturally parametrized to suit the application/range of applications desired. Maybe I should go back to grad school. Jeff Carroll carroll@atc.boeing.com