Xref: utzoo comp.protocols.nfs:1934 comp.arch:21405 Path: utzoo!news-server.csri.toronto.edu!rutgers!usc!wuarchive!zaphod.mps.ohio-state.edu!rpi!crdgw1!crdos1!davidsen From: davidsen@crdos1.crd.ge.COM (Wm E Davidsen Jr) Newsgroups: comp.protocols.nfs,comp.arch Subject: Re: Incremental sync()s and using disk idle time Message-ID: <3253@crdos1.crd.ge.COM> Date: 12 Mar 91 22:31:28 GMT References: <28975@cs.yale.edu> <10773@dog.ee.lbl.gov> <3236@crdos1.crd.ge.COM> <1991Mar12.194704.17859@zoo.toronto.edu> Reply-To: davidsen@crdos1.crd.ge.com (bill davidsen) Followup-To: comp.protocols.nfs Organization: GE Corp R&D Center, Schenectady NY Lines: 64 In article <1991Mar12.194704.17859@zoo.toronto.edu> henry@zoo.toronto.edu (Henry Spencer) writes: | The only sensible place to put smarts is in host software, where it can | be changed to match the workload and to keep up with new developments. | "Smart" hardware almost always makes policy decisions, which is a mistake. | The money spent on "smartening" the hardware is usually better spent | on making the main CPU faster so that you can get the same performance | with smart *software*... especially since a faster CPU is good for a | lot of other things too. And in a perfect world the faster CPU to provide the boost will cost the same as the smart controller. Unfortunately I don't live there, and I suspect most readers don't either. The incremental price to get i/o improvement is *in most cases* much smaller to upgrade a controller (add cache, whatever) than to upgrade the CPU speed and all the support circuits that implies. For a multiprocessor system this becomes incrementally true. There's also an issue of reliability. For any hardware or software failure other than power failure the smart controller seems more likely to complete moving the data from cache to disk than the kernel to move it from disk buffers to disk. That's a gut feling, the smart controller may have a higher failure rate than the dumb controller, but it seems likely that the smaller hardware and software content of a controller will make it more reliable than the CPU, memory, and any other controllers which could do something to make the o/s crash. The interesting thing is that in systems with multiple CPUs, if one CPU is handling all the interrupts it has a tendancy to become an extremely expensive smart controller. Yes, it can do more for the CPU bound jobs, but is that cost effective for any load other than heavy CPU? I see no reason why an expensive CPU should be used to handle scatter gather, remap sectors, log errors and issue retries, etc. It doesn't take much in the way of smarts to do this stuff. It certainly doesn't take floating point or vector hardware, processor cache, or an MMU which supports paging. A CPU designed for embedded use can have a small interrupt controller and some parallel i/o built into the chip. This lowers chip count, connections, and latency, which means smaller, less expensive, and more reliable devices. And i/o buffers can be made from cheap slow memory and still stay ahead of the physical devices. Moving the filesystem to another CPU isn't really a "smart controller" it's "distributed processing" more or less. That's certainly not what I mean by smart controller, at any rate, so maybe the term is being used loosely. I'm all in favor of having the decisions made by the o/s, but when it comes time to actually move data from memory to disk, I'll find something better for my CPU to do than keep an eye on the process. If I can issue an i/o and tell when it's done, and if the controller is configured to insure that data don't sit in the cache for more than time X (you define X), then I don't see any problem providing ordered writes as needed for data security, and good performance as needed by loaded and i/o bound machines. That's what I mean by a smart controller and that's what I think is optimal for both performance and cost effectiveness. -- bill davidsen (davidsen@crdos1.crd.GE.COM -or- uunet!crdgw1!crdos1!davidsen) "Most of the VAX instructions are in microcode, but halt and no-op are in hardware for efficiency"