Path: utzoo!utgpu!jarvis.csri.toronto.edu!clyde.concordia.ca!uunet!cs.utexas.edu!tut.cis.ohio-state.edu!pt.cs.cmu.edu!spice.cs.cmu.edu!af From: af@spice.cs.cmu.edu (Alessandro Forin) Newsgroups: comp.os.mach Subject: Re: Mach performance? [Long] Summary: Some numbers for pmaxen Message-ID: <7372@pt.cs.cmu.edu> Date: 20 Dec 89 19:05:12 GMT References: <14246@jumbo.dec.com> Organization: Carnegie-Mellon University, CS/RI Lines: 136 In article <14246@jumbo.dec.com>, discolo@jumbo.DEC.COM (Anthony Discolo) writes: > Does anyone have benchmarks that compare Mach to BSD and/or Ultrix > in the areas of RPC/networking/scheduling? > > Any pointers would be greatly appreciated. > > Anthony Seems to me large part of the question wants to compare apples and oranges. 1- Do you really believe BSD and derivatives to have an RPC mechanism ? Are you thinking of Sun's/Apollo/... RPCs or .. what ? How do you believe they can be correctly compared to Mach's IPC ? 2- How do you "measure" a scheduler ? On a multiprocessor ? Under which load ? Do you believe the functionalities of the Mach scheduler (e.g. user processor allocation, handoff scheduling, fixed/timeshare priorities ) are in any way comparable to U*x ? If you have any more precise definition of the comparisons you want to make I'd be glad to provide you with an answer. Assuming for now you just want *some* comparison of Ultrix & Mach 2.5 on a pmax here is some data collected some time ago on my pmax, comparing under the same conditions (machine, disk, environment, programs&input, buffer cache, network, time of day, etc etc etc) Mach and Ultrix. [Since we have made progress since, I sometimes added the numbers I get _right_now_ on the same machine, but clearly under different & less controlled conditions]. As far as networking, when I tested our ethernet device driver I was satisfied by measuring a binary ftp of a large file (the Ultrix image :-) into /dev/null using Ultrix's ftp, multiuser, on the same thinwir-ing. Among two Ultrix pmaxen I got no better than about 190 kb/sec, among two Mach pmaxen I got up to 300 kb/sec, as measured by ftp itself. We do have internal test programs for Mach IPC. Here are the results I get right_now on my pmax (multiuser, X11 running, my emacs & news programs in the background, some 20 assorted systems servers for the fancy CMU environment). binding to host testarossa Test syscall: Iters:10000 E:228 U:31 S:197 Test localLoop: Iters:10000 E:0 U:0 S:0 Test localNull: Iters:10000 E:0 U:0 S:0 Test localAdd: Iters:10000 E:16 U:15 S:0 Test localBigIn: Iters:10000 E:203 U:203 S:0 Test localBigOut: Iters:10000 E:203 U:203 S:0 Test localBigInOut: Iters:10000 E:390 U:390 S:0 Test remoteNull: Iters:10000 E:1985 U:15 S:1125 Test remoteAdd: Iters:10000 E:2062 U:141 S:985 Test remoteBigIn: Iters:10000 E:2797 U:172 S:1234 Test remoteBigOut: Iters:10000 E:2016 U:141 S:1000 Test remoteBigInOut: Iters:10000 E:2765 U:390 S:1266 The localXXX entries are for local procedure calls, remoteXXX are for the same operation invoked on a server process. Add just adds two numbers, BigIn passes (by value) as IN parameter a 200bytes string, BigOut returns it (by value), BigInOut does both. All tests ran 10000 times, Elapsed/User/System times are as given, e.g. a null RPC takes 200 microseconds, elapsed. My standard use of the machine is for compilation, so here are two compilation benchmarks: a small one and a large one: Compilation benchmark, small programs. Ultrix Mach Elapsed 17.9 15.4 Breakdown: 1.5 real 1.2 real 2.4 real 2.1 real 1.5 real 1.1 real 1.7 real 1.5 real 2.0 real 1.7 real 2.0 real 1.7 real 1.6 real 1.3 real 2.7 real 2.4 real 1.6 real 1.4 real Mach kernel compilation benchmark. Ultrix Mach Mach-nbc Elapsed 2325.0 2139.0 2180.0 User 1404.0 1471.6 1388.8 System 413.0 323.5 351.1 Utilization 78% 83% 77% I/O 7120+17397 3278+17756 291+6271 [The Mach-nbc entry if for a no-buffer-cache kernel] Some specific U*x tests are performed by a suite of little programs we wrote ourselves. Ultrix Mach right_now Elapsed 15.1 secs 15.6 secs exec 16 ms 12 ms touch 0.566 ms 0.684 ms [0.440] fork 4.960 ms 4.218 ms [3.125] getpid 2.4 secs 2.8 secs [1.9] iocall 8.1 secs 8.2 secs puzzle 0.6 secs 0.6 secs pgtest 1.0 secs 1.3 secs File system performance is compared by Satya's benchmark (see SOSP87) Satya's Filesystem benchmark results (1) On local disk Ultrix Mach Total elapsed 121 secs 108 secs Phase I: Creating directories 2 secs 4 secs Phase II: Copying files 9 secs 9 secs Phase III: Recursive directory stats 9 secs 7 secs Phase IV: Scanning each file 18 secs 9 secs Phase V: Compilation 83 secs 79 secs Hope this helps, sandro- Disclaimer: The Ultrix group have different ideas on what should be tested and by which benchmarks. Their opinions might be very different from ours. The opinions of our users seem to agree with our own opinions.