Path: utzoo!attcan!uunet!ns-mx!ceres!zaphod.mps.ohio-state.edu!tut.cis.ohio-state.edu!ucbvax!SMITHKLINE.COM!dixons%phvax.dnet From: dixons%phvax.dnet@SMITHKLINE.COM Newsgroups: comp.sys.sgi Subject: Multi-processor problems Message-ID: <9001120157.AA15338@smithkline.com> Date: 12 Jan 90 01:57:57 GMT Sender: daemon@ucbvax.BERKELEY.EDU Organization: The Internet Lines: 45 I have been working on getting a FORTRAN program running parallel. I seem to have gotten it running with reasonable load balance, etc but have observed a curious phenomenon which depends on the system load. Here's what happens: When I run on a system with no other users, I see a speedup which depends on the number of processors used in a sensible way. The final speedup with 4 processors is about 1.75x. But if I run the same job on the system when one other compute bound (single processor, non-mp) job is running here are the running times as a function of the number of processor used in the parallel job: 1 proc 2 proc 3 proc 4 proc 7:14 5:17 4:32 about 22 min I say about 22 minutes since time returns the rather strange results: real 30:35.19 user 1:06:58.08 sys 6.41 A ps on the 4 proc job just before it finishes show the following 5451 ? 22:03 pdg 5439 ? 22:54 pdg 5452 ? 22:01 pdg 5453 ? 21:58 pdg In other words, using four processors suddenly takes 3 times longer than 1 processor. This seems to be repeatable. Also if two other computer bound jobs are each using a processor then the problem starts when three processors are used for the mp job. Four single processor versions of the same job all running against the same other compute bound job all finish in about 7:20 each. Someone else with a 240 has mentioned to me that he has seen similar behaviour. Have others of you observed the same? Is there a fix for this? It seems to me to be a rather serious problem which would effectively prevent multi-processor SGI boxes from being used in parallel mode unless they were dedicated to a single compute job. The system in question is a 4D240 with 32Megs running Irix 3.2. The programs are CPU bound, do little I/O and are not swapping much (almost not at all). CPU utilization is high (>90%) in user mode on all cpus. I believe that similar behaviour occurred with earlier releases of Irix as well but I haven't gotten around to looking systematically till now. Scott Dixon (dixons@smithklin.com)