Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!rutgers!sun-barr!decwrl!hplabs!hpfcdc!daveg
From: daveg@hpfcdc.HP.COM (Dave Gutierrez)
Newsgroups: comp.sys.hp
Subject: Re: Experience sought with large HP 9000 clusters
Message-ID: <5570225@hpfcdc.HP.COM>
Date: 26 Jun 89 23:51:01 GMT
References: <3517@cps3xx.UUCP>
Organization: HP Ft. Collins, Co.
Lines: 67

<We are beginning to set up a large network of HP 9000 machines running
<HP-UX 6.5. Our target network, which we will be assembling over the course
<of the next year, will have four cluster servers with approximately 30 cnodes
<per cluster. I would like to hear from anyone who has done this sort of thing
<with this number of machines.
<
<The cluster servers will be 9000/360's with 12mb of memory and a fast

	A 370 would be nice, but the 360 is a sweet box.

<and a slow SCSI interface. We will put four 700mb disks on the fast SCSI
<interface, an 8mm tape drive (Perfect Byte EXB-8200) on the slow SCSI
<interface, and an HP QIC drive and a printer on the HPIB interface (the
<console, a 700/92, is using the serial port).

	Sounds reasonable. Increasing to 16Mb may be nice but may not be
	necessary. Define root server configurable parameters as follows:

		o num_cnodes 12	(you will just waste ram if larger)
		o nbuf    768	(maximum = 3Mb file-system buffer cache)
		o configure out all unnecessary device drivers or other 
			capabilities to conserve RAM (i.e. NFS, etc)
<
<The cnodes will mostly be 9000/340's with 8mb of memory and a 150mb HP7958B
<disk on the HPIB interface.

	I assume that you have a high-speed HPIB interface, if not you will
	be able to swap faster over the network. Your application will 
	certainly determine your mileage, but you may not need the local
	swap discs on all cnodes.

<The cnodes will all be configured for local swap.
<Applications include C and FORTRAN programming, Frame word processing, and
<various engineering packages such as SDRC I-DEAS.
<
<We'll have a sprinkling of other machines, including a 370/SRX and an 835,
<which we'll problably run as standalone systems (on the network, but not
<part of a cluster).

	no problem... I would however put two lan cards in the root-server
	and put all cnodes on a private thin-lan, using the root-server as
	a gateway to the facility backbone. Easier to troubleshoot, lan-break
	detection will still work, diskless traffic will be isolated, etc.
<
<Questions: Is this a reasonable number of cnodes per cluster?

	Given the described configuration, this should not be an issue.
	OF COURSE IT ALL DEPENDS on the applications.

< Has anyone experienced problems running out of process ids in a large cluster? Does

	The HP-UX implementation recycles pids across the cluster guaranteeing
	unique PIDS across the cluster. This should not be an issue.

<anyone have a workaround for the inability to put spooled devices, e.g.,
<printers, on cnodes? Any advice? Horror stories?

	Unsupported scripts have already been supplied in other responses.
	Sorry, no horror stories to contribute...
<
<Thanks for your help.
< - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  O
< John Lees                     A. H. Case Center for CAE/M     OoO
< UNIX Systems Manager          236 Engineering Building        /O
<                               Michigan State University       |
< lees@frith.egr.msu.edu        East Lansing, MI 48824-1226    (|)   flower
< ...!uunet!frith!lees          Phone: (517) 355-7435/6453    __|__   power