Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!ucbvax!RELAY.NSWC.NAVY.MIL!snorthc
From: snorthc@RELAY.NSWC.NAVY.MIL
Newsgroups: comp.protocols.tcp-ip
Subject: Re:  Comparing Performance of TCP/IP or OSI on Multivendor Equipment
Message-ID: <8908231639.AA02739@ucbvax.Berkeley.EDU>
Date: 23 Aug 89 12:43:55 GMT
Sender: daemon@ucbvax.BERKELEY.EDU
Organization: The Internet
Lines: 68

> From: dvnspc1!gary@burdvax.prc.unisys.com  (Gary Barrett)
> Organization: Unisys Corporation, Devon, PA
> Subject: Comparing Performance of TCP/IP or OSI on Multivendor Equipment

> I would be interested in hearing from anyone on the net who is
> wrestling with the problem of benchmarking TCP/IP and OSI software
> across equipment from various vendors.

Here at the Naval Surface Warfare Center we are facing the same problem!

> The only benchmark tool seemingly available (for TCP/IP) is ttcp.  

I get the impression that tools like nfsstones and the X benchmark sw
available on expo aren't what you are after.  I have two subnets assigned
for testing purposes only.  I have measurement tools (Excelan Lanalyser
and FTP SW Lanwatch) stationed on these subnets.  By isolating the
hardware and software being tested I am getting reproducible results.
The problem with this approach is that it is very manual, because the
collection points are MS-DOS machines.  I am currently exploring using
SNMP agents on routers between the subnets to collect data and using
the SNMP client software from CMU to get the data from the agents for
analysis.  YES, there HAS to be a BETTER way!

I am trying to answer questions like these:

Given two protocols/applications with similar functionality such as:
	- TCP's FTP;
	- OSI's FTAM.
Which is more efficient?  This has to be considered over a wide
range of tests: establishing the connection; xfer small, large files;
various hw/sw vendor implementations; file management capabilities.....

Given a single protocol or application which vendors implementation
is "best":
	- meet standards such as RFC?
	- interoperate with other major vendors implementations?
(of course the first case should ensure the second :-) )
	- handle exception conditions.  This last can probably never
be tested properly with a pure standard benchmark.  There are zillions
of things worth testing; sticking to file xfer, try ftping a fairly
large file from a noisy subnet that has a Bridge/3com GS/3 as a router.
Many FTPs will break here because large efficient packets often won't
get through.  FTP SW's FTP has an option that allows you to change
the window size.  (anyone know how to do this with SUNOS or Ultrix?)

Given identical protocol/application/implementation what is the
effect of changing the network adapter or the router?

What are the protocols/applications required to support a:
	- OA user;
	- "business (lots of database access) user;
	- scientific user?
 
I am making some slow progress in collecting this data, the really
hard task is to build a model from the empirical base.  If anyone
has a good recommendation for a good simulation for a 
non-deterministic network (UNIX minis on ethernet with PC/MAC/workstation
clients) I am quite interested.

Any suggestions, comments are solicited.  Flames are OK, I used to be
a potter and own asbestos, fiberfrax, and kevlon protective gear.

	thank you for your support

	stephen northcutt (snorthc@relay.nswc.navy.mil)

My management is too busy trying to give my office to someone else
to worry about what I say.