Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!ucsd!hub!eiffel!nosmo
From: nosmo@eiffel.UUCP (Vince Kraemer)
Newsgroups: comp.lang.eiffel
Subject: Re: Mosaic Benchmark on other platforms
Summary: Slower, why, how to improve
Message-ID: <276@eiffel.UUCP>
Date: 29 Mar 90 01:11:05 GMT
References: <2110@kiwi.mpr.ca>
Organization: Interactive Software Engineering, Santa Barbara CA
Lines: 120

>In article <21110@kiwi.mpr.ca>, Gail Murphy (murphy@eric.mpr.ca) writes:

>[Some intro comments and some really embarassing statistic deleted]
>
>These results are significantly different from the results posted previously, 
>where the authors found:
>
>[Quote from Kim Rochat's article deleted]
>
>The executions were conducted as described in the previous article.
>(ie.  Eiffel C-package generation was used with no assertions, the
>program was run twice in succession, etc.).  Similar results were found
>using the Ultrix CC compiler.  The following differences to the previous
>benchmark (besides Host) exist:
>
>+ the platforms have 8Mb memory as compared to 12Mb Hosts in the
>  previous benchmark 
>
>+ different versions of g++ and Eiffel were used.
>
>Use of prof on the Eiffel C-package outputs (the first 10 lines):
>
> %time  cumsecs  #call  ms/call  name
>  36.3   210.061637611     0.13  _sigblock
>  12.1   279.99                  mcount
>  10.8   342.59 997002     0.06  __0a8005_row_contrastcolorcanappearat
>   5.8   375.97                  _c1_item
>   5.4   407.151637611     0.02  _setjmp
>   5.3   437.85   1000    30.69  __0a8000_row_create
>   3.6   458.61                  _c1_put
>   2.9   475.46 633603     0.03  __0a9001_random_dont_care
>   2.7   490.96 633603     0.02  _random
>   2.4   504.76                  _c_put
>

From the evidence that I see here, I think that assertion checking was
not turned completely off.  The way to do this, which is a tad bit
obtuse in the documentation is to set up the SDF as follows:

NO_ASSERTION_CHECK (Y): ALL
PRECONDITIONS (N): ALL
ALL_ASSERTIONS (N): ALL

C_PACKAGE (Y): {some dir name}

This is the only explanation for the presence of the _sigblock and
_setjmp calls.

>A profile of the Eiffel C-package on the Apollo, however, did not reveal
>anytime spent in _sigblock. A profile of the g++ code was not possible
>as the -p and -pg options are not yet supported.
>
>The call to sigblock (an operating system routine) seems to come from the
>SETJMP2 macro.  The SETJMP2 and OUTJMP2 calls in the 
>row_contrastcolorcanappearat routine were commented out (I think
>this disables proper exception handling).

This disables the "default" exception handling behavior; printing out
an exception stack.  Rescues will still be taken care of - if present
with all assertions off.

(It should be noted here that the use of rescue inside a loop will also
produce many calls to setjmp.  Therefore, don't use rescues inside inner
loops is a good rule to remember when doing Eiffel programming "for speed".)

>
>This resulted in the following:
>
>Host      Language   Compiler     Execution Time (secs)   
>                                  User   System  Wall     
>----      --------   --------     ----   ------  ----     
>
>VAX 3100  Eiffel     gcc         109.8    66.5   2:58
>
>Ap 3500   Eiffel     Apollo CC   111.1     4.4   1:58
>
>Sun-3/50  Eiffel     Sun CC      109.2   106.6   3:36
>
>There is a notable cost, then, for performing the SETJMP2 and OUTJMP2 
>macros.  Is this a change from Eiffel Version 2.1?  Why else are the
>results of the benchmarks remarkably different?  Is there a more performance
>cost efficient means for performing these actions?

The above I'll take in order:

1. There has been very little change in the implementation of assertion
handling from Eiffel 2.1.

2. I'm not too sure why these benchmark results are so different.  One
possible reason may be, was the SDF OPTIMIZE line set to true, for all
classes?  What was the performance on the VAX 3100, using the Ultrix C
compiler?  I know that the gcc optimizer is a real winner: if the gcc vs
Ucc time is comparable, it would indicate that C compiler optimization
was not being done.  I have a gut feeling that the time difference
between the Vax3100/gcc combo vs. Sun-3/50/Sun cc in this second set of
trials should be higher, if gcc is optimizing.

Another cost that is due to preconditions being used is the use of a
routine to access row members, instead of a more direct access method
through a macro expansion.  This is still being done here (See the note
above about turning off all assertions), thus adding to the number of
functions being called by about 2,000,000

3. There are ways of implementing exception handling with a better
cost/benefit.  The problem is they aren't very portable.  Our goal was
portability for the systems produced.

Also worth noting, we see assertions as a development tool.  They are to
be stripped out (via recompilation) at delivery time.


I hope this has helped shed some light on the subject and not cloudy it.
I too would be interested in find out about the results of this
benchmark on other platforms -- especially 386 boxes using the
implementations of c++ available for them.

Vince Kraemer
ISE Jack-of-all-Trades
business related reply-to: eiffel@eiffel.com
personal correspondences reply-to: nosmo@eiffel.com