Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!tut.cis.ohio-state.edu!ucsd!hub!eiffel!kimr From: kimr@eiffel.UUCP (Kim Rochat) Newsgroups: comp.lang.eiffel Subject: Re: Benchmark article - Eiffel vs. C++ vs. Smalltalk-80 Summary: Number of function calls was supposed to be the same Keywords: benchmark, Eiffel, C++, performance Message-ID: <275@eiffel.UUCP> Date: 28 Mar 90 23:01:00 GMT References: <274@eiffel.UUCP> <10617@alice.UUCP> Organization: Interactive Software Engineering, Santa Barbara CA Lines: 73 In article <10617@alice.UUCP>, ark@alice.UUCP (Andrew Koenig) writes: > Of the six C++ functions shown (below), four of them were so short that > many C++ programmers would have written them as inline as a matter of > routine, namely Row::contrastColor, Row::addRandomColorAt, > Row::addBaseColorAt, and Row::addContrastColorAt. The Eiffel and C++ benchmarks were intended to be constructed to make the same number of function calls, so as not to be comparing the function call overhead of the various machines. In reviewing the benchmarks, it appears that I didn't quite succeed. Looking only at the inner loop of subsequentRow, where the vast majority of the time is spent, the following diagram is an attempt to convey the call structure of each routine. Horizontal lines indicate entering a new function. C++ Eiffel subsequentRow subsequentRow ----------------------------- ----------------------------- if contrastColorCanAppearAt then if contrastColorCanAppearAt then addRandomColorAt dont_care ----------------------------- ----------------------------- if random then randbit addBaseColorAt -------------- else addContrastColorAt random ----------------------- -------------- ContrastColor ----------------------- ----------------------------- else addBaseColorAt else ----------------------------- ----------------------------- There are no function calls in the Eiffel code corresponding to the C++ calls to ContrastColorAt or the conditional calls to addBaseColorAt by subsequentRow. As ark implies, the benchmarks would have been fairer if these two routines had been inlined. This entire discussion brings up another interesting topic, which is how can you tell how many function calls you're generating with your code? Under C++, inlining is directly under programmer control. With Eiffel, it's under the control of the compiler and optimizer. For example, I don't count calls to "enter" and "entry" for the Eiffel code, since I happen to know that the compiler inlined" them. Since I used C package generation for Eiffel, the compiler also ran an optimizer which analyzes routines and inlines suitable candidates. In reviewing what the compiler actually did, it chose to inline the routines "firstrow" and "subsequentRow". Their omission accounts for only 1000 calls out of several million. As the readers of the original article pointed out, any of the three versions of the program could be further optimized for improved performance. The authors were attempting to benchmark obvious implementations of the algorithm - i.e. versions which conform to the spirit and style of each language without any iterations tuning performance. Ark might suggest that we didn't conform to the spirit of C++ because we didn't use inline functions. I accept that criticism with the defense that we didn't use them because our local style guidelines suggested that inlines not be used for C++, both because they violate encapsulation by placing implementation in the header files, and because inlining was unreliable under cfront 2.1, under which the C++ version was first developed. I'd like to thank everyone for their comments on the article and reports on subsequent experiments. I regret that I cannot revise my benchmark results due to a change in employment. Kim Rochat Interactive Software Engineering kimr@eiffel.com