Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!usc!jarthur!uunet!mcsun!ukc!dcl-cs!aber-cs!athene!pcg From: pcg@cs.aber.ac.uk (Piercarlo Grandi) Newsgroups: comp.lang.misc Subject: Re: C's sins of commission Message-ID: Date: 19 Nov 90 13:50:17 GMT References: <27376@megaron.cs.arizona.edu> <-M.6315@xds13.ferranti.com> <1990Nov18.031503.2728@bushido.uucp> Sender: aro@aber-cs.UUCP Organization: Coleg Prifysgol Cymru Lines: 69 Nntp-Posting-Host: odin In-reply-to: dbc@bushido.uucp's message of 18 Nov 90 03:15:03 GMT On 18 Nov 90 03:15:03 GMT, dbc@bushido.uucp (Dave Caswell) said: [ ... DMBS ar usually better than custom written code ... ] pcg> The same applies for example to sort utilities -- normally it takes a pcg> lot of effort to beat a well written generic sort utility on any system pcg> I know of. dbc> This sure doesn't my experience. My experience is that custom dbc> written code can take a problem that takes 10-20hrs. to solve and dbc> change it into a 10-20 minute problem. In other words the custom dbc> written code is what allows the solution to actually be used. I would say that this is *extremely* unlikely to be common for problems that fall within the scope of a DBMS query planner. And if these are pathological problems, than you are right, and I had said that these indeed require csutom coding. But I would be very skeptical of anybody's claiming that they can routinely do better than a DBMS by using custom code. If you want, I have anecdotal evidence that custom written DB applications using B-tree libraries or similar low level things were substantially slower than loading the same data into a DBMS and letting it do its query planning. dbc> I'm not saying that Oracle or whatever isn't a very useful tool dbc> just that there is a whole host of problems that it can't solve. But it solves the problems of surely more than 50% of the people that use a computer (CS types tend to forget that computers are not used solely to develop compilers and operating systems, or clever numerical codes), and this is damn important. I would say that for the extremely large class of application that can be done with a DBMS using anything else should be justified with great and good reasons. For things that are not within the scope of a DBMS, there is something equivalent to a DBMS. Before rolling one's own, if you do statistics you had better use S or the BDMP library, and so on. dbc> Many of these problems are relatively easy to hand-code using well dbc> known algorithms. (well known means published in CACM or something dbc> similar). And here we have a big misunderstanding. Programs and utilities are not algorithms. A sort utility source is maybe one or two orders magnitude larger and more sophisticated than that of its core algorithm. Admittedly a lot of this is not value added -- it would not be there if the environment were more cooperative. But, a lot of it is value added. It is a common delusion to think that algorithms and programs are the same thing (I would call it the "Dijkstra book syndrome"). To make a useful program out of an algorithm takes a lot of effort. Take for example sorting: if all you have to sort is a few kilobytes, almost any algorithm will do and you can code it straightforwardly. But if you have large masses of complicated data either you get into difficult things like handling of external storage, or, if you have single level storage, you must make sure you have good locality. Just as an example of a relatively simple minded sort utility, take the UNIX sort(1). Its logic is "relatively easy to hand code using well known algorithms". Have a look at the source for a demonstration :-). I'd bet that not many programmers around can do significantly better than the UNIX sort(1) command, even on special cases. -- Piercarlo Grandi | ARPA: pcg%uk.ac.aber.cs@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk