Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!otello!gear!am!alex From: alex@am.sublink.org (Alex Martelli) Newsgroups: comp.unix.aix Subject: Re: Paging-space problems Message-ID: <1990Nov22.224453.908@am.sublink.org> Date: 22 Nov 90 22:44:53 GMT References: <1990Nov16.001840.2783@gpu.utcs.utoronto.ca> <311@cadlab.sublink.ORG> <1990Nov14.223820.29154@arnor.uucp> <1990Nov15.180919.17089@gpu.utcs.utoronto.ca> <4285@awdprime. Organization: Premiata Famiglia Martelli & Figli Lines: 73 rogers@rogers.austin.ibm.com (Mark D. Rogers/100000) writes: ... [after a clear explanation of what AIX is doing re malloc()] >We have at Least two very distinct classes of customers where >paging space allocation is concerned: >1. Those such as yourselves, who want your application to either > run, or not, based upon how much backing storage you have. > If your app. doesn't run, go buy another disk & add paging space. >2. Customers who like to allocate all the virtual memory they can, > knowing that it will never all be used. This, alleviates the > need for any complicated run-time memory management schemes in > their model. It is a very convenient programming construct. Thanks for the explanation. However, in our case it's not really that the application "would not run" - it would run fine as long as the user was only doing solid models whose complexity were compatible with his/her amount of paging space, and would give a harmless warning if a model turned out to be too complex for that (and in many cases there will be some work-around for such resource limitations). I would assume that many applications where the user interactively asks for memory-consuming operations, from symbolic maths to statistical data analysis, would be similar to solid modeling in this respect. Pity that this class of apps has been sacrificed to others who apparently need garbage-collection stuff and appear to be too lazy to do it:-). Another consideration is that many applications will be PORTED from the huge existing Unix base to AIX; these will NOT expect unreliable, over-committed malloc()! For newly developed apps it may be a draw, but considering potential portings to AIX, I believe the 2 vs 1 choice was inferior. Pity! Touching all pages and calling psdanger() on EACH malloc() is WAY too much overhead, I think. I believe I will have to rewrite a malloc()/free()/realloc() package, putting all overhead only on the sbrk(); it's either that, or have to explain to customers how and why our solid modeler is so fragile on AIX, while it's solid as a rock on HP/UX, Ultrix, Sun/Os, SONY/NeWS, and so on and on! >From a historical perspective (for your information) we had >a number of `type 2' customers Early on on the RT, and >that influenced the System V-like behaviour of the RT somewhat. >It also has something to do with why we, on the RISC System/6000 >still want to allow large virtual memory objects efficiently. >Basically what we have on the RISC System/6000, is a hybrid attempt >at allowing both types of customers. SIGDANGER is a compromise attempt >to allow them to co-exist. Admittedly, it is not perfect, however, >no matter what route one goes in, on this issue, we have found >thorns in the path. We are continuing to investigate the entire >issue, and welcome any comments. I hope this input is some use! If you would just include, say, a libmalloc.a which does all necessary checks on sbrk(), for "compatibility with non-overcommitting Unices", it would be nice. Best, I believe, would be a tunable parameter to force sbrk() to non-overcommittance on a system-wide basis; I don't really see how you could make SOME processes overcommit while all defaults to safe allocation, or viceversa, but if you could, that would definitely be a jump upwards in quality, to go with others that AIX has undoubtedly. Feel free to follow this up either here or by email (but in this case to staff@cadlab.sublink.org, please - it's not really a "personal interest" thing, although I'm replying from home - I can't really afford a RS/6000 machine at home...:-). And thanks again for your clear explanation and comments. -- Alex Martelli - (home snailmail:) v. Barontini 27, 40138 Bologna, ITALIA Email: (work:) staff@cadlab.sublink.org, (home:) alex@am.sublink.org Phone: (work:) ++39 (51) 371099, (home:) ++39 (51) 250434; Fax: ++39 (51) 366964 (work only), Fidonet: 332/401.3 (home only).