Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!cornell!batcomputer!rpi!rpi.edu!deven From: deven@pawl.rpi.edu (Deven Corzine) Newsgroups: comp.sys.amiga.tech Subject: Re: Unix V7 functionality under (or along with) AmigaDOS? (*LONG*) Message-ID: Date: 15 Mar 89 18:55:51 GMT References: <0776.AA0776@julie> Sender: usenet@rpi.edu Reply-To: shadow@pawl.rpi.edu (Deven Thomas Corzine) Organization: RPI Public Access Workstation Lab, Troy NY Lines: 436 In-reply-to: mcr@julie.UUCP's message of 9 Mar 89 18:08:28 GMT In article <0776.AA0776@julie> mcr@julie.UUCP (Michael Richardson) writes: >Someone said: (beats me, my copy of Amiga Usenet isn't up that type of thing) >[Reading in the message and deleting everything but the Subject: and >the Message-ID: line and then changing Message-ID: to References: works >nicely. Anyway--- ] SOMEONE SAID: >>>>>The shell has the responsibility of unloading (or not unloading >>>>>in the case of resident segments) the programs. ^^^^^ Randell Jesup said. >>>> >>>>But does this hold true when the shell is NOT running under the CLI? >>>>It will be running as a DOS Process, but NOT as a CLI process. The ^^^^^ I said. (Alternating between him and I.) > I don't have any real AmigaDOS docs (just Exec docs) and I've yet to >completely understand just what is in the Process structure that is >not found in the Task structure.... ^^^^^ You said. :-) You can look at the Process structure in for starters... >>>>> Shells are shells, period. The Workbench is a visual shell. >>>Exit() itself does almost nothing, except return control to the "shell" >>>as if the program had exited. > But this also implies (correctly) that CLIs are NOT shells. They are >special. That is the very first thing that strikes people about Unix, >and where most if not all of its power comes from. Programs are programs. Well... I wouldn't say that. The CLI *is* a shell. A clumsy and quite poor one, but it is. It should be a cleaner implementation. >>I'm willing to use AddTask if it will still operate as a DOS process >>and be able to call dos.library routines. How should I go about doing >>this? Also, what makes pr_SegList bizare? (aside from usage of >>BPTRs?) > So am I!!!! I've been thinking about it, and I'm not really sure I *do* want to run as a DOS Process. Methinks maybe as a task instead. >>Hmm. Looking at "The AmigaDOS manual," (AmigaDOS V1.1, I believe; >>published February 1986.) it lists the Process structure starting with >>BPTR SegArray, describing it as an array of SegList pointers with its >>size in the first longword. [seems consistent with BSTR's.] > I really have to get this a copy. It could be better, but it's definitely better than nothing. >>On the other hand, in the include file, the >>Process struct is defined with pr_task, pr_MsgPort, pr_Pad, and then > ^^^^^^^ ^^^^^^^^^^ > I would assume that the pr_MsgPort has something to do with receiving >packets back from the file system task. (For Reads and Writes) Indeed it does. I wasn't questioning *what* pr_MsgPort was for... >>Hmm. This seems the only reasonable way to go; replace CreateProc(), >>and use AddTask(). I take it the extra segments in the SegArray >>(pr_SegList) are such as that cleanup code for CreateProc() which are >>not to be unloaded. May I safely assume that if I replace >>CreateProc() with my own function which sets up a structure starting > What stuff do you intend on adding? Not entirely decided yet. Probably will have some Unixy things like maybe UID's/GID's, etc... I'll have to think about it. >>output of the executed program, or set them to zero and define the >>standard input, output and error channels in my own structure? (I >>want stderr as a passed file descriptor, along with stdin and >>stdout...) I want to break dependency from the CLI, yet still be able > ^^^^^^^^^^^^^^^^^^^^^^^ >>to run programs which depend on the CLI themselves. (possibly by >>using or replacing the AmigaDOS RUN command.) > I disagree. CA= won't document a lot of things because they say > "it will break in 1.4" > but they won't change things because > "too many things would break" I suppose there is some of that, but it IS a rather difficult position to be in whatever they say. Don't be so hard on them. > I think the problem is: > "will user x be able to run program y under your shell" > (tell them to start a CLI) > the problem is > "will program x be able to run under the CLI or your shell" > That way, you can encourage people to write programs that conform >to your standard (generic programs will just need a relinking with >a new startup) instead of letting them run all their old >code unmodified. If they need the old stuff, "C=+ESC" gets most >people a CLI... NO. The idea here is to improve the shell and simplify life for the programmer and the user. Saying "What, you need the CLI? Well, go start one, then!" doesn't help at all. I can see adding a program, command or special syntax which would run the specified program in a CLI-type environment, if that's what it needs, but telling the user to f*ck off and die just because he wants to run a CLI-dependant program is not what I'm after. That sort of attitude is all too prevalent already. I believe in doing things right the first time. (Or as best as reasonable.) If I'm going to do it, I want it done right. If I didn't care if it were done right, I wouldn't bother doing it myself. There's plenty of programs done poorly out there. >>Similarly, should I zero pr_ConsoleTask and pr_FileSystemTask or not? >>(I intend to have read(), write(), etc. calls... functionally similar >>to Lattice's, but not dependant on the Lattice compiler or lc.lib >>library.) > Wish read(), write(), were better documented. I don't really understand >what they do... (internally) I'm not worried about how AmigaDOS Read() and Write() calls work internally. They're written in BCPL anyhow. (Ugh.) I can extrapolate well enough from the function's functionality (hmm - time to join the redundancy department of redundancy department...) and the data structures (i.e. the file system format on-disk) they access. I don't need to know exactly how it was done. (Though I'm always curious.) >>I have programmed quite a bit under Unix and rather prefer the Unix >>file system and system calls to many of those available on the Amiga. >>I don't much care for AmigaDOS or BCPL. Some of the user interface >>issues between Unix and AmigaDOS are trivial, like "../" vs. "/" for >>parent directory. >>Other differences, such as the way directories are implemented, differ >>more significantly. Unix-style directories are far more efficient at >>directory listing (but no the hash-lookup searching AmigaDOS does) >>than AmigaDOS with its backpointers. Also, the lack of links in >>AmigaDOS is a big loss. > Rah! Rah! Rah! Gee, wow. A supporter. :-) >>I want to write a shell which will be more cleanly implemented than >>the CLI (i.e. no BCPL) and which will start programs with argv AND >>envp arrays, and have Unix-type calls available. > Get out my head! That was my idea! :-) Foo. My idea first. It was part of my motivation behind the whole thing. (I just hadn't bothered to mention it yet when I first posted to the net for information...) > Actually that would be a good initial difference for this shell. >Set a flag (pr_CIN and pr_COS==NULL? hmm. ) Eeew. Yuck. No. I decided against that sort of approach already. Smacks of kludginess (what a word) and will probably be incompatible later on. (oh, but of course.) >and then pass the argv and envp. The program which >is executed could then check for the flag and (if not set) do the "standard" >startup or just use the argv and envp. (Or things can be linked >with a startup that doesn't have this. Programmer choice) Questionable. I'd rather not introduce such complications into the program startup; it goes against everything I'm trying to accomplish here. I'm looking for a cleaner implementation of both the shell and generally for running programs as well. Also, keeping the startup simple and whatnot will help keep program executable sizes small, which IS important to me. Besides, directly supporting the CLI environment for new programs like that not only encourages the kludginess already extant, but it also discourages people actually making full use of the setup I'm building. Not a help. > How to pass argv and envp? This is a message passing operating >system right? Well, send the task a message... How do you tell when >the task is finished? Well, exit() returns this message (with an error >code) and then does a Wait(). The exec.library (woops that name >is taken. The Execute.library then--- I guess I mean Execute.device see >below) then RemTasks it, and deallocates it (if it isn't on the resident >list) and makes the error code available to the task. I may well use message passing internally to the functions, but I do NOT want to force the programs to deal with message passing just to run. Workbench does. I don't want to. I'll keep it internal; that's fine. I'm trying to make things EASIER here. > What else? How about a standard tc_ExeceptionCode? > Make SIGBREAKF_CTRL_C, and ^E exceptions. ^C will >cause an "abort()" routine to be run. (signal() or sigvec()'able of course) >The standard abort() should be able to run some number of clean up >functions, and then it should call "exit()" I'm not so sure about that. I've considered such an idea. But, I don't want to tie this to any compiler, first off. (abort() and exit() ARE compiler-dependant, though ought to be fairly consistent across compilers.) And I don't want to use Exit(), certainly. (And I CAN'T if the task isn't a dos process.) But there's no telling what you might be interrupting with it. And what cleanup code might be missed... Needs more thought. > Also, when ^E is received, the task does a "Wait(SIGBREAKF_CTRL_F)" >and PRESTO, Job control!!! (Or course, ^E is signal()/sigvec() settable too.) >When the shell wishes the job to continue, just send a ^F signal. (Or >any other agreed upon signal) aCk. I don't want to use ^E and ^F regardless; if I implement any sort of job control, I'll try to use ^Z if I can. I'm not sure what the way to go is... > This assumes something about the shell though: that it isn't part of the >same task as the programs it runs. This is fine, as this CLI "subroutine" >stuff bugs me a lot. Ugh. Wasn't it clear from the start that the programs would be separate tasks from the shell? Perhaps not. Well, they most certainly would be separate tasks. I hate the "subroutine" structure of the CLI; it makes it seem like you've got a bunch of virtual MS-DOS (or some such simgle-tasking system) machines in one instead of a true multitasking system. Everyone's answer to wanting to run multiple programs at once is "That's easy! Just start a new CLI!" Can you say screen and system clutter? It's also clumsy as all hell, and totally unnecessary to boot. And WHY do I get this sinking feeling that about half the Amiga programmers out there (i.e. the mediocre ones) pulled their programming practices straight from the C-64?? Argh. Yeah, I know. They probably DID. *sigh* > It ISN'T fork()/exec(), but it does make a much more orthogonal >system, shells send messages to "execute" servers, much the same >way that comm programs send messages to "serial" devices... What you're describing is basically Workbench in text form. >>What I would like to do is write a file system based in part on Minix, >>and in part on AmigaDOS (but *no* BCPL!) which would work more >>effectively with floppy-based systems. (i.e. Mount file systems like >>in Minix/Unix, but volume-oriented, as in AmigaDOS.) > Have you talked to Colin (Plumb?) (microsoft!w-colinp) > His extent based file system looks quite nice... No, but I have noted his postings about his file system with some interest. It's a bit limited, however, being tied to floppies. But it could work well. Myself, I'd just assume stick with a block-oriented format, as it is much more portable. (like, to hard drives and file servers and whatever.) > I suggest that you start by writting a file system handler that >understands the AmigaDOS file system in C. Then rewrite it >(with a different file system type) that is completely C. (Get >rid of BSTR, etc... Incompatible, but that's ok.) What I will do is write a file system handler in C which will interface with AmigaDOS behind the scenes, handling the conversions for BPTRs and BSTRs so as to provide a clean and consistent interface for programs to use. I *could* write a handler which directly accesses AmigaDOS disks, but then I'd still have to deal with BCPL crap and I would merely be tying myself to a specific version of AmigaDOS. I couldn't use it for FFS, for example. No, just as well to have AmigaDOS play with its disks itself and coordinate that instead. >>I DO rather like the low level Exec. The ROM Kernal is quite well >>designed. I consider AmigaDOS to be rather poor, on the other hand. >>It DOES contain some rather clever design features; assigned devices >>are useful (though not good as a general replacement for environment >>variables) and being able to mount devices is a definite plus. > Unmount is need though. It would be good to have, for consistency, but I don't see it as a particular problem; devices which are mounted and unused don't really use much in the way of resources. And it shouldn't be too hard to write an unmount; Forbid() and dig around in the AmigaDOS structures and remove the entries mount put in. But it's not a nice thing to do. :-) >>But what I DO want now is the fork() and exec() system calls from >>Unix. (execve() for purists) And I'm willing to take a stab at >>writing them myself. > Why? In a way I rather like the idea of calling CreateTask with >an entry point. I think that the underlying mechanism could be >changed to support a more fork() like system, but in how many >cases is fork() not followed immediately by an exec()? Many cases. Simplest example, to implement redirection. > In this case, the vfork() call was invented, and also the >"copy-on-write" MMU stuff (although that is very usefull for >shared libraries too) vfork() is more efficient than fork() when *soon* followed by an exec(). a fork() *immediately* followed by an exec() is functionally identical to a task spawning function like CreateProc. The point is, *between* the fork() and exec(), you can set up the environment for the new process in the context of that new process. It is a valuable ability. >>Fork() would start with a FindTask(0L) call, allocate space for a new >>Task structure, copy the data over, and similarly duplicate the text >>and data segments, modifying the appropriate fields to point to the > ^^^^^^^^^^^ >>copies, and link in the new task to Exec with either AddTask() >>(obviously preferable) or manually if necessary. > As has been pointed out, this is VERY difficult without an MMU. Not difficult, just slow and sorta kludgy. As Randell said, it involves swapping the stack and data segments around, at every task switch. (of the forked task, that is.) If it is *soon* followed by an exec(), everything will be dandy. >>void AddTask(struct Task *, char *, char *); > task initialPC, finalPC Yeah, I looked it up after I posted. >>Anyhow, I see several points of possible difficulty. > So do I. >>First, is whether this fork() routine would need to disable >>multitasking (interrupts seem fine) at all, such as when copying the > I don't think so. Nor do I. >>Task structure of the current process. Or, is it perfectly safe and >>consistent (no race conditions) to let multitasking continue >>unhindered while it initializes the new Task structure? (Clearly, it > ^^^^^^^^^^^ > Nobody but you knows about the task structure until you AddTask. >And at that point, you had better be finished fiddling with it, because >the process is ready to run. Indeed. I don't think it's a problem, really. >>Second, how should the fork() call make execution begin at the return >>point of said fork() call? (Perhaps this question will answer itself >>when I look up AddTask().) > Look up the return address of the fork(). Put the return value under >it, and give the address of a routine that will just do an "RTS". >This stack is, of course a freshly copied stack, having all >the frame pointers, changed. Even if you leave all the data and >code shared, you must at least duplicate the stack. Must duplicate all, really. I don't know whether or not it's really worth implementing vfork() also. Maybe. As for getting the return address, a JSR to a point within the function code coupled with a JMP seems cleanest. >>Finally, (I hope) how to handle file descriptors. The solution for >>this would seem to be to have file i/o operations in this library - >>open(), close(), read(), write(), etc. and have a table of file >>descriptors, file pointers, and AmigaDOS file handles (for now, at >>least.) that the "parent" and "child" share. I don't know whether or >>not to try to preserve the parent/child relationship as Unix does, or >>try some other setup. I'll have to think about it. > The fexec() library function would lookup the highest FileHandle assigned >to ask (kept in an Exec list of course to eliminate that 20 files open bit) >and duplicate each one. This list would be part of the standard stuff >passed to the execute.device (along with argv, envp and the name of the >program and/or address at which to execute) WTF? No. The list of file descriptors is NOT to be duplicated. It is to be shared. (As it is under Unix.) > Parent/child is preserved without being forced. How is it preserved, pray tell? > (How do you do redirection? Well, I guess one has to write >a custom fexec() as one couldn't go around changing the your own >FileHandles, there being no seperation like fork(). --- We don't need >to duplicate a running process structure, just get a new >"OSInterface" structure. Modify that, and pass it to the execute.device. ??? Why? >Actually, that could be interesting: >OpenDevice("Execute.device","ls",&lsInterface,0) and then modify the >lsInterface (instance of OSInterface) to change the environment, etc... >and then do a "DoIO()" on it with a CMD_START (or Send/BeginIO() for >asynchronous processes...)) It is NOT going to be implemented as a "device". End of subject. It is NOT a device, nor does it pretend to be a device in any way, shape or form. And it is NOT going to use OpenDevice and DoIO to start a new process. Do you have ANY clue how f*cking complicated Exec device I/O IS to anyone new to the Amiga??? You have to do quite a bit to get something simple done. Fine if you already know it, but a supreme bitch to deal with the first 2 dozen times around. This does not NEED to be implemented as a device, not SHOULD it. It won't be. Externally OR internally. It WILL be a Library, though whether link-time or run-time (as in OpenLibrary, that is) I don't yet know. >[lots of stuff on exec() deleted] >>How to handle resource deallocation of the old process presents yet >>another difficulty. (oh, but of course!) >>allocated by the prior process need to be released. One solution (the >>cheap and easy way out) is to free malloc() allocated memory (if I >>write a malloc(), etc. set of routines) and leave other resources >>allocated, (such as memory gained from AllocMem) to allow resources to >>be passed to the called program. > This makes sense to me. If you need to pass the resource, you >are already Amiga specific, use the amiga specific call... Basically. >>initialization program ("Unix" bootstrap program) which would install >>the library in ram, start an "init" process, and then hang around as >>the "system task" to coordinate everything. > I called it "Execute.device" Wrong. Like I said, it will NOT be any "Execute.device". I understand the logic behind it, but it just isn't the way to do it. >>Someone... anyone... please reply. I wouldn't want this to be a >>wasted effort. (And more information/ideas is always helpful.) >> >>Enough for now. > I was going to send this as a reply (and some people may wish that >I had) but I changed my mind... It's ok by me; been having trouble with mailers to reply, and there may well be lurkers which are actually interested in this trash. :-) And those that don't care are welcome to hit 'n' or 'k'. Oh, and I meant I wanted someone, anyone to reply... either to me directly, OR to the net... Deven -- ------- shadow@pawl.rpi.edu ------- Deven Thomas Corzine --------------------- Cogito shadow@acm.rpi.edu 2346 15th Street Pi-Rho America ergo userfxb6@rpitsmts.bitnet Troy, NY 12180-2306 (518) 272-5847 sum... In the immortal words of Socrates: "I drank what?" ...I think.