Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!ncar!gatech!dscatl!nanovx!msa3b!kevin From: kevin@msa3b.UUCP (Kevin P. Kleinfelter) Newsgroups: comp.sys.ibm.pc.rt,comp.unix.i386 Subject: SOLUTION to AIX Modem Wierdness Keywords: it works Message-ID: <677@msa3b.UUCP> Date: 20 Jul 89 19:34:21 GMT Followup-To: comp.unix.i386 Organization: Management Science America, Inc., Atlanta, GA Lines: 178 Recently I queried the net for assistance with AIX PS/2 not handling modem control lines correctly. Several people said that they had the same problem, and they had found no solution. In documenting the problem for IBM, I found a work-around. To summarize the problems: Problem 1: A 'fopen ("/dev/tty0", "r");' would block if CD were low (as expected), BUT IT WOULD NOT CONTINUE WHEN CD went high. Problem2: If you get past problem 1, dropping CD would NOT cause the HANGUP trap to be executed until the NEXT attempt to open the tty (i.e. You still had a shell running after the line was dropped). Both problems are solvable! The bug is in the "devices" program. Running "devices" and selecting "dvam = 1" will create a minor device number of 96. The asy driver documents this minor device number to mean "tty (not a printer) with modem control on SERIAL_1". However the correct number should be 64. The 96 is the correct 64 with an additional, undefined bit set. DO NOT CREATE tty DEVICES WITH THE "devices" COMMAND! DO IT BY HAND! The following is the text of a fax I just sent to IBM on our APAR. ################################################################ We have been experiencing trouble with the way AIX handles modem control on the motherboard serial port. To summarize: If a process attempts to open "/dev/tty0", the process will sleep forever if CD is not active BEFORE the open. The expected behavior is that the process would block until CD goes active, at which time it would continue. This causes a problem because we plan to have clients dial-in to a PS/2 model 80, running AIX. The only way to prevent getty from sleeping forever on a port is to strap CD active, so that getty only sees CD as true. If CD is always true, then, if a client is disconnected due to line noise, the next client to dial-in on that phone line will get the previous client's process, and not a new login. The following will reproduce APAR # IX04764: Take a PS/2 Model 80/386 with 8192KB of memory and a 300MB ESDI disk. Install AIX (Base operating system) using default minidisks. Update Base AIX. Install DOS Merge C Language Graphics Support Application Development Toolkit Asynch. Terminal Emulation O/S Extensions Workstation Host Interface Prog. Update all installed programs. Attach one end of a straight, 25-pin ribbon cable to the motherboard serial port (configured as COM1). Attach the other end to a breakout box (the connector labelled "To DTE"). Attach the other end of the breakout box to NOTHING. Issue the following command: mknod /dev/tty0 c 5 64 (Note that 5 was chosen as the major device number because the default "/etc/master" defines the major device number for the "sa" driver to be 5. 64 was chosen as the minor device number directly from the "asy" documentation in Tech Ref. vol 2, page 5-3 (manual #SC23-2033-0). Strap DSR, CTS, and CD off (I strapped them to TD, since it is low at this time). Compile and run the following sample program (as root):: #include void main (void) { fprintf (stderr, "program begins\n"); open ("/dev/tty0", O_RDWR, 0); fprintf (stderr, "program ends\n"); } The PS/2 displays "program begins" and asserts RTS and DTR. Strap DSR, CTS, and CD on (I strapped them to RTS, since it is active at this time). The program then displays "program ends". This is as expected. Repeat the test (strap low, run program, strap high). The program continues to work as expected. Now rm /dev/tty0 Run the devices command, add ttydev tty, with the following: tt vt100 dvam 1 bpc 8 pt none rts 19200 ae false nosb 1 elevel 1,4 ixp false xscan false aa false pro dc The result includes the creation of "/dev/tty0" with a major device number of 5, and a minor device number of 96. The 96 is the expected 64, with an additional, undocumented bit set. Strap DSR/CTS/CD low. Run the test program -- it prints "program begins". Strap DSR/CTS/CD high. The program does NOT proceed. Interrupt the program via the Break key. Leave the pins strapped high. Run the test program -- it runs to completion. Strap the pins low again. Run the test program -- it does not complete. Interrupt via the Break key. CONCLUSION 1: The devices command creates /dev/tty0 with a bad minor device number of 96, when 64 is desired. ########################################################## "rm /dev/tty0". "mknod /dev/tty0 c 5 64". Run the test program -- it prints "program begins". Strap the pins high -- the program does not complete. Interrupt via the break key. Leave the pins high. Run the test program -- it DOES complete. CONCLUSION 2: After the minor device number is set to 96, the device driver is still "broken" when the device number is set back to 64. ########################################################## "rm /dev/tty0". "mknod /dev/tty0 c 5 96". Strap DSR/CTS/CD to RTS. Run the test program -- it completes. "rm /dev/tty0". "mknod /dev/tty0 c 5 64". Strap DSR/CTS/CD low. Run the test program -- it prints "program begins". Strap the pins high -- the program completes. CONCLUSION 3: To clear the "broken" device driver, you must SUCCESSFULLY open /dev/tty0 with minor device of 96. ########################################################## SUGGESTED ACTION: Fix "devices" command to create /dev/tty0 for com1 with a minor device number of 64. -- Kevin Kleinfelter @ Management Science America, Inc (404) 239-2347 gatech!nanovx!msa3b!kevin