Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!rutgers!apple!ames!pacbell!ditka!bucket!servio!penneyj
From: penneyj@servio.UUCP (D. Jason Penney)
Newsgroups: comp.unix.questions
Subject: Detecting Presence of Debugger
Summary: How Can a Process Tell if it is being Debugged?
Keywords: debugger,SIGTRAP,process-control,ptrace
Message-ID: <130@servio.UUCP>
Date: 24 May 89 16:37:48 GMT
Reply-To: penneyj@servio.UUCP (D. Jason Penney)
Organization: Servio Logic Development Corp.; Beaverton, OR
Lines: 29

We have a host-independent function HostCallDebugger, which is supposed
to synchronously invoke the host debugger (typically dbx in Unix systems).
If the debugger is not present, it generates a core, which is reasonable.

It is implemented by issuing a SIGTRAP (SIGTERM on Sun-3's).  If dbx is
present, you get an "Unexpected SIGTRAP" message, which you can subsequently
continue from.

If dbx is not present, the default core-producing behaviour is invoked.

Now, I saw some discussion a little while back on renaming your core files.
I've incorporated this into our general-purpose fault handler with good
results.

The problem is, if HostCallDebugger is invoked when no debugger is present
(such as a program assert failure), core gets dumped without any renaming.
It's not acceptable to always generate a core when calling HostCallDebugger
-- what if the program is being debugged?

What I would like is a way for a process to query whether or not it's being
debugged.  If it is, I'll just call SIGTRAP.  Otherwise I'll print a
nice message and invoke the smart coredump routine.

Thanks in advance.
-- 
D. Jason Penney                  Ph: (503) 629-8383
Beaverton, OR 97006              uucp: ...ogccse!servio!penneyj
STANDARD DISCLAIMER:  Should I or my opinions be caught or killed, the
company will disavow any knowledge of my actions...