Path: utzoo!utgpu!news-server.csri.toronto.edu!bonnie.concordia.ca!uunet!zaphod.mps.ohio-state.edu!casbah.acns.nwu.edu!hayes.ims.alaska.edu!raven.alaska.edu!milton!sumax!polari!rwing!seaeast!sunbrk!Usenet From: Orville.R..Weyrich@sunbrk.FidoNet.Org (Orville R. Weyrich) Newsgroups: comp.lang.c++ Subject: Summary: Restructuring and Reverse Engineering. Message-ID: <675397058.4@sunbrk.FidoNet> Date: 25 May 91 14:58:06 GMT Sender: Usenet@sunbrk.FidoNet.Org Lines: 363 Reply-To: orville%weyrich@uunet.uu.net Please e-mail responses to orville%weyrich@uunet.uu.net or else post to comp.software-eng (note the followup line). > My questions: > > I understand that there are several program restructuring tools > currently available which help turn old GO TO laden COBOL and FORTRAN > code into more structured code. > > What products are available now, and what (if any) research is > presently being done in this field? > > I have heard rumors to the effect that the present tools are less > than satisfactory. I would be interested in hearing of the experiences > (good and bad) of anyone who has used these tools. Bad experiences > are particularly useful, since they help clarify where improvements > need to be made. > > References to any articles on the subject would also be appreciated. > > What companies are currently using/developing program restructuring > tools that I might offer my services to as either a consultant or an > employee? Does anyone know of names of people withing these > organizations that I could contact? > > >"Reverse Engineering" can mean a lot of things to different people, including >the following: > 1) Support for extracting "business rules" from an existing program. > This becomes necessary when dealing with dusty-deck code > when the people in an organization that were responsible for > creating the program have disappeared, along with those > that knew how the business was run prior to the installation > of the program. Here, we are talking about "knowledge > acquisition" as is done in building expert systems, > but in this case, there are no human experts left, only a > bunch of undocumented source code (or worse!). > 2) Support for "Program Archeology" or "Design Recovery" on an > existing program. This is done to recover the program design > documents which have been lost (or never made). > This becomes necessary in order to maintain the code. > 3) Producing a "work-alike" product by working backwards from the > external behavior to figure out how to code a new application > which is a clone, but which does not use the original code. > An example would be the "NewWord" product which cloned the > "WordStar" word processor. This technique is under legal attack, > and is not what I have in mind. > >I am aware of rumors that companies such as ViaSoft and Transform Logic >Corporation are preparing products to support Reverse Engineering, but >would like any additional information which the net knows of. In particular, >I know that Transform Logic Corporation demonstrated at their "Transform User's >Group" last year a product under development which converts dirty old COBOL >decks into a hypertext format for browsing by the programmer. > >I suspect that many consulting/contract job shops have tool suites that they >use in-house. > >My questions: > > Does anyone know of companies interested in this kind of stuff that > I might offer my services to (as either a consultant or an employee?) > > What products are available now, and what (if any) research is > presently being done in this field? > > I have heard rumors to the effect that the present tools are less > than satisfactory. I would be interested in hearing of the experiences > (good and bad) of anyone who has used these tools. Bad experiences > are particularly useful, since they help clarify where improvements > need to be made. > > References to any articles on the subject would also be appreciated. > I am aware of the issue of IEEE Software on Reverse Engineering > (January 1990). > >Thanks, > >Orville. -------------------------------------- ****************************** Orville R. Weyrich, Jr., Ph.D. Certified Systems Professional Internet: orville%weyrich@uunet.uu.net Weyrich Computer Consulting Voice: (602) 391-0821 POB 5782, Scottsdale, AZ 85261 Fax: (602) 391-0023 (Yes! I'm available) -------------------------------------- ****************************** Thanks to all who replied to my postings regarding program restructuring and reverse engineering. I would be happy to get any additional comments and/or feedback on this information. Note that I also posted a question regarding program translation, the responses to which I will summarize later [when responses stop coming in]. ------------------------------------------------------------------------------- I SUMMARIZE BELOW COMMENTS BY: straub@cs.UMD.EDU (Pablo A. Straub) johnson@natasha.ics.hawaii.edu (Philip Johnson) lamson@el1.crd.ge.com (Scott Lamson) coxs@dg-rtp.dg.com (Stan Cox) hen@bucsf.bu.edu (Wm. H. Henneman) colbrook@concerto.lcs.mit.edu (Dr Adrian Colbrook) Keith.Bierman@Eng.Sun.com (Keith Bierman fpgroup) cassel@sce.carleton.ca (Ron Casselman) jls@netcom.com (Jim Showalter) hartman@cs.utexas.edu (John Hartman) Kevin.Lano@prg.oxford.ac.uk (Kevin Lano) oneel@heawk1.rosserv.gsfc.nasa.gov (Bruce Oneel) render@zeppo.Colorado.EDU (Hal Render) franklin@enuxha.eas.asu.edu (Malcolm L. Franklin) metcalf@crnvma.cern.ch (Mike Metcalf) dalamb@umiacs.UMD.EDU (David Lamb) dent@DIALix.oz.au (Andrew Dent) agnew@ast.saic.com (Robert Allen Agnew) paulg@tplrd.tpl.oz.au (Paul Gittings) nol2105@dsacg2.dsac.dla.mil (Robert E. Zabloudil) iwen2@c3po.d.umn.edu (ichang wen) alan@mel.dms.csiro.au (Alan Miller) map@cadillac.siemens.com (Michael Platoff) ------------------------------------------------------------------------------- MY CONCLUSIONS: I found no comments on successful reverse engineering products beyond simple cross-reference generators and flow-chart generators. Restructuring is widely done; the problems reported have to do with obfuscation of the code when a "forbidden" control structure (such as an exit in the middle of a loop) is translated into a much less obvious WHILE loop with less obvious nested if statements in the body. Other problems include ignoring the program semantics and data dimension of the program. Work continues at a number of research and commercial institutions in the area of restructuring and reverse engineering. ------------------------------------------------------------------------------- PUBLICATIONS TO LOOK AT: "From Code to Z Specifications" P. Breuer, K. Lano, in the 1989 Z User Workshop Proceedings: Springer-Verlag Workshops in Computer Science. "Reverse-Engineering and Validating COBOL" by K. Lano. Ward M., Calliss F.W., Munro M. "The Maintainer's Assistant", Proceedings of Conference on Software Maintenance 1989, Miami, Florida, IEEE Computer Society Press, Oct 1989. pp307-315. The Proceedings of the Conference on Software Maintenance (1986-1990) published by IEEE have various articles on reverse engineering and related topics. Look at a few articles in Girish Parikh's Tutorial on Software Maintenance (IEEE publication). Keep you eye open for the "Proceedings of the 13th International Conference on Software Engineering" that was held in Austin earlier this month [May 1991]. Names to search for in the literature are: Harry Sneed Mitchell D. Lubars Balasubramaniam Ramesh Ted J. Biggerstaff. Ted J. Biggerstaff, "Design Recovery for Maintenance and Reuse," IEEE Computer, Vol. 22, No. 7, July 1989, pp 36-50. "Extracting and Restructuring the Design of Large System " IEEE S.E. Jan 90 by Song C. Choi and Walt Scacchi in USC. "An Ada Restructuring Assistant", P. Johnson, D. Hildum, A. Kaplan, C. Kay, J. Wileden. In the Proceedings of the Fourth Annual Conference on Artificial Intelligence and Ada, Fairfax, VA. 1988. The Programmer's Apprentice work done at MIT. Look through SIGPLAN and Communications of the ACM mid-seventies. A big name in the was Amir Pnueli (sp?) from the University of Texas at Austin. John Hartman. Understanding Natural Programs Using Proper Decomposition. 13th International Conference On Software Engineering - ICSE-13, May 13-16, 1991. John Hartman. AUTOMATIC CONTROL UNDERSTANDING FOR NATURAL PROGRAMS. PhD thesis, The University of Texas at Austin, Dept. of Computer Sciences. May 1991. John Hartman. Pragmatic Program Understanding For Automated Re-engineering. In International Computer Software and Applications Conference - COMPSAC '90. IEEE Computer Society Press, October 31-November 2, 1990. Position paper for panel: ``Industrial Experience in Automating Software Re-engineering''. John Hartman. Automatic Program Understanding and the Maintenance Interface. In Second Annual User-System Interface Conference, Austin, Texas, February 20-21, 1987. John Hartman. Automatic Program Understanding For Cobol Restructuring. In Computer Science Research Review, Dept. of Computer Sciences, November 1986. The University of Texas at Austin. John Hartman. Restructuring Cobol Programs Into Abstract Data Type Modules. Technical Report SDBEG-21, Dept. of Computer Sciences, The University of Texas at Austin, 1980. Implementation is described in ``XTC User's Guide'', SDBEG-18. ------------------------------------------------------------------------------- PRODUCTS AVAILABLE: SPAG program marketed by Polyhedron Software. Try their toll-free fax no.: 1-800-777-5519. It is my understanding that OTG takes care of normal orders: OTG voice 717 222 9100 fax 717 229 9103 Alternatively, one may wish to go to the source Polyhedron Software Ltd Magdalen House 98 Abingdon Road Standlake Witney Oxon OX8 7RN Tel 0865 300 579 [See review below]. PRISM OTG Systems [same as above?] Mentor Graphics has a CASE tool called CASEStation. One of the support tools for this product will take "C" source and produce a structure chart. Support for CASEStation has recently moved to the UK. CADRE Technology, the makers of TEAMWORK, also have a similar product called C-REV. This again takes "C" source and produces a structure chart. CADRE unlike Mentor also have products to aid in the development of Cobol programs I don't know if they have any reverse engineering products for Cobol. RECODER tool from Language Technology, Inc. [See review below]. Toolpack has tools for restructuring code. [See review below]. FORGE Pacific Sierra Research analysis/restructurer/vectorizer/parallelizer FAUST from U of Illinois/U of Indiana dennis gannon Commercial tool is available from Bachman. Commercial tool is available from XA Systems. FORWARN/etc Quibus Enterprises Fortran Development Tools Quibus. Pretty sort of like SPAG. Seems less powerful. Outline: indentation creator with bonus graphic symbols to highlight logic. PREP : another conditional compilation, includization tool. Not exciting as such, but prices start at $500, so it is at least cheap. FOR_STRUCT from Cobalt-Blue. Like SPAG. Restructures Fortran particularly old Fortran (Fortran 66/ Fortran IV) into Fortran 77. The same company produces FOR_C which translates Fortran -> C. The company really pushes automated fortran->c|c++ translation which [some folks think] is a very, very bad idea. May be a great implementation though. 408 723 0474. ------------------------------------------------------------------------------- OTHER COMMENTS BELOW ------------------------------------------------------------------------------- [This was posted independently rather than in reply to my posting, but is relevant] Summary: Any experiences with Reverse Engineering report by Ovum Ltd.? Sender: news@siemens.siemens.com We're considering purchasing a research report from Ovum Ltd. entitled "Reverse Engineering: Markets Supplier and Products." The report is fairly expensive, so I'd like to make sure it's worth the money. I asked Ovum for references or excerpts from a section of the report, but they seem to have overlooked my request. Has anyone purchased an Ovum report? What did you think of it? In particular, has anyone read the Reverse Engineering report? Michael Platoff email: map@cadillac.siemens.com Siemens Corporate Research phone: (609) 734-3354 755 College Road East Princeton, NJ 08540-6668 ------------------------------------------------------------------------------- Although [Dent's] FORTRAN-Pascal translation currently performs minimal restructuring, there is much potential. This has come about through the extensive use of Object-oriented programming techniques and decomposition of programs into chained and nested objects. If possible, could you please append a note to your posting suggesting OOP can play an important part in writing restructuring tools, particularly if the programmer wants to stay away from languages such as LISP. It would be very interesting to see performance comparisons between a similar tool in some of the more exotic languages vs Object Pascal or C++. ------------------------------------------------------------------------------- As part of the REDO ESPRIT project, on the maintenance of systems, we are doing work on restructuring COBOL code by means of abstracting these codes to a functional language, and then performing transformations on this representation, to eliminate recursions that are not in WHILE-IF form. We are interested in applying machine-learning techniques to determine useful restructuring strategies, but not much work has yet been done by us on this. Tools that transform code to the functional language, and support the user in re-writing this code, have been developed in Prolog. Work at Durham University by Martin Ward also involves program transformation using formal methods. Their REFORM Project, to reverse-engineer assembly language code to Z, is described in his paper cited above. ------------------------------------------------------------------------------- There is a guy at MCC (Consortium in Austin TX) that is trying to apply neural network technology to reverse engineering; it's called the TAO subsystem of the DESIRE project. A lot of the stuff they publish is in MCC Proprietary publications (you gotta pay MEGA$ to get access to these). ------------------------------------------------------------------------------- I am currently working on a translation of MS-DOS FORTRAN programs to THINK Pascal on the Macintosh, for a local government department. As part of the job I've produced (well, nearly finished) a translator which will do most of the job, leaving only minor issues such as filename strings to be completed by hand. ------------------------------------------------------------------------------- We have developed these tools for Fortran and Cobol and are developing tools for Jovial. Contact John Hampton, 619-456-6017. ------------------------------------------------------------------------------- I've seen the Mentor Graphics tool called CASEStation and it worked rather well with the code we supplied, it could use some more work in the actual layout of the structured chart. ------------------------------------------------------------------------------- Our organization has the RECODER too * Origin: Seaeast - Fidonet<->Usenet Gateway - sunbrk (1:343/15.0)