Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!dimacs.rutgers.edu!mips!news.cs.indiana.edu!ux1.cso.uiuc.edu!midway!ellis.uchicago.edu!goer
From: goer@ellis.uchicago.edu (Richard L. Goerwitz)
Newsgroups: comp.unix.questions
Subject: Re: Text Processing Question
Message-ID: <1991Mar18.013647.7570@midway.uchicago.edu>
Date: 18 Mar 91 01:36:47 GMT
References: <31134@usc>
Sender: goer@midway.uchicago.edu (Richard L. Goerwitz)
Distribution: usa
Organization: University of Chicago
Lines: 17

In article <31134@usc> rkumar@buddha.usc.edu (C.P. Ravikumar) writes:

>I was wondering if there is a utility to check
>for repitition of words in a document....
>
>I have the feeling this can be done using "awk".

The hard part, as always, is settling on a field separator -

BEGIN	{ FS = "['.]*[^0-9A-Za-z-']+" }
{    for (i = 1; i < NF; i++) {
         if ($i == $(i+1))
             print NR ":  " $0
     }
}

-Richard