Path: utzoo!attcan!uunet!datapg!sewilco From: sewilco@datapg.MN.ORG (Scot E Wilcoxon) Newsgroups: comp.sources.d Subject: Re: Call for discussion: comp.sources.archives Message-ID: <1601@datapg.MN.ORG> Date: 26 Aug 88 07:12:50 GMT References: <222@pigs.UUCP> <6031@orstcs.CS.ORST.EDU> <236@pigs.UUCP> <478@c10sd1.StPaul.NCR> <362@pigs.UUCP> <389@snjsn1.SJ.ATE.SLB.COM> <12292@ncoast.UUCP> <480@utkcs2.cs.utk.edu> Reply-To: sewilco@datapg.MN.ORG (Scot E Wilcoxon) Followup-To: comp.sources.d Organization: Data Progress, Minneapolis, MN Lines: 96 In article <480@utkcs2.cs.utk.edu> moore@utkcs2.cs.utk.edu (Keith Moore) writes: ... >If this newsgroup is set up, (and I think it's a good idea), I'd like to The recommended method is to just start posting in an almost-right place to build up the volume. Then request to get kicked out to our own newsgroup. >see submissions put into some simple kind of format that could be read >by computers as well as humans. I've been regionally posting entries of this format (head and tail, omitting part of too-long list of directories, and access info removed to avoid getting requests from the universe before I'm ready): Subject: archive list - datapg - Sun Aug 21 21:09:27 CDT 1988 Newsgroups: mn.general Distribution: mn Organization: Data Progress, Minneapolis Archive-Name: index.datapg Site: datapg Support: Automatic server, automatic indexing Latency: 0.04 days Service: # this contains an example of requesting something Usage: # this shows how to get more detailed usage instructions Contact: sewilco@DataPg.MN.ORG ===== datapg summary NETLIB index ===== Sun Aug 21 21:09:29 CDT 1988 ----summary of subdirectories------ bench doc graphics misc news news/alt news/alt/sources news/comp news/comp/ai ... -------summary of contents--------- Archive-Name Pathname Message-ID =========Subject=============================================================== =============================================================================== tools/2dpipe.art <188@metro.oz> 2dpipe - implement 2 dimensional sh(1) pipes news/unix-pc/sources/38 <828@hsi.UUCP> 3b1plot - plot(1) filter for 3b1 news/comp/sys/att/161 <1133@cooper.cooper.EDU> Analysis & test for 3b inode problem: applies to ALL users of SYSTEM V vplot/part23 news/comp/sources/unix/166 <599@fig.bbn.com> v14i028: Device-independant graphics system, with drivers vplot/part24 news/comp/sources/unix/164 <600@fig.bbn.com> v14i029: Device-independant graphics system, with drivers vtree news/comp/sources/unix/252 <766@fig.bbn.com> v15i005: Visual display of directory tree whichtape news/comp/sources/unix/271 <864@fig.bbn.com> v15i023: Tools to help find files on backup tapes window-srch text/window-srch <972@fig.bbn.com> v15i082: Windowing search (not unlike context grep) xenix-fuser news/comp/sources/misc/271 <8807130006.AA13998@rpp386.UUCP> v03i090: fuser for 386 xenix (+ repost of Unix PC version) > The news articles could then automatically >be fed into a database update program at sites that wished to track this >information. ... Two-line entries used for human readibility. Archive-Name: First word, if first char of first line non-whitespace. Pathname: Field after first whitespace on first line. Message-ID: Field after second whitespace on first line. Subject: Field after first whitespace on second line. Can be machine processed with the above definition. Message-ID is given specifically to simplify comparison and merging with index from other sites. I've been maintaining the index by having at least "Message-ID" and "Subject" lines in a header of each archived file. I then use a modified version of "search" (from net.sources years ago) with some scripts which create the index. The scripts could easily be modified to also report files without a USENET header. I will be posting these tools within a few weeks, as we're shaking them out locally. I do not have a program to extract the indexes from news, merge and update an index database, nor search and issue requests (help in issuing requests would be nice, as requests might be made by uucp, mail, or ftp). Those are some of the tools which are also being mentioned in this discussion. Anyone have pieces of these? (Don't forget to remove old information from the index DB) -- Scot E. Wilcoxon sewilco@DataPg.MN.ORG {amdahl|hpda}!bungia!datapg!sewilco Data Progress UNIX masts & rigging +1 612-825-2607 uunet!datapg!sewilco