Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!rutgers!aramis.rutgers.edu!webber
From: webber@aramis.rutgers.edu (Bob Webber)
Newsgroups: news.groups,news.misc,news.software.b
Subject: How long would it take to actually read all of news?
Message-ID: <1475@aramis.rutgers.edu>
Date: Thu, 10-Sep-87 05:27:12 EDT
Article-I.D.: aramis.1475
Posted: Thu Sep 10 05:27:12 1987
Date-Received: Sat, 12-Sep-87 06:39:44 EDT
Organization: Rutgers Univ., New Brunswick, N.J.
Lines: 85
Xref: mnetor news.groups:1498 news.misc:908 news.software.b:802
To: pleasant@klinzhai.rutgers.edu, amos%nsta@nsc.com, webber


[In this message I present stats on reading 40 hours worth of news and
then discuss software to cut 20% off of read time.]

Many people seem to think that Usenet has grown too big to read.  I
disagree.  So, I let 40 hours of news pile up unread and then sat down
and timed myself reading it.

I subscribed to all 415 news groups available locally rather than my
usual 401 to eliminate the empact of not reading os.vms on the results.
I made a typescript of the entire session.  I read it on a Sun III
where my own files were local but news was NFS mounted.  Although I
was subscribed to 415 groups, new messages had only occurred in 201 of
them over the 40 hour period.

At 20:57 I initiated vnews -A.  I did not read any of the messages but
rather used the header information (subject and author and length) as
a basis for determining whether or not I wanted to look at the message
later.  The messages I did want to look at later, I saved into
Articles (the default save file for vnews).  At 21:52, after scanning
the headers on 2132 messages, I was finally able to type vnews -A and
be told that there was no new news available.

I then proceeded to invoke gnu emacs over the file Articles and use
control-V and control-S to find my way through the Articles.  There
were 386 articles in Articles, including all the digests (since the
Subject line on a digest doesn't tell you if there is anything
interesting inside -- sigh).  The entire Articles file was 927,274 bytes.
I was done reading it at 23:25.

In summary, it took an hour to find the articles I actually wanted to
look at and an hour and a half to look at them (saving out a couple of
ones that I wanted to refer to later and making notes as to wish ones
called for a reply (sent later through mail and postnews as appropriate)).
I made a script of the entire session allowing the gathering of the
above statistics after the session.  The script was 1,249,028 bytes.

What do I conclude from this:
   1) It takes 3 hours to read 2 days worth of news when one ignores
      the news groups boundaries.
   2) It is alot faster to separate the scanning and reading phases
      of news.
   3) Current news software doesn't support 2.  Although emacs handled
      the read phase quite well, vnews -A was awkward for building the
      Articles file.  Specifically:
      a) Slowness of saves;
      b) Awkwardness of typing sequence ``s CR n'' occassionally out
         of order.  Would be nice to be able to just type one key and
         get the save in default file and go to next message action.
      c) Slowness of transitions between news groups and skipping over
         groups with no messages.

Proposed solution:
   Run 3 processes.  The first process builds a list of all message
headers, piping them to second process.  Second process shows headers
to user, a screenful at a time (not a screen/group at a time) and then
accepts save/ignore requests.  The second process forwards save
requests to a separate process that handles the actual append to
Articles operation.  Also, the news message number is saved to aid
interaction with postnews when appropriate.

[Note: readnews -c mail -f % looks plausible, but somehow I think 4
meg tmp files are worse than 1 meg Articles files.  Also, it doesn't
do the discussion grouping that is convenient in vnews -A (although
somewhat messed up by news group boundaries when discussion spread off
a cross posted article).]

Cheap solution:
   Use vnews -A for first process feeding it a sequence of ``n's'' and
feeding output on to second process.  Leave parsing of file name for
save request to background process 3.  Forward news group transitions to
background process 3 as well.  Allow option of saving message in a
named file to handle sources and RFC-type postings.

Expected result: 
   20% reduction in total time spent to read news.

Make sense?  Is there a simpler, more efficient way to put it
together?  Would anyone be interested in seeing the result?

[Note: this solution doesn't support all the frills of normal vnews
such as figuring reply addresses automatically - which don't seem to
me to actually be important from a point of view of how long one
spends with news.]

------ BOB (webber@aramis.rutgers.edu ; rutgers!aramis.rutgers.edu!webber)