Path: utzoo!mnetor!uunet!husc6!cmcl2!brl-adm!umd5!rutgers!rochester!rocksanne!z
From: z@rocksanne.UUCP (Jim Ziobro)
Newsgroups: news.software.notes
Subject: Re: Evolutionary dead end?
Message-ID: <484@rocksanne.UUCP>
Date: 2 Jan 88 11:45:44 GMT
References: <107300003@datacube> <276@fig.bbn.com>
Organization: Xerox: Webster Research Center, Rochester, NY
Lines: 80
Summary: Notes has technical advantages over news

I disagree that the notesfile file format is a dead-end.  For a
discussion format dialog it has some real technical advantages over the
B-news format method of storing articles.  On the other hand it is
slightly inferior when handling bulletin board type dialog.

Consider that for a discussion the most machine intensive tasks are the
article sorting to topics.  Ideally this is done once when the article
is posted and not each time a group is read like 'rn'.

In the notesfile format an article is not expired until there have not
been responses for a while.  In the B-news format after a certain
period the article is deleted whether it is important or not.  The
B-news expire could be fixed to operate in this desirable fashion but
at a significant cost in machine use.  If everyone had notesfiles there
would be no problem of including massive amounts of text when replying
to an article since it would be much more likely that the original text
would be at the reader's fingertips.

Considering that an open is much more expensive than a read the
notesfile format is much better suited for reading agents.  The posting
operation excluding article sorting is about as expensive in both the
notesfile and B-news file formats.  One of the ugly operations
necessary in any news system is the expire operation.  The B-news
format ideally would have the big advantage but let us look a bit
closer...

1.) To expire B-news articles a history file (and associated dbm file) is read
and written but to determine the expiration date each article must be opened
and read.  Thus:
bytes read = #articles x (small) + entire header text
bytes written = #articlse x (small)
file opens = #articles + 8
file deletes = #expired articles + 3

To expire notes the index files are opened in each notesfile then the text
file is copied selectively.  Thus:
bytes read = entire text database
bytes written = entire text database
file opens = #newsgroups x 6
file deletes = #newsgroups x 3

Thus B-news expire is proportional to the number of articles while the
notesfile expire is proportional to the number of newsgroups plus some
factor for the size of the text data.  In practice the reading/writing
overhead is not great since it tends to happen in large chunks.  On an
11/780 with massbus type disks the notes expire operation happened
considerably quicker than the B-news expire operation.

2.) If B-news expire happens to run just as you are replying to an
article and you are unlucky enough to be accessing an article on expire's
hit list then TOO BAD its gone!  The notesfile's expire first moves
the database to a temporary file thus once a user has opened the database
they have access until they are done.

In summary:

	Notes has a relatively high cost of article insertion but that
	is because it is also usefully sorting the article.

	Reading agents can operate very efficiently using the notesfile
	format considering that open calls are generally expensive.

	In the current implementation (under Unix) the expire operation
	for notesfile is much quicker than the B-news format.

	The high article insertion cost means notesfile is most useful
	where there is some correlation between the articles.  Bulletin
	board type data where any particular article likely goes to
	many groups is poorly suited to notes.


The NYU version of the notes system is well tuned and debugged for
4BSD+ Vaxen.  If you are not running the NYU notesfile and you have a
BSD derived system then you should be.  The NYU version is
characterized by the presence of the "universal sequencer key" (l) and
occurrences of the name "salkind" in the source.  It is ARPA accessible
from NYU.EDU.

//Z\\
Jim Ziobro