Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!munnari.oz.au!manuel!cmf851
From: cmf851@anu.oz.au (Albert Langer)
Newsgroups: comp.society.development
Subject: Re: Low-cost Usenet (Re: usenet in Nepal)
Message-ID: <1991Jun21.231902.24754@newshost.anu.edu.au>
Date: 21 Jun 91 23:19:02 GMT
References: <1991Jun14.100804.4867@iccgcc.decnet.ab.com> <1991Jun15.023819.5589@newshost.anu.edu.au> <1991Jun18.065605.6955@zorch.SF-Bay.ORG>
Sender: news@newshost.anu.edu.au
Organization: Computer Services Centre, Australian National University, Canberra, Australia.
Lines: 330

In article <1991Jun18.065605.6955@zorch.SF-Bay.ORG> xanthian@zorch.SF-Bay.ORG (Kent Paul Dolan) writes:

>Probably be a useful idea to start off by defining
>some "do and don't do" rules (or "motherhood and
>apple pie" baselines) for the vaporware plug and
>play Usenet system under consideration.

Kent, thanks for kicking this along. I'm going to go through your
points with a fine tooth comb, mainly raising objections, but
thanks for providing something to comb. My main point is that we
should start from requirements and carefully distinguish those from
design decisions (although some requirements may be policies or rules
to follow in making design decisions).

>1) Not MS-DOS. Why? I spent a mercifully brief nine
>months working on a ten month MS-DOS project to
>upgrade an existing commercial software package with
>much more functionality to "beat the competition";
>it's delivery was just announced, three YEARS late.
>
>Every problem _I_ saw was due to overstressing an
>architecture and an operating system already pushed
>far past its limits.  Cheap as the hardware and OS
>are, building on an MS-DOS base is a recipie for
>failure or immediate obsolescence.

Depends what we are building. If the implied architecture
is based on multi-user systems providing a service to
a number of simultaneous dial-up online users then
MSDOS may be inappropriate (though a number of BBSes
do in fact use it).

But with phone lines the dominant cost, and a requirement for
cheap connection and use, surely an architecture for developing
countries should focus even more than in the developed world
on making full use of the users own PCs.

I have in mind (for use by community groups in developed
countries as well) an architecture where each user has
a complete MTA and some users provide relay services for
other users. e.g. An organization in some town can simply
leave their office computer on all night and local users can automatically
deliver mail to it before the network schedule for
forwarding to the next relay and can then call back again
later to pickup mail from it after the network schedule
for picking up mail from the next relay. That is the model
FidoNet uses for overnight mail exchange among about 10,000
BBSes and it is extended to individual "point" users that
run essentially the same network software but have no
dial-up dumb terminal callers.

FidoNet BBS sysops put as much time into operations as unix
sysadmins but on average have less skills and the
extension already occurring to "point" operators suggest
the sysop functions can be eliminated. (With a GREAT
deal of design effort).

Such an architecture would greatly reduce costs
(if it is feasible to overcome reliability problems
through redundancy etc and to design the needed
network management). It should not be eliminated from
consideration in favour of multiple dial-up users
without careful analysis.

It would certainly require software to be running
on MSDOS boxes and also (less urgently in developing
countries) on Macs. The problems with MSDOS may
just HAVE to be overcome to meet other requirements.

>Something like SCO Unix on a fairly cheap 80386 box
>would probably be fairly effective.  AmigaUX is a
>SYSVr4 Unix, but a bit pricy compared to the Intel
>boxes, but it might be worth considering too, since
>the software would all be PD after the initial
>purchase.

If we do need a unix-like system then Xenix, Coherent
and Minix should also be considered. One model might
be to switch operating systems at night for network
operations. Very likely unix will be desirable at
gateways and for additional functions like real-time
chat and database lookup. Also unix may be the PC
operating system of the future and certainly
unix is the best development platform even if the
development is for DOS. So unix should be kept in mind.

Possible Design Rule: All software developed should
be as generic and POSIX compliant as possible with
clear separation of platform specific modules from
generic modules.

>2) Preemptive multitasking -- so the operator can be
>using the system, while the filing software is
>working at sorting out news/mail, while at least
>one user is logged in.

Desirable, but not a requirement if the user logged
in is at the console. For developing countries,
overnight mail may be quite acceptable with no need
to do any sorting out or other network activities
during times when the user needs the computer. Also
it may be possible to do background network transfers
with only very minimal multi-tasking that is oriented
towards a fancy comms driver like XPC or Mirror
rather than providing a complete multi-tasking operating
system. (After all PCs on LANs do their networking in 
the background while still running what looks to the
user like plain ordinary DOS).

Certainly pre-emptive multi-tasking does not mandate
a shift from the DOS world to unix (though it is
easier under unix). Many BBSes use Desqview and
Windows 3 might also be a possibility.

>3) Diligient, highly directive error reporting; not
>just "error ### occurred", but "The last transfer
>was halted due to lack of spool partition space;
>your current expire is 3 days, run expire with a two
>day parameter (do a 'man expire' to see how) to get
>more room, or look for suspiciously overlarge or
>over-age files using 'find' (see 'man find')."

Again implies a local "operator" but just less skilled
(or less tolerant of brain damaged software) than at
present. I'm not sure the REAL requirements can be
met without eliminating "operator" functions ENTIRELY
- which means sending error messages to network
management instead of expecting anybody trying to
USE the computer to deal with them (which of course
implies having a lot less error messages to deal
with).

>4) A complete online glossary and help/manual set:
>"what's a spool partition" should be an easy
>question to answer.

Nope. Why should a doctor or librarian who wants to use
their word processor to communicate with others want to
know what a "spool partition" is?

The sort of stuff they should see would be along the lines
of:

"The disk is filling up too quickly. Select 1, 2 or 3.

	1. Reduce the period for which old messages are kept.

Or, to postpone the problem without solving it.

	2. Select some messages to delete now.
	3. Move some messages to archive diskettes."

Requirement: Error messages should be reported only to network
management. System messages visible to the user should walk
the user through the procedure required, including insertion
and removal of diskettes etc, and should be expressed in
terms of concepts the user is necessarily familiar with.

>5) Brilliant error recovery from all the current
>problems; dropped user lines, dropped feed lines,
>overlarge file transfer, file system corruption,
>power loss, etc.

DEFINATELY. (But I wouldn't call it "brilliant".
Having operators deal with these things is simply
ridiculous.)

>6) A prioritizing system to transfer files in aged
>"smallest files first" fashion to encourage brevity.

That is a policy statement for brevity which may
have less impact than providing for reader selection
and prioritizing of articles to read with length
as a criteria. That could be added as a requirement
though it seems low priority to me.

The X400 prioritizing of urgent normal and non-urgent
items seems adequate for all transfer purposes. It
would allow normal mail to go through while "files"
are delayed by lack of disk space at intermediate
relays and would allow system messages and other
priority traffic to get through while capacity is
strained beyond normal limits. This eliminates a
substantial part of the need for sysadmins.

>7) Backing store for files await transmission; this
>may (probably will) require operator intervention,
>but we shouldn't be afraid to make the system labor
>intensive to save costs; we have the example of
>amateur radio operators to show us how hard
>hobbiests are willing to work, we just need to make
>the system not be knowledge intensive, so that those
>diligent but unskilled can just follow directions to
>make it work.

I can see the point that unskilled labor may be 
cheaply available for taking diskettes in and out
even though operator labor is unavailable, but I
doubt that there is much scope for making use of
this and certainly can't see it helping with a
queue of files waiting for transmission. (The
files should just stay on the machine they were
produced on or that they have been relayed to
until there is room to send them further. With
priority queues etc that just results in users
being told they can't send the file, not in
anything clogging up. Users told they can't
send a file can copy it off their hard disks
if they really want to, but they are more likely
to get bigger disks if it happens regularly,
and network capacity has to be increased if it is
a problem significant enough to be worth designing
a manual backing system store for.)

Possible uses for cheap labor might be archiving
old messages to diskette (though I suspect tape
will soon work out cheaper because of the media
costs anyway and tape is not labor intensive).

Another one is carrying news and files by
diskette or tape instead of phone.

>8) Robust and commonly available components; we
>should _expect_ phone line quality problems, power
>brownouts, lightning storms, etc. Since repairs
>will be a problem, select stuff that doesn't break
>often, and where local spares are an affordable
>option.

Yep. Implies a strong preference for commodity
PC hardware used locally for ordinary wordprocessing etc.

Another reason to try adding ONLY the modem to what
users already have.

>9) Redundancy; use two lesser grade $300 systems
>instead of one $600 system, so that graceful failure
>modes exist; do this even at the sacrifice of
>greater overall throughput from the monolithic
>system.

Yep. But only sites with a critical need for
continuous access to the network need duplicate
their own equipment (and they would be likely
to have multiple PCs anyway). The network itself
should simply be designed to assume that a
certain percentage, perhaps even quite a HIGH
percentage of nodes will be down at any given
time and re-route the traffic automatically
to provide the lowest cost with the
nodes that ARE available (AND to ensure nodes
that are down will get their traffic when they
come back up, which may be harder). Basically
this just means extra disk space apart from
the software design. No other extra hardware
at all. (Well, maybe ensure there is more
than 1 or 2 sites with high speed modems at
each end of an expensive and important international
link - though even there it is just a trade off with
the increased traffic costs from using other nodes
with lower speed modems).

Our requirements are not the same as corporate
networks.

>10) A _very_ simple editor, character and line
>oriented, that automatically avoids transmitting
>overlong lines, etc.

See my reply to Sue.

>11) Lots of optional ways to do things; hooks to use
>radio instead of phone instead of satellite; swap
>editors at the user level, locally present news as
>mail and vice versa, etc.

Careful. Some things just have to be optional, but
providing the network management will require a lot
of standardization - at least to the extent that any
option must include the remote logging and remote
configuration needed to manage it without a local
operator.

>12) Respect for local character sets; probably best
>to just start with a 16 bit or variable length
>character size; this implies that each document
>transmitted should have a header that identifies
>language and character set, so that we monolinguals
>can pick our favorite language out of a wider
>discussion. While I love English as a standard
>language for world discussion, and it is probably
>the most common second language in the world, it is
>still not reasonable to ask local conversations to
>occur in English.

X400 has given thorough consideration to character
set and language issues and has for very good reasons
standardized on ISO 6937 (T61) for normal text in
latin alphabet languages and ISO 2022 escapes to multi-
byte character sets for languages like Chinese etc. It
also provides for language labels. This is the best
solution for permitting use of multiple languages and
character sets in the same discussions among users with
different equipment each suited to their own language.
(Details of why would be a diversion best left to
a detailed design stage. Let's not get diverted into
re-inventing sheels that international standards
organizations have already invented.)

Internal character sets on the user's own equipment are
a separate issue for which Unicode etc may be relevant.

The basic requirement is that network character sets
should allow the full expression of each user's own
language with maximum possible interchange with users
that have equipment designed for other languages. This
permits multi-lingual conferences and the emergence of
new pidgin and creole languages through networking
between people that do not understand each other's language
fluently.

>13) Probably tons more; someone else take a turn.

Thanks for the opportunity. There is indeed tons more to come,
but let's try to keep it primarily at a "requirements" level
before taking design decisions.

--
Opinions disclaimed (Authoritative answer from opinion server)
Header reply address wrong. Use cmf851@csc2.anu.edu.au