Path: utzoo!utgpu!water!watmath!clyde!att-cb!att-ih!pacbell!ames!pasteur!ucbvax!hplabs!sdcrdcf!trwrb!desint!geoff
From: geoff@desint.UUCP (Geoff Kuenning)
Newsgroups: comp.sources.d
Subject: Re: Wiping out /bin in OS upgrades
Message-ID: <1704@desint.UUCP>
Date: 2 Apr 88 08:01:21 GMT
References: <1703@desint.UUCP> <756@stride.Stride.COM> <47576@sun.uucp>
Reply-To: geoff@desint.UUCP (Geoff Kuenning)
Organization: Interrupt Technology Corp., Manhattan Beach, CA
Lines: 162

Kudos to Chuq for keeping this discussion rational.  I'll try my best
to reciprocate.

In article <47576@sun.uucp> chuq@sun.UUCP (Chuq Von Rospach) writes:

> If you look at the shell script I posted, one of the files I modify is
> /usr/ucb/quota. Now, I'm going to upgrade my system. How, without telling
> the upgrade process what that change is for, does the upgrade process do
> the right thing?
> 
> 	o You can tell the file is changed because the checksum is different,
> 	right? But how much information about checksums do we keep? The
> 	previous release? What if I'm upgrading from 3.2 directly to 3.5?
> 	Or for 3.0? 2.2? Should the upgrade notice that the version of
> 	/usr/ucb/quota isn't from ANY sun tape?

Obviously, the user would ideally like near-perfection;  equally
obviously, such is rather expensive.  It's pretty easy for Sun to keep
checksum files from all releases in-house, of course.  At Callan, our
support people kept paper copies in filing cabinets, and often used
them during phone support calls, to verify that the user had a
particular (and uncorrupted) version of a troublesome utility.

Putting multiple checksum files on an upgrade kit is a little more
bothersome, especially if the kit is floppy-based.  Checksums for 5000
files take quite a bit of disk space.  At Callan, we took the easy way
out, and insisted that customers not skip any upgrades.  If you wanted
the "naive" upgrade from X.0 to X.2, you had to install X.1 first.
This was especially important because our floppy-based upgrades were
incremental, so there was no guarantee that X.2 had the /bin/foo
upgrade that was included in X.1.

I think the trick is how you define your "supported" customer base.  If you
want to define them as "Unix gurus", you don't have to hold hands at all.
If you want to define them as anyone who ever bought a Sun machine from
anybody, regardless of the current state, you've got an AI problem
worthy of at least a thesis, if not an endowed chair.  Nobody picks
these extremes, of course.  Sun picked "intelligent system managers who
have the foresight to keep records."  Callan picked "anyone who bought
a system, modified 'reasonable' system management-related files, and
followed our upgrades when they came out."

(I should also mention that, at Callan, we did *not* try to be "good"
about every single file on the system.  For example, I run with private
versions of cron, crontab, tellcron, and at.  I wouldn't expect any
upgrade process to respect my changes to those files;  it's not
reasonable for an "average naive" user to change them.  As an allegedly
expert user, I consider it my own responsibility to take care of such
things.)

> 	o What about patches? Suppose I'd modified quota because the new 
> 	version fixes a bug? (for the record, this isn't the case....) And
> 	say that bug is fixed in the release I'm installing? Shouldn't it
> 	ignore the fact that the binary is patched and install the new
> 	standard version? How do you tell this?

In my view, this depends somewhat on where the patch came from.  See my cron
example above for one alternative, when only "expert" users would be patching
things.  However, the simple answer for the general case is to output the
following warning, and also mail it to root:

	WARNING:  /usr/ucb/quota is not the X.Y version.  For safety, it
	has been preserved in /usr/ucb/quota.old, and replaced by the
	latest version from the update tape.  You should investigate and
	decide which version is the one you want to run, and remove the other.

Obviously, you could also leave the old version alone, and put the new
one in /usr/ucb/quota.new.  The theory is that, if the user is expert
enough to patch a program, he's also expert enough to decide whether he
wants to go with an upgrade of it.  (Also, you should watch out for disk
overflows if very much of this happens.  Disk overflow is a knotty problem;
about all you can do is spawn a shell for the guy to use to clean up.  The
Callan script checked at the beginning to be sure you had the amount of disk
a "standard" upgrade would need to add new features, but it's not so
easy to predict at startup time exactly how much extra you'll need to
keep new/old copies of things the user has updated.)

> 	o Local changes? I've modified quota to do something different than
> 	the standard release. How does the system recognize this as a
> 	legitimate change that it should NOT override? And how does it tell
> 	that this is really a legitimate change and not some trojan horse
> 	that someone ELSE put into your system for you?

For legitimate changes, the new/old procedure given above should suffice.
For ASCII files such as /etc/passwd, the documentation should point the user
at useful integration tools such as /usr/bin/sdiff, rcs merge, or Larry Wall's
patch program.

For Trojan horses, the new/old procedure will call root's attention to
the possibility of such a critter.  Perhaps it would also be a good idea
to provide an option like cpio's "-u", which says "I don't know or care how
screwed-up this system used to be, just install the latest stuff and make
it clean again."

> 	o And what if there's significant new functionality in our version
> 	of the binary that doesn't get overridden because your private patch
> 	means we don't install over it. Let's even say that the program
> 	involved is, say, /bin/login. Now you can't log in, because your 
> 	old private version isn't compatible with our new system, and we
> 	didn't install our copy of the new, improved program.

Again, see the new/old procedure above.  In an important incompatibility
case like login, obviously you should move the user's version to ".old"
so you can install a version you're sure is compatible.  The user can
clean up later if that's wrong.

Also, I would hope that there are bootable floppies or similar procedures
which allow non-destructive recovery of severely damaged disks.

Finally, on System V, it is a one-character (and thus easily reversible)
change to /etc/inittab to cause a system to boot into single-user mode.
If you had /bin/login inconsistencies, the upgrade procedure could do this
edit, thus making sure that the system can be worked on even if login is
broken.  The user can easily put it back later.  (Of course, this wouldn't
be of much use if the program in question were /bin/sh instead!)

> Private changes in the system directories aren't easy. There are so many
> possibilities and the ways of dealing with them are exceptionally complex.
> And there are no really clean answers that don't involve manual intervention
> anyway. There's no way to install on a modified system directory structure
> and do it cleanly without adding a lot of complexity to the install system
> AND requiring a knowledgable installer. 

The trick is to separate things into categories.  The files listed in your
script (crontab, passwd, rc.local, etc.) are ones you COUNT on the user to
muck with.  And they *have* to be in system directories.  So your upgrade
script should deal with them in a clean fashion.  The few people who change
/bin/login are willing to re-install it after an upgrade, especially because
(a) they've got source and makefiles handy, and (b) you helpfully sent
them a mail message reminding them that they need to do so.

As to the manual intervention issue, you're entirely correct:  manual
intervention in such cases is necessary to achieve a clean answer.  The
question is whether the upgrade process makes manual intervention a
difficult process involving foresight, or an easy one involving reading
your mail and typing "make install" a few times.

> Rather than building all the complexity (and likely a few bugs) into the
> install program, especially when it won't really solve the problem
> completely, doesn't make sense. In my eyes makes more sense to have the
> administrator keep track of what they've changed and re-install them after
> they upgrade. This gives them a chance to figure out whether the changes
> they have they really still want (or need) and deal with changes in
> functionality or filesystem structures. And if you keep documentation and
> copies of all the changes in the system trees, you no longer have to back up
> those file systems, saving tapes and time -- and you no longer have ulcers
> when disks go down. You re-install, and restore your changes.

To me, this is a question of whether Sun or the customer does the work.
*Somebody* has to keep track of what's in /usr/lib/crontab.  Anything Sun
does, is done once.  Anything the customer does, has to happen <however
many machines Sun has sold to date> times.

I hope this helps.  I strongly feel that it *is* possible to write an
upgrade procedure that holds the average customer's hand without
forcing him to be an "ideal" system manager (remember, some of these
people are coming from the DOS world).  It also wouldn't hurt for Sun
to ship Chuq's script with every machine, and to document his system
management suggestions prominently in the basic-installation manual
(if they haven't done this already),  because they *are* good suggestions.
-- 
	Geoff Kuenning   geoff@ITcorp.com   {uunet,trwrb}!desint!geoff