Newsgroups: news.admin
Path: utzoo!henry
From: henry@utzoo.uucp (Henry Spencer)
Subject: verification of Brad's vote
Message-ID: <1989Mar20.181533.22112@utzoo.uucp>
Organization: U of Toronto Zoology
Date: Mon, 20 Mar 89 18:15:33 GMT

Some skeptics expressed some doubts about the strikingly one-sided results
that Brad Templeton announced on his opinion survey about the notion of
gatewaying rec.humor.funny to and from commercial networks.  He asked me
to spot-check his survey results.  He sent me everything he'd received
up to the afternoon of March 14.  (Brad informs me that ballots continue
to trickle in, with the overall picture unchanged, but I have not verified
this myself.)

The raw results, counted by my software (not Brad's), are 474 yes, 32 no,
4 "spoiled" ballots (all really "yes" votes, but with the "> survey no"
line included from Brad's posting), and 7 blank.  My software takes care
to ignore multiple votes per ballot, unless they disagree, in which case
it designates the ballot "spoiled".

I inspected the data by eye.  I found no evidence of systematic generation
of addresses or other monkey business, but there were a few duplications.
(Brad had mentioned this; he hadn't bothered to weed them out because there
were too few to affect the result significantly.)  I did a systematic search
for duplicates, and deleted 7 duplicate ballots:  5 yeses and 2 blank ones.
This makes the counts 473 yes, 32 no, 5 blank.

I did not systematically examine the data by time of arrival, but a quick
check suggests that the cumulative yes:no ratio remained roughly constant
throughout.  (In particular, the inclusion of late ballots, arriving after
Brad's posting of results, does not appear to make a big difference.)

I decided to verify roughly 10% of the ballots.  A quick hack with a
random number generator (seeded by time of day to make it even more
thoroughly unpredictable -- the point of this is not high-quality
statistical randomness, but merely the fact that Brad couldn't have
predicted my choices) selected 52 ballots to be verified.  A manual
address-munging pass rejected 4 (all "yes" votes) because of complex
mixed-mode addresses that I doubted could be mailed to successfully.
Of the remainder, 3 were "no" votes and the rest (45) were "yes" votes.
48 confirmation requests went out late in the afternoon of the 15th.

As of now (1220 on the 20th), 41 responses have come back.  One message
bounced, the remaining 7 have not replied for unknown reasons.  All
41 responses confirmed that Brad's records were accurate.  (Several
added lengthy comments, showing that they'd thought about the issues
in depth, but I didn't keep track of the specifics.)  The missing ones
are all "yes"es, by the way -- all three "no"s replied.

From this I conclude that the ballots collected by Brad were real and
legitimate, and that the 473:32 result is real.

There are three possible sources of error that I cannot check in any
straightforward way:  the possibility that significant numbers of people
posted multiple ballots by using different accounts on different machines,
the possibility that mail fraud was committed with the same net effect, and
the possibility that Brad quietly discarded all but a handful of the "no"
ballots.

The multiple-voting possibility cannot be entirely ignored, but given the
numbers involved and the geographic diversity (even the verification
list included people in Europe and Australia (all of whom responded, by
the way -- the missing 7 are all USAnians)), it would have been a
formidable task to influence the result significantly.

Mail fraud would make cheating easier, but the ballots came by a variety
of routes, the only major common element being a tendency to converge
near the end since Brad's machine has few mail links.  As I mentioned
above, I hand-munged the addresses for more efficient paths (utzoo is
very well connected).  Very few of the resulting paths went through any
of Brad's neighbors, and the diversity of routes was even greater.
There was no obvious common pattern among the addresses of people who
did not response to the verification request.  Mail fraud on a significant
scale is most implausible.

As for deliberate discarding of "no" ballots, Brad could have made
the results rather less striking, and hence rather more likely to pass
unchallenged, by retaining more "no" votes.  Whatever one may think of
Brad, he's not stupid.  Such selective reporting seems unlikely.  The
response to Brad's survey is abnormally high by Usenet standards already;
the hypothesis that there were several hundred more "no" votes just is
not credible.

Brad tells me that there is at least one case where a known "no" ballot
did not reach him, but the fact is that uucp mail is not 100% reliable,
and there is no reason to expect that it would discriminate against
"no" ballots particularly.  Occasional mail losses would be reason for
concern if the vote were close, but it's not.

It has been alleged that the survey was biased by it being publicized
in rec.humor.funny only.  I will not comment on this here, except to say
that I do not find it credible that non-readers would respond in the truly
unprecedented numbers needed to reverse the result.

My conclusion is unless implausible sources of bias were present, it
is fair and correct to state that the voters support Brad's proposal
by a majority of approximately 473 to 32.  I further conclude that it
is most unlikely that the removal of any credible source of bias
could reverse such a lopsided result.  I finally conclude that Brad's
survey, although perhaps not entirely above reproach on small details,
was honestly run, and that the results he announced were at least
approximately correct.
-- 
Welcome to Mars!  Your         |     Henry Spencer at U of Toronto Zoology
passport and visa, comrade?    | uunet!attcan!utzoo!henry henry@zoo.toronto.edu