From: utzoo!utcsrgv!outer
Newsgroups: net.crypt
Title: Re: Digram statistics question
Article-I.D.: utcsrgv.1418
Posted: Fri May 20 04:10:37 1983
Received: Fri May 20 04:24:32 1983
References: <1245@psuvax.UUCP>

The idea is to use digrams to aid in the anagramming of a complete columnar
transposition.  Foster goes a little wild, but he's basicly right according to
the "classical" literature.  EG:
	Friedman, Elements of Cryptanalysis: "The method is to select a column
	which has a good assortment of high-frequency letters and try to find
	columns which can be added before and after the selected column to
	build up high-frequency digraphs and trigraphs."

	Sinkov, Elementary Cryptanalysis: [After failing to reconstruct the
	first line by sight] "Then the next step should be to try to fit two
	columns together so as to get good digraphs.... including, in
	particular...occurences of TH."

	Gaines, Cryptanalysis: "Sometimes this can be decided by simple
	observation.  Otherwise, the combinations can be subjected to a digram
	test.  This is made by setting down beside each digram, as formed by 
	each pair of columns, its frequency as taken from a digram chart....
	and the supposition is that the combination furnishing the highest 
	frequency-total will be the correct one..."

Why?  Because it is the redundancy of the frequent digrams that is the key to
putting the puzzle together.  "qu" and such are useful because they are so rare
as to be almost certain, but in general I think what you want to do is maximize
the latent patterns.  You look for what you expect (hope?) to find.  I know
that doesn't explain it very well, so I won't even guess at what sort of
probabilities are summed up in a digram chart.  They are not independent: in
"THE MAN" you take TH, HE, EM, MA & AN, sometimes omitting EM as it cuts across
words.  What does that make them?

	- richard outerbridge / UofT / CSRG
	  (...decvax!utzoo!utcsrgv!outer)