Showing posts with label substitution cipher. Show all posts
Showing posts with label substitution cipher. Show all posts

Thursday, 7 August 2008

Review of "The Montefeltro Conspiracy"...

Note: this article has now moved to review-of-the-montefeltro-conspiracy on Cipher Mysteries

Once upon a time, when I was trying to research the cryptographic history of Sforza Milan 1450-1500, it became painfully obvious that I had to build up a proper understanding of Francesco Sforza's chancellor Cicco Simonetta: more than just a 'gatekeeper' or even a 'lynchpin', Simonetta was the very lintel above the door, the central architectural feature silently and powerfully holding the whole enterprise together.

However, for the most part histories have tended to treat Simonetta as a marginal figure, as if he was simply some gouty old henchman beavering away in the Sforza family's shadows. Only when contemporary historians (Evelyn Welch perhaps most famously, but there are now quite a few others) began relentlessly chiselling away at the Sforza propaganda facade did Cicco become foregrounded as a useful object of study.

Despite my efforts to collate what fleeting references to Cicco I could find, he remained an elusive figure. But then I found a relatively unknown book in Italian called "Rinascimento Segreto: Il mondo del Segretario da Petrarca a Machiavelli" (2004) by Marcello Simonetta: chapters III and V covered the key people & period I was particularly interested in. The author's surname is no coincidence: when Marcello went to Yale in 1995, his professor from the palaeography class (the very excellent Vincent Ilardi) "immediately suggested that [he] write a biography of [his] ancestor Cicco Simonetta". Poignantly, Marcello had been born in a hospital in Pavia "only a few yards from where Cicco Simonetta was imprisoned at the end of his long life."

I should have been delighted: but my Italian comprehension has only ever been tourist-plus, and "Rinascimento Segreto" was written in (to me) full-on academese. Yet even though reading it was a hard, hard slog, it really did have everything I needed to build up a fuller picture, as well as plenty on other related stuff (such as the Visconti, the Pazzi conspiracy, Roberto da Sanseverino, Filfelfo, and so forth). In many ways, Simonetta's book was one of the ten or so key texts that substantially contributed to my research back then.

Since then, Marcello has been busy digging further trenches within the same Quattrocento patchwork of fields. Most notably, in 2001 he uncovered (in the Ubaldini family archive in Urbino) an enciphered letter from Federico da Montefeltro to his envoys in Rome, dating from 14th February 1478 - a mere ten weeks before the Pazzi conspiracy attack on Lorenzo and Giuliano de' Medici. Marcello had already accumulated plenty of material implicating Federico in the whole plot: and so wondered whether this letter might be connected...

During 2002 or 2003, he therefore decided to see if he could break the letter's cipher using only the set of "Regule" (rules) famously written down by Cicco Simonetta in his diary: these described how to break unknown ciphertexts. "After a few weeks of hard work", Marcello was finally able to decipher it: and it revealed, just as he had inferred from other documents, that Federico da Montefeltro had indeed been utterly involved with the whole plot against the Medicis. Marcello published his results in the well-respected Archivio Storico Italiano: but it was not historians who responded in 2004, but the world's media, bringing him a small measure of international fame: in 2005, a documentary even came out on the History Channel describing Marcello's story.

Fast forward to 2008, and here's Marcello's brand new popular history book "The Montefeltro Conspiracy", which does everything you'd expect: it tells the interlinked stories of Cicco Simonetta, Gian Galeazzo Sforza, Federico da Montefeltro, Lorenzo de' Medici, Pope Sixtus IV, and the whole Pazzi conspiracy (and the subsequent Pazzi war), particularly focusing on the political machinations from 1476 to 1482, together with the story of the ciphered letter.

Well, that's the making-of-the-book covered, the kind of human-interest story PR people love to feed to tame journalists (not that I've received a single PR release to date, let alone a review copy of anything): but what is the book actually like?

For the first 50 pages, I have to say that I really didn't enjoy the book. To me they read like 19th century jut-jawed Italian popular histories, such as Count Pier Desiderio Pasolini's "Catherine Sforza". Even though I happened to love that book, it's really not something that could be sensibly released nowadays, because sensibilities and presentation styles have moved on so far: modern history is so much better than that.

All the same, beyond that point, Marcello progressively got into the swing of it: and by about page 150, he had really got the measure of the material and the pacing, and his story was really flying. Yet the very final section appended to the structure (where he proposes a link between Botticelli's uber-famous "La Primavera", his "Punishment of Korah" (the fifth fresco on the walls of the Sistine chapel), and the whole Medici-Pazzi thing) just doesn't work at all (sorry); and so the whole book ends on a bit of an historical down note, which is a shame.

Having tried my own hand at writing an accessible historical account of the mid-Quattrocento (and it is a far harder challenge than it looks), I'd put the lull of the first 50 pages down to popular writing inexperience on Simonetta's part (trust me, he can do full-on academese just fine): so in the end, I'd still recommend his book overall as a good piece of historical writing on a fascinating era.

As an aside, an article last week by Juliet Gardiner in the Sunday Times eulogizing contemporary British historians (almost to the point of hagiography, it should be said) also criticized European historians' writing for being too polarized between high and low culture:-

"[Richard] Evans makes the point that, on the Continent, the divide between academic and popular history is far deeper. Elsewhere in Europe, history is seen as a social science (Wissenschaft), so it tends to be written in 'high academese', a theoretical, technical style that is all but impenetrable to all but the committed specialist. In Britain, history is seenas a branch of literature, rather than science, and the tradition of writing narratic, empirical history, often with an emphasis on biography, provides a vivid 'story' that can be appreciated by the educated reader."
I would say that "The Montefeltro Code" amply demonstrates of all these historiographical trends: yet I do look forward to further historical books by Simonetta, particularly as his popular writing style continues to improve (as it undoubtedly will).

However, when considering his book as a piece of cryptographic writing, I have a whole heap of issues. Despite the huge influence of the Da Vinci Code on the publishing trade, there are very few recent books that could genuinely qualify as both historical and cryptographic non-fiction (Simonetta's "The Montefeltro Code" and my book "The Curse of the Voynich" are pretty much the only two I can think of right now), as long as you put the torrent of titles on the whole Enigma / Bletchley Park thing to one side.

In this context, Simonetta would have been aware that cryptography historians would take a keen interest in his book, and should therefore have checked his work accordingly. Unfortunately, this seems not to have happened.

I'll give some immediate examples from p.26. Though his mention of "the insecure roads of Europe" is true for most cipher dispatches, my understanding is that Sforza cipher dispatches were (according to Francesco Senatore) folded up inside a littera clausa, powerfully deterring anyone from even trying to peek inside. In each cipher, Marcello says "there were about 250 random symbols, which stood for single, double, and triple characters": actually, they stood for single letters, doubled letters, and nulls, as well as some common short words, and occasionally common consonant-vowel or vowel-consonent pairs. In fact, Cicco Simonetta's Regule pointed out that the only Latin word with a tripled letter is "uvula" (egg), making this an even more obvious mistake (even though Cicco himself seems to have miscopied this as "mula"). "Some fifty other[ symbol]s designated people or powers": actually, this number varied widely. "Every few months, the sets were completely changed": I don't think this is true at all - Tristano Sforza's cipher was changed only after about 15 years in use, and only because of Tristano's petulance (his old cipher wasn't ornate enough for his position) rather than any cryptographic need. In fact, as far as I know, the only Milanese cipher of the period that was updated much was the one to Tranchedino in Florence... and so on.

All very minor and (frankly) unnecessary: but it is Marcello's claims relating to Federico da Montefeltro's ciphered letter that require the most careful scrutiny. In a recent email, Augusto Buonafalce flagged to me that Marcello had not made it transparently clear how he had decoded the nomenclator (the list of people/place/etc, each represented by a single symbol): and that this was central to whether his deciphering claim was cryptologically valid or not.

Certainly, when Simonetta first published his findings in 2003, he had (though this is not made clear anywhere) only guessed at the "persons and powers" code-table section of the nomenclator: many of these symbols appear in the two pages he reproduced (for example, you can see instances of c24, j6, p1, p2, p12, r1 dotted around the page). In January 2004, I suggested to him that he should examine the Urbinate Lat. 998 cipher ledger (held in the Vatican), which contains various Urbino ciphers, and pointed out that, from what I had seen, it seemed to be common practice in Urbino to reuse & extend codebooks rather than to create entirely new ones. When Marcello had a look at Urb. Lat. 998 in the summer of 2004, he was pleasantly surprised to find two symbols reused from a (then ten-year-old) cipher codebook: yet the remainder were still educated guesses on his part. Though he included two small images of the "Montefeltro Codebook" on p.91 (but with no folio reference), these are not at the level of cryptographic proof that would satisfy a Cryptologia readership: his code-table cracks were based more far on historical inferences than on cryptography.

Though Marcello took several weeks to break the cipher, it should also be pointed out that this was because Cicco's rules were simplistic (and, I suspect, hardly ever used in practice): had Marcello passed his transcription to a cryptologer, it would probably have yielded up its secrets in mere minutes - code-table aside, it was a very simple cipher.

Ultimately, the irony of the situation is that the Sforza camp (and specifically Cicco Simonetta, I argued in my book) had provided the Montefeltro camp with far better ciphers than this since the 1440s: yet because Federico was now moving his loyalty away from Milan, the new cipher his administrators created for him was far simpler - but one unknown to his former allies.

All this points towards what I found so maddeningly annoying about "The Montefeltro Code": that neither the cryptological methodology nor the cryptographical history were treated fairly and in context. In the end, the book presents a good historical rendering of a fascinating period with only a light dusting of crypto confetti on the surface - much as I liked its historical side (and would indeed have walked across broken glass to get a copy of it when writing my book), anyone hoping for a brilliant synthesis of that with cryptography may well come away disappointed.

Friday, 13 June 2008

20th Century Voynich Manuscript!?

Copies of a curious little apparently enciphered object were being given away in Dillons Arts bookshop about 12 years ago: I saw this last year mentioned on Cylob's blog (he's a musician now living in Berlin), but haven't found any further mention of it anywhere on the Internet.

To my eyes, it looks like a simple substitution cipher (you can see several of the shapes repeating, and you can probably guess at least some of the vowels), with a kind of vaguely pigpenesque quality to them (so there is probably some underlying rationale behind the alphabet). Maybe one day I'll ask Cylob for a copy & post a transcription here...

Sunday, 11 May 2008

The d'Agapeyeff Cipher...

Back in 1939, Alexander d'Agapeyeff wrote a tidy little book called "Codes and Ciphers" on cryptography history: though you can now buy it print-on-demand, cheap copies of the original book often come up on the various second-hand book aggregators (such as bookfinder.com), which is where I got my copy of the "Revised and reset" 1949 edition.

What is now generally understood is that d'Agapeyeff wasn't really a cryptographer per se: he had previously written a similar book on cartography for the same publisher, and so thought to tackle cryptography.

On the very last page of the text (p.144), d'Agapeyeff dropped in a little cipher challenge, saying "Here is a cryptogram upon which the reader is invited to test his skill."

75628 28591 62916 48164 91748 58464 74748 28483 81638 18174
74826 26475 83828 49175 74658 37575 75936 36565 81638 17585
75756 46282 92857 46382 75748 38165 81848 56485 64858 56382
72628 36281 81728 16463 75828 16483 63828 58163 63630 47481
91918 46385 84656 48565 62946 26285 91859 17491 72756 46575
71658 36264 74818 28462 82649 18193 65626 48484 91838 57491
81657 27483 83858 28364 62726 26562 83759 27263 82827 27283
82858 47582 81837 28462 82837 58164 75748 58162 92000


This modest little cryptogram, now known as "the d'Agapayeff Cipher", has somehow remained unbroken for 70 years, and is often to be found alongside the Voynich Manuscript on lists of cipher enigmas.

The first thing to note is that adjacent columns are formed alternately from 67890 and 12345 characters respectively: which is a huge hint that what we are looking at is (in part, at least) a grid cipher, where each pair of numbers gives a position in a grid. If so, then we can throw away the "patristrocat" spaces between the blocks of numbers and rearrange them as pairs.

75 62 82 85 91 62 91 64 81 64 91 74 85 84 64 74 74 82 84 83 81 63 81 81 74
74 82 62 64 75 83 82 84 91 75 74 65 83 75 75 75 93 63 65 65 81 63 81 75 85
75 75 64 62 82 92 85 74 63 82 75 74 83 81 65 81 84 85 64 85 64 85 85 63 82
72 62 83 62 81 81 72 81 64 63 75 82 81 64 83 63 82 85 81 63 63 63 04 74 81
91 91 84 63 85 84 65 64 85 65 62 94 62 62 85 91 85 91 74 91 72 75 64 65 75
71 65 83 62 64 74 81 82 84 62 82 64 91 81 93 65 62 64 84 84 91 83 85 74 91
81 65 72 74 83 83 85 82 83 64 62 72 62 65 62 83 75 92 72 63 82 82 72 72 83
82 85 84 75 82 81 83 72 84 62 82 83 75 81 64 75 74 85 81 62 92 00 0[0]


The first hint that the order of these might have been scrambled ('transposed') comes from the two sets of tripled letters: 75 75 75 and 63 63 63. Five centuries ago, even Cicco Simonetta and his Milanese cipher clerks knew that tripled letters are very rare (the only one in Latin is "uvula", 'little egg'). The second hint that this is a transposition cipher is the total number of characters (apart from the "00" filler at the end): 14x14. If we discard the filler & rearrange the grid we get:-

75 62 82 85 91 62 91 64 81 64 91 74 85 84
64 74 74 82 84 83 81 63 81 81 74 74 82 62
64 75 83 82 84 91 75 74 65 83 75 75 75 93
63 65 65 81 63 81 75 85 75 75 64 62 82 92
85 74 63 82 75 74 83 81 65 81 84 85 64 85
64 85 85 63 82 72 62 83 62 81 81 72 81 64
63 75 82 81 64 83 63 82 85 81 63 63 63 04
74 81 91 91 84 63 85 84 65 64 85 65 62 94
62 62 85 91 85 91 74 91 72 75 64 65 75 71
65 83 62 64 74 81 82 84 62 82 64 91 81 93
65 62 64 84 84 91 83 85 74 91 81 65 72 74
83 83 85 82 83 64 62 72 62 65 62 83 75 92
72 63 82 82 72 72 83 82 85 84 75 82 81 83
72 84 62 82 83 75 81 64 75 74 85 81 62 92

This is very probably the starting point for the real cryptography (though the presence of tripled characters in the columns implies that it probably isn't a simple "matrix-like" diagonal transposition. Essentially, it seems that we now have to solve a 14x14 transposition cipher and a 5x5 substitution cipher simultaneously, over a relatively small cryptogram - an immense number of combinations to explore.

However, we know that d'Agapeyeff wasn't a full-on cryptographer, so we should really explore the psychological angle before going crazy with an 800-year-long brute-force search. For a start, if you lay out the frequencies for the 5x5 letter grid (with 12345 on top, 67890 on the left), a pattern immediately appears:-

** .1 .2 .3 .4 .5
6. _0 17 12 16 11
7. _1 _9 _0 14 17
8. 20 17 15 11 17
9. 12 _3 _2 _1 _0
0. _0 _0 _0 _1 _0


Here, the 61 (top-left) frequency is 0, the 73 frequency is 0, and the final nine frequencies are 3, 2, 1, 0; 0, 0, 0, 1, 0. I think this points to a 5x5 mapping generated by a keyphrase, such as "Alexander d'Agapeyeff is cool" (for example). To make a keyphrase into a 5x5 alphabet, turn all Js into Is (say), remove all duplicate letters (and so it becomes ALEXNDRGPYFISCO), and then pad to the end with any unused characters in the alphabet in sequence (BHKMQTUVWZ)

* 1 2 3 4 5
6 A L E X N
7 D R G P Y
8 F I S C O
9 B H K M Q
0 T U V W Z

For a long-ish (but language-like) keyphrase, rare characters would tend to get moved to the end of the block: which is what we appear to see in the frequency counts above, suggesting that the final few letters are (for example) W X Y Z or W X Z.

Yet 61 and 73 have frequency counts of zero, which points to their being really rare letters (like Q or Z). However, if you read the frequency counts as strings, 61 62 63 = 0 17 12, while 73 74 75 = 0 14 17: which perhaps points to the first letter of the keyphrase (i.e. 61) being a rare consonant, and the second pair being Q U followed by a vowel. Might 73 74 75 76 77 be QUIET or QUITE?

I don't (of course) know: but I do strongly suspect that it might be possible for a cunning cryptographer to crack d'Agapeyeff's keyphrase quite independently of his transposition cipher. It can't be that hard, can it? ;-p

----------
Update: a follow-up post is here...