Magic (cryptography)
|
|
In World War II, Magic was the United States codename for intelligence derived from the cryptanalysis of PURPLE, a Japanese foreign office cipher.
| Contents |
Information carried in PURPLE traffic
The PURPLE machine itself was first used by Japan in 1939, but US and British cryptographers had broken some of its messages well before the attack on Pearl Harbor. US cryptographers decrypted and translated the 14-part Japanese diplomatic message breaking off relations (ominously) with the United States at 1PM Washington time on 7 December 1941 before the Japanese Embassy in Washington could do so. Difficulties at the Embassy were a major reason the note was delivered late.
The US found no hint of the attack on Pearl Harbor in the PURPLE traffic at the time, nor could they have as the Japanese were very careful to not discuss the planned attack in Foreign Office communications. In fact, no detailed information about the planned attack was even available to the Japanese Foreign Office; it was regarded by the military, particularly the more nationalistic military, as insufficiently 'reliable'. US access to private Japanese diplomatic communications (even the most secret ones) was less useful than it might otherwise have been because policy in Japan in the pre-War period was controlled largely by military groups (eg, in China and Manchuria), not by the Foreign Office. The Foreign Office itself deliberately kept from its embassies and consulates much of the information it did have, so the ability to read PURPLE transmissions was less than definitive regarding Japanese tactical or strategic military intentions.
Handling of Magic prior to Pearl Harbor attacks
Even so, the diplomatic information discovered was of even more limited value to the US because of its dissemination pattern within the US Government. "Magic" traffic was distributed in such a way that many policy makers who should have access to it to do their jobs knew nothing of it, and those to whom it actually was distributed (at least before Pearl Harbor) saw each message only briefly, as the courier stood by to take it back, and in isolation from all others (no copies or notes were permitted). Before Pearl Harbor, in any case, they saw only those decrypts thought "important enough" by the distributing Army or Navy officers. Nonetheless, being able to read PURPLE messages gave the Allies a great advantage in the War; for instance, the Japanese ambassador to Germany produced long reports for Tokyo which were encrypted with the PURPLE machine. They included reports on personal discussions with Hitler and a report on a tour of the invasion defenses in Northern France (including the D-Day invasion beaches).
Post-war debates
The break into the PURPLE traffic, and into Japanese messages generally, was the subject of acrimonious hearings in Congress post-WWII in connection with an attempt to decide who, if anyone, had allowed the disaster at Pearl Harbor to happen and who therefore should be blamed. During those hearings the Japanese learned, for the first time, that the PURPLE cypher machine had been broken. They had been continuing to use it, even after the War, with the encouragement of the American Occupation. Much confusion over who in Washington or Hawaii knew what and when, especially as "we were decrypting their messages," has led some to conclude that "someone in Washington" knew about the Pearl Harbor attack before it happened, and, since Pearl Harbor was not expecting to be attacked, the "failure to warn Hawaii one was coming must have been deliberate, since it could hardly have been mere oversight".
History
When PURPLE was broken by the US Army's Signals Intelligence Service (SIS), several problems arose for the Americans: who would get the decrypts, which decrypts, how often, under what circumstances, and crucially (given interservice rivalries) who would do the delivering? Both the US Navy and Army were insistent that they alone handle all decrypted traffic delivery, especially to highly placed policy makers in the US. Eventually, after much toing and froing, a compromise was reached: the Army would be responsible for the decrypts on one day, and the Navy the next.
The distribution list eventually included some—but not all—military intelligence leaders in Washington and elsewhere, some—but again not all—civilian policy leaders in Washington. The eventual routine for distribution included the following steps:
- the duty officer (Army or Navy, depending on the day) would decide which decrypts were significant or interesting enough to distribute
- they would be collected, locked into a briefcase, and turned over to a relatively junior officer (not always cleared to actually see the decrypts!) who would 'make the rounds' to the appropriate offices.
- no copies of any decrypts were left with anyone on the list. The receipient would be allowed to read the translated decrypt, in the presence of the distributing officer, and was required to return it immediately upon finishing. Before the beginning of the second week in December 1941, that was the last time anyone on the list saw that particular decrypt.
Decryption Process
There were several prior steps needed before any decrypt was ready for distribution:
- First, it had to be intercepted. The Japanese Foreign Office used both wireless transmission and cables to communicate with its off shore units. Wireless transmission was intercepted (if it was possible to do so) and any of several listening stations (Hawaii, Guam, Bainbridge Island in Washington State, etc) and the raw cypher groups forwarded to Washington. Eventually, there were decryption stations (ie, including a copy of the Army's PURPLE machine) in the Philippines as well. Cable traffic was (for many years before late 1941) collected at cable company offices by a military officer who made copies and started them to Washington. Cable traffic in Hawaii was not interecepted (illegality remember) until David Sarnoff of RCA agreed to allow it during a visit to Hawaii the first week of December 1941. At one point, intercepts were being mailed to (Army or Navy) Intelligence from the field!
- Second, the raw intercept had to be decyphered. This was done by either the Army or the Navy (depending on the day) and, because of the nature of the cypher and the break into it, was usually successful.
- Third, having obtained the plaintext, in Latin letters, it was translated. Because the Navy had more Japanese speaking officers, much of the translating burden fell onto the Navy. And because Japanese is a difficult language, leaving much meaning to context, effective translation required not only fluent Japanese, but considerable knowledge of the context within which the message was sent.
- Fourth, the now translated decrypt had to be evaluated for its intelligence content. For example, is the ostensible content of the message meaningful? If it is, for instance, part of a power contest within the Foreign Office or some other part of the Japanese government, its meaning/implications would be quite different than if it were a simple informational or instructional message to an Embassy. Or, might it be another message in a series whose meaning, taken together, is more than the meaning of any individual message. Thus, the fourteenth message to an Embassy instructing that Embassy to instruct Japanese merchant ships calling at that country to return to home waters before, say, the end of November would be more significant than a single such message meant for a single ship or port. Only after having evaluated a translated decrypt for its intelligence value could anyone decide whether it deserved to be distributed.
In the period before the attack on Pearl Harbor, the material was handled awkwardly and inefficiently, and was distributed even more awkwardly. Nevertheless, the extraordinary experience of reading a foreign government's most closely held communications, sometimes even before the intended recipient, was astonishing. It was so astonishing, that someone (possibly President Roosevelt) called it magic. The name stuck.
Magic and United States Executive Order 9066
One aspect of Magic remains controversial to this day - the amount of involvement, if any, the intercepts played in the issuing of United States Executive Order 9066, which lead to the internment of Japanese Americans living on the US West Coast. Those defending the internment, most notably author Michelle Malkin, point to Magic intercepts as being justification for the internment. The rationale for this is that several Magic intercepts discuss the development of a spy ring among Japanese Americans by the Japanese consulates, showing that the Japanese American community was an espionage risk.
Those in opposition point out that:
- the commanding officer who executed the internment, Lt. Gen. J. L. DeWitt, was not on the Magic intercept list,
- his superior, Secretary of War Henry Stimson, was on the intercept list, and
- Stimson requested justification for the internment from DeWitt (if Magic intercepts provided justification, why ask DeWitt for further justification?).
The issue has been inflamed recently due to the release of Malkin's recent book, In Defense Of Internment, in which the Magic intercepts play a major role in the defense of her thesis.
Other Japanese ciphers
In fact, PURPLE was an enticing, but quite tactically limited, window into Japanese planning and policy because of the peculiar nature of Japanese policy making prior to the War (see above). Early on, a better tactical window was the Japanese Fleet Code (an encoded cypher actually), called JN-25 by US Navy cryptographers. Breaking into the version in use in the months after December 7, 1941 provided enough information to lead to US naval victories in the battles of the Coral Sea and Midway, eliminating most of air power of the Japanese fleet at the latter and stopping the Japanese advance south with a 'draw' at the former. Later, broken JN-25 traffic also provided the schedule and routing of the plane Admiral Yamamoto Isoroku would be flying in during an inspection tour in the SW Pacific, giving US Army Air Corps pilots a chance to assassinate the officer who had devised the Pearl Harbor attack plan. And still later, access to Japanese Army messages from decrypts of Army communications traffic assisted in planning the island hopping campaign to the Philippines and beyond.
Public notice had actually been served that Japanese cryptography was dangerously inadequate by the Chicago Tribune, which published a series of stories just after Midway in 1942 directly claiming — correctly, of course — that the victory was due in large part to US breaks into Japanese crypto systems (in this case, the JN-25 cypher, though which system(s) had been broken was not mentioned in the newspaper stories). Fortunately, neither the Japanese nor anyone who might have told them seem to have noticed either the Tribune coverage, or the stories based on the Tribune account published in other US papers. Nor did they notice announcements made on the floor of the US Congress to the same effect. There were no changes in Japanese cryptography which could, then or now, be connected with those newspaper accounts or Congressional disclosures.
Other claimed breaks into PURPLE
In the book Sword and Shield, by C. Andrew, based on the "Mitrokhin Archive" smuggled out of Russia in the early '90s by a KGB archivist, the claim is made that the Soviets independently broke into Japanese PURPLE traffic (as well as the Red predecessor machine), and that decrypted PURPLE messages contributed to the decision by Stalin to move troops from Far Eastern Asia to the area around Moscow for the counterattack in December of '41. Those messages are said to have been credible enough to convince the Soviets there would not be a Japanese attack on them.
Fictional treatment
Neal Stephenson's novel Cryptonomicon includes a fictionalised version of Magic, with the Japanese cryptosystem being named "Indigo" rather than "PURPLE".