The Puzzle Palace: A Report on America’s Most Secret Agency
“Brute force” is a phrase which turns up often in James Bamford’s useful investigation of the National Security Agency, the largest but least known of American intelligence services. As used by NSA officials, “brute force” refers to the method of last resort in cracking secret codes. The best method is to obtain the key to the code by stealing it, buying it, or figuring it out, but that is often impossible. A “brute force” attack simply tries out all the possible keys to a code identified through intercepted messages.
These possibilities can be very numerous. A commercial cipher with a fifty-six-bit key devised by IBM in the mid-1970s, for example, would present an inquisitive outsider with about seventy quadrillion possibilities. Large as that number is, a computer could be built with speed and capacity enough to try out every last one of them in less than a day. This helps to explain why the NSA’s headquarters at Fort Meade, Maryland, is the computer capital of the world. Lt. Gen. Marshall S. Carter, director of the NSA (DIRNSA) for just over four years between 1965 and 1969, told Bamford he was in charge of five and a half acres of computers. Another NSA official told him that computer acreage is now about double what it was in Carter’s day.
It is the scale of the NSA that impresses. According to Bamford, the NSA’s main building at Fort Meade contains about 1.9 million square feet of floor space, roughly equal to the CIA’s headquarters in Langley, Virginia, and the Capitol building in Washington combined. It has employed as many as 95,000 people, classifies 50 million to 100 million documents a year, produces 40 tons of classified waste per day, has up to $1 billion in contracts out at any given time, and has an annual budget of perhaps $10 billion.
Making, breaking, and protecting codes is only part of the work conducted by this vast establishment, and the smaller part at that. The larger part is collecting and analysing COMINT, SIGINT, and ELINT. COMINT and SIGINT are communications and signals intelligence—messages of every conceivable type, from ordinary commercial cables to Soviet naval communications to ships at sea. When the Soviet space capsule Soyuz I ran into trouble during reentry in April 1967, technicians at an NSA listening station in Turkey taped the whole awful event, from the first discussion of problems with the parachute through the cosmonaut’s final farewell to his wife and a terminal scream when the capsule burned up as it plunged into the earth’s atmosphere. Also included under the general heading of SIGINT are radio transmissions of a technical nature such as radar or telemetry broadcast by Soviet missiles during test flights, the principal source of American intelligence about the performance of Soviet strategic weapons. ELINT is electronics intelligence, defined as electromagnetic radiators with a non-atomic origin. (HUMINT—human intelligence collected by traditional espionage—is the province of the CIA.)
The size of the NSA budget is partly explained by the huge technical apparatus required to collect so many radio transmissions of so many different types. Here, too, the approach is one of brute force. The NSA attempts to collect all Soviet transmissions—the full daily broadcast of every conventional radio station in all the Soviet republics, every transmission to every Soviet embassy abroad, every broadcast to a ship at sea, every transmission by military units on maneuvers in Eastern Europe, the radio traffic of every control tower at Soviet airports, the radar signature of every Soviet system. (If B-52s or a new bomber are to penetrate Soviet air space in the event of war, for example, they have got to know where Soviet air defenses are located. The NSA helps to provide the maps.)
Even when the transmissions can’t be read, which is often the case, the volume of traffic itself is analysed for whatever it might reveal Shortly before the Berlin Wall was built in 1961, it is said, the sheer number of radio messages and the level of the codes in which they were encrypted should have indicated that something was up. In August 1968 Soviet military units on the periphery of Czechoslovakia were “lost” for several days. Apparently this was achieved by sophisticated jamming which disguised actual Soviet military radio traffic. Richard Helms, then the director of Central Intelligence and nominally in charge of the NSA, confessed to the President’s Foreign Intelligence Advisory Board that he was much embarrassed by the failure, but assured them the intelligence community would have done better if the Soviets had been heading West.
Monitoring Soviet radio transmissions is only half the work. Since it’s always possible that a code might be cracked in future, the unread transmissions—all the unread transmissions—are stored, more or less forever, on magnetic tape.
The National Security Agency was established by President Truman—secretly: its official charter has never been made public—in the fall of 1952, but its true origins go back to the First World War when a State Department code clerk, Herbert O. Yardley, tried his hand at a secret message addressed to Woodrow Wilson in May 1916. He solved it within a couple of hours and immediately concluded that the British, who controlled the eastern terminal of transatlantic cables to North America, were reading the US government’s most secret diplomatic traffic. When the United States entered the war Yardley transferred to the military, founded a Code and Cipher Solution Subsection in the office devoted to military intelligence, and at the war’s end was arranging liaison with the French Chambre noire, or “black chamber,” a name Yardley borrowed when he was appointed to head a permanent code-breaking office in the War Department in 1919.
Yardley’s initial budget was $100,000, with which he hired a staff of fifty at salaries ranging from $1,200 per year for clerks up to $3,000 for senior code breakers. But the end of the war presented Yardley with two major problems. The first was the end of official censorship, which meant he no longer had automatic access to international cable traffic. Indeed, it was now against federal law to intercept messages, but Yardley quietly arranged with the management of Western Union and Postal Telegraph, the two major international carriers, for the Black Chamber to temporarily “borrow” messages. The two companies agreed to this illegal arrangement in the interest of national security, a precedent that was to be enduring.
Yardley’s major coup during this period was cracking the Japanese diplomatic code, which allowed the US to read Tokyo’s fallback position during the negotiations in 1921 to establish a fixed ratio of capital ships among the navies of Britain, the United States, and Japan. Knowing that Japanese negotiators had been instructed to accept the 10 to 6 ratio pressed by Britain and the United States, the Western team simply sat tight until the Japanese gave in.
But this was the high point of the Black Chamber and of Yardley’s career as well. His second major problem in the immediate years after World War I—indifference at high levels—was never overcome. It is war and the threat of war that turn a government’s thoughts to espionage. The Black Chamber’s budget was gradually reduced to $25,000 a year, most of it provided by the State Department. When Herbert Hoover’s secretary of state, Henry L. Stimson, took office in March 1929, Yardley hesitated to brief him on the Black Chamber’s work. When he finally did so a few months later, Stimson was appalled by the whole undertaking and ordered it to cease immediately with words about as close to immortal as any ever uttered by an American statesman: “Gentlemen do not read each other’s mail.”
But the end of the Black Chamber and Yardley’s departure from government did not end American code-breaking efforts. The War Department established a Signal Intelligence Service in 1930 which limped along on tiny budgets (never more than $17,400) until 1937, when the obvious approach of war brought high-level interest and funds to match. By September 1939 the staff of the SIS had grown from seven to nineteen. By December 7, 1941, it had reached 331. (The SIS had even cracked the new Japanese diplomatic code, called “Purple,” and had intercepted a message that indicated war was near, but a comedy of errors delayed a warning to the US commander in Hawaii until several hours after the attack on Pearl Harbor had begun.)
By the war’s end the SIS staff exceeded 10,000 and its official duties had vastly expanded to include the collection and analysis of all sorts of communications and signals intelligence. In 1949 the various communications intelligence branches were combined and named the Armed Forces Security Agency; three years later—largely as the result of high-level dissatisfaction with the AFSA’s performance during the Korean War, as well as weariness with the service rivalries which drove the White House half crazy during the postwar years—the entire process of reorganization was repeated. In its present form the National Security Agency not only is bigger than the postal, telephone, and telegraph services (known as PT&Ts) of most major nations, but it draws heavily on the services of the Army, the Navy, and the Air Force as well.
James Bamford has assembled all that was known, and much that was unknown, in his history of the NSA, but the result does not make for light reading. The chapter about Yardley and a handful of stories scattered throughout the rest of the text provide the only narrative. For the rest, his book reads like a study of AT&T, with methodical lists of its directors and deputy directors, the divisions and subdivisions of the NSA, and the listening stations of various kinds scattered about the world. A great many officials are identified by name and job title. Much hardware is paraded across the page, from elaborate computer retrieval systems to giant dish antennae. Much of this material is new, gleaned from 6,000 pages of NSA newsletters and Justice Department documents that Bamford obtained under the Freedom of Information Act (and which the government, a year ago, attempted to retrieve with the claim they had been released in error). Bamford has also conducted several revealing interviews, especially with General Carter and Francis Raven, formerly in charge of reading Soviet and, later, third-world communications traffic.
Bamford deserves special praise for largely avoiding the mocking, ironic, superior tone adopted by many journalists when writing about the intelligence and defense communities. These virtues add up to a considerable achievement. But that should not obscure the fact that the secrecy surrounding the NSA is still largely intact. Bamford has mapped the landscape, much as David Wise and Thomas B. Ross did in 1965 with their similar trail-breaking study of the CIA, The Invisible Government. Future researchers will owe Bamford a considerable debt, but they will also have plenty to do.