Who's in Big Brother's Database?

The Secret Sentry: The Untold History of the National Security Agency by Matthew M. Aid Bloomsbury, 423 pp.

On a remote edge of Utah’s dry and arid high desert, where temperatures often zoom past 100 degrees, hard-hatted construction workers with top-secret clearances are preparing to build what may become America’s equivalent of Jorge Luis Borges’s "Library of Babel," a place where the collection of information is both infinite and at the same time monstrous, where the entire world’s knowledge is stored, but not a single word is understood. At a million square feet, the mammoth $2 billion structure will be one-third larger than the US Capitol and will use the same amount of energy as every house in Salt Lake City combined.

Unlike Borges’s "labyrinth of letters," this library expects few visitors. It’s being built by the ultra-secret National Security Agency – which is primarily responsible for "signals intelligence," the collection and analysis of various forms of communication – to house trillions of phone calls, e-mail messages, and data trails: Web searches, parking receipts, bookstore visits, and other digital "pocket litter." Lacking adequate space and power at its city-sized Fort Meade, Maryland, headquarters, the NSA is also completing work on another data archive, this one in San Antonio, Texas, which will be nearly the size of the Alamodome.

The Secret Sentry: The... Aid, Matthew M. Best Price: $4.31 Buy New $24.15 (as of 05:25 UTC - Details)

Just how much information will be stored in these windowless cybertemples? A clue comes from a recent report prepared by the MITRE Corporation, a Pentagon think tank. "As the sensors associated with the various surveillance missions improve," says the report, referring to a variety of technical collection methods, "the data volumes are increasing with a projection that sensor data volume could potentially increase to the level of Yottabytes (10^24 Bytes) by 2015." Roughly equal to about a septillion (1,000,000,000,000,000,000,000,000) pages of text, numbers beyond Yottabytes haven’t yet been named. Once vacuumed up and stored in these near-infinite "libraries," the data are then analyzed by powerful infoweapons, supercomputers running complex algorithmic programs, to determine who among us may be – or may one day become – a terrorist. In the NSA’s world of automated surveillance on steroids, every bit has a history and every keystroke tells a story.

In the near decade since September 11, the tectonic plates beneath the American intelligence community have undergone a seismic shift, knocking the director of the CIA from the top of the organizational chart and replacing him with the new director of national intelligence, a desk-bound espiocrat with a large staff but little else. Not only surviving the earthquake but emerging as the most powerful chief the spy world has ever known was the director of the NSA. He is in charge of an organization three times the size of the CIA and empowered in 2008 by Congress to spy on Americans to an unprecedented degree, despite public criticism of the Bush administration’s use of the agency to conduct warrantless domestic surveillance as part of the "war on terror." The legislation also largely freed him of the nettlesome Foreign Intelligence Surveillance Court (FISA). And in another significant move, he was recently named to head the new Cyber Command, which also places him in charge of the nation’s growing force of cyber warriors.

The Shadow Factory: Th... James Bamford Best Price: $1.92 Buy New $11.41 (as of 06:15 UTC - Details)

Wasting no time, the agency has launched a building boom, doubling the size of its headquarters, expanding its listening posts, and constructing enormous data factories. One clue to the possible purpose of the highly secret megacenters comes from the agency’s British partner, Government Communications Headquarters. Last year, the British government proposed the creation of an enormous government-run central database to store details on every phone call, e-mail, and Internet search made in the United Kingdom. Click a "send" key or push an "answer" button and the details of the communication end up, perhaps forever, in the government’s data warehouse to be scrutinized and analyzed.

But when the plans were released by the UK government, there was an immediate outcry from both the press and the public, leading to the scrapping of the "big brother database," as it was called. In its place, however, the government came up with a new plan. Instead of one vast, centralized database, the telecom companies and Internet service providers would be required to maintain records of all details about people’s phone, e-mail, and Web-browsing habits for a year and to permit the government access to them when asked. That has led again to public anger and to a protest by the London Internet Exchange, which represents more than 330 telecommunications firms. "We view…the volume of data the government now proposes [we] should collect and retain will be unprecedented, as is the overall level of intrusion into the privacy of citizenry," the group said in August.

Body of Secrets: Anato... James Bamford Best Price: $1.31 Buy New $7.99 (as of 12:20 UTC - Details)

Unlike the British government, which, to its great credit, allowed public debate on the idea of a central data bank, the NSA obtained the full cooperation of much of the American telecom industry in utmost secrecy after September 11. For example, the agency built secret rooms in AT&T’s major switching facilities where duplicate copies of all data are diverted, screened for key names and words by computers, and then transmitted on to the agency for analysis. Thus, these new centers in Utah, Texas, and possibly elsewhere will likely become the centralized repositories for the data intercepted by the NSA in America’s version of the "big brother database" rejected by the British.

Matthew M. Aid has been after the NSA’s secrets for a very long time. As a sergeant and Russian linguist in the NSA’s Air Force branch, he was arrested and convicted in a court-martial, thrown into prison, and slapped with a bad conduct discharge for impersonating an officer and making off with a stash of NSA documents stamped Top Secret Codeword. He now prefers to obtain the NSA’s secrets legally, through the front door of the National Archives. The result is The Secret Sentry: The Untold History of the National Security Agency, a footnote-heavy history told largely through declassified but heavily redacted NSA reports that have been slowly trickling out of the agency over the years. They are most informative in the World War II period but quickly taper off in substance during the cold war.

Aid begins his study on the eve of Pearl Harbor, a time when the entire American cryptologic force could fit into a small, half-empty community theater. But by war’s end, it would take a football stadium to seat the 37,000 military and civilian "crippies." On August 14, 1945, as the ink dried on Japan’s instruments of surrender, the linguists and codebreakers manning the thirty-seven key listening posts around the world were reading more than three hundred diplomatic code and cipher systems belonging to sixty countries. "The American signals intelligence empire stood at the zenith of its power and prestige," notes Aid. But within days, the cryptanalysts put away their well-sharpened pencils and the intercept operators hung up their earphones. By the end of December 1945, America’s crypto world had shrunk to 7,500 men and women.

A Pretext for War: 9/1... Bamford, James Best Price: $1.25 Buy New $6.15 (as of 01:45 UTC - Details)

Despite the drastic layoffs, the small cadre of US and British codebreakers excelled against the new "main enemy," as Russia became known. The joint US-British effort deciphered tens of thousands of Russian army and navy messages during the mid-to-late 1940s. But on October 29, 1948, as President Truman was about to deliver a campaign speech in New York, the party was over. In what became known within the crypto world as "Black Friday," the Russian government and military flipped a switch and instantly converted to new, virtually unbreakable encryption systems and from vulnerable radio signals to buried cables. In the war between spies and machines, the spies won. The Soviets had managed to recruit William Weisband, a forty-year-old Russian linguist working for the US Army, who informed them of key cryptologic weaknesses the Americans were successfully exploiting. It was a blow from which the codebreakers would never recover. NSA historians called it "perhaps the most significant intelligence loss in US history."

In the 1970s, when some modest gains were made in penetrating the Russian systems, history would repeat itself and another American turncoat, this time Ronald Pelton, would again give away the US secrets. Since then, it has largely been a codemaker’s market not only with regard to high-level Russian ciphers, but also those of other key countries, such as China and North Korea. On the other hand, the NSA has made significant progress against less cryptologically sophisticated countries and, from them, gained insight into plans and intentions of countries about which the US has greater concerns. Thus, when a Chinese diplomat at the United Nations discusses some new African venture with a colleague from Sudan, the eavesdroppers at the NSA may be deaf to the Chinese communications links but they may be able to get that same information by exploiting weaknesses in Sudan’s communications and cipher systems when the diplomat reports the meeting to Khartoum. But even third-world cryptography can be daunting. During the entire war in Vietnam, writes Aid, the agency was never able to break the high-level encryption systems of either the North Vietnamese or the Vietcong. It is a revelation that leads him to conclude "that everything we thought we knew about the role of NSA in the Vietnam War needs to be reconsidered."

Because the book is structured chronologically, it is somewhat difficult to decipher the agency’s overall record. But one sees troubling trends. One weakness that seems to recur is that the agency, set up in the wake of World War II to prevent another surprise attack, is itself frequently surprised by attacks and other serious threats. In the 1950s, as over 100,000 heavily armed North Korean troops surged across the 38th parallel into South Korea, the codebreakers were among the last to know. "The North Korean target was ignored," says a declassified NSA report quoted by Aid. "North Korea got lost in the shuffle and nobody told us that they were interested in what was going on north of the 38th parallel," exclaimed one intelligence officer. At the time, astonishingly, the Armed Forces Security Agency (AFSA), the NSA’s predecessor, didn’t even have a Korean-language dictionary.

Unfortunately for General Douglas MacArthur, the codebreakers were able to read the communications of Spain’s ambassador to Tokyo and other diplomats, who noted that in their discussions with the general, he made clear his secret hope for all-out war with China and Russia, including the use of nuclear weapons if necessary. In a rare instance of secret NSA intercepts playing a major part in US politics, once the messages were shown to President Truman, MacArthur’s career abruptly ended.

Read the rest of the article

October 24, 2009