How Did Spy Services Develop in the United States?
Many Americans today would see that espionage work and spying by the United States is critical for their national security. The history of the services that have provided this, however, is more complex, and often mirrored the development and transformations that affected the US. Through its development as an independent state to one that expanded across the West, and its experiences in the Civil War and World Wars, history has shaped what spying means to the US.
What is different from many other powers that developed spying services is the relatively short history of the US. The experiences, therefore, are different in how they have influenced government's approach to spying. One of the first spy groups derived prior to the American Revolutionary War in 1765. The Sons of Liberty formed as a group that fought against the Stamp Act and would observe British troop movements in Boston and other places. This group included well known revolutionaries such as Samuel Adams and John Hancock.
During the Revolutionary War, George Washington was one of the first to write and recognize the need for espionage. Threats to the colonists/revolutionaries, and eventual early US military, were from Britain. War espionage depended on a network of spies throughout different areas, where Washington wanted to stay one step ahead of the pursuing British Army. This included using Native Americans, who were adept at secretive observations that Washington had firsthand experience with during the French and Indian War. Washington also used a cryptanalyst Samuel West to read secretive letters. Double agents were also used by the British to counter Washington, such as Dr. Benjamin Church who fed important information about the revolutionaries.
Benjamin Franklin (Figure 1) was placed in charge of the Committee of Secret Correspondence, which worked to change sentiment about the Revolution in European states so that the revolutionaries would gain increasing support. His most successful efforts were with the French, who ultimately helped aid the revolutionaries in fighting the British. This proved crucial in helping to turn the tide, where French troops and aid greatly helped the revolutionaries to successfully fight.
After the war, American spying took a downturn as the country was preoccupied with building the young country. In fact, the US was ill prepared for the War of 1812, where US soldiers were often unaware even where the frontier with Canada was even located. It was the Civil War where things changed once again. The North, in the Civil War, developed a network of spies who were able to cross between the borders with the South, including using such individuals as William Alvin Lloyd. The North also developed two spying agencies, including one run by Allan Pinkerton and another run by Lafayette Baker. There was two due to infighting between generals and Lincoln's cabinet who depended on the different spymasters. This often lead to information not being shared and sometimes counterproductive efforts that even led to the arrest of spies working for the other agency. There were some notable successes in troop movement observations, but often the information did not have a clear chain where the intelligence could be used for actionable purposes.
The South, on the other hand, had some notable successes, often creating an elaborate network in major northern cities. The most successful group was the so-called Canadian Cabinet, which ran a Southern spy network from Montreal. It is likely John Wilkes Booth, Lincoln's assassin, derived from this group. They also succeeded in several sabotage operations and creating fires in major cities that disrupted Northern cities.
Rise of Modern Spying
Modern American espionage begins in the late 19th century. This included Grover Cleveland calling for assigning military attachés in foreign countries to gather more information about different countries. John Wilkie became head of the US Secret Service in the 1890s and became noted for breaking up a Spanish spy ring in Montreal during the Spanish-American War. The Secret Service (Figure 2), in fact, was created as a result of the Civil War (in 1865) and for its first few decades mostly focused on combating counterfeiting of the US currency and other acts that could sabotage the US economy, such as smuggling.
In 1908, the Bureau of Investigation, what eventually became as the Federal Bureau of Investigation (FBI), was formed. Initially, they focused on policing activities and against organized criminal activities such as prostitution. However, they also engaged in domestic surveillance. This was tested in World War I, where they were able to discover German agents and saboteurs. Nevertheless, in World War I, the US was still not effective in foreign espionage. Most intelligence from World War I depended on British spy services that were established.
After World War I, most activity focused on domestic surveillance, particularly against communists and others deemed as threats. In the 1920s, modern cryptography developed in the US. William Friedman, a Russian immigrant, was appointed as cryptanalyst for the Army Signal Intelligence Service, which was successful in breaking the Japanese code used in their diplomatic cables. This enabled the US to be aware that an attack like Pearl Harbor was going to occur, although they were not able to determine where it would occur. World War II, particularly with the attack on Pearl Harbor, led to major new developments in intelligence gathering in the US. The US Navy's Combat Intelligence Unit was appointed with creating a a code breaking unit called "Magic." This work help lead to early developments of computers, such as it had in Britain by work conducted by Allan Turning during World War II.
World War II also began the long relationship between private industry and government for espionage work, with IBM developing their punch card machines to help in cracking Japanese code used in the war. This culminated in the Battle of Midway, where the US had advance knowledge of the battle and were able to defeat the Japanese because they had known when and where the attack would take place. During World War II, the Office of Strategic Services (OSS) began to coordinate special operations and spying activity.
The 1947 National Security Act created the Central Intelligence Agency (CIA) and led to the National Security Council (NSC) that had authority over it. The 1950s and 1960s saw the CIA leading many anti-communist and Soviet activities, where the US began to get far more involved in foreign politics, such as elections of left-wing leaders in different states. The most noted success was the uncovering of nuclear missiles sent to Cuba in the 1962 Cuban Missile Crisis. In the 1960s, the Directorate of Science and Technology (DST) was created to coordinate technology efforts to aid in spying, including the production of spy satellites and other technologies.
Meanwhile, the FBI became more active in domestic surveillance, where communism as well as organized crime were of keen interest. The next major change was the Patriot Act of 2001, which gave the government increased authority to collect and analyze private data. There was also a mandate to better coordinate foreign and domestic intelligence. This was in reaction to the September 11th attacks on New York and Washington. Since then, controversies have often revolved how much private information should the government posses and the balance of security and counter-terrorism efforts.
Spying has had many phases in the United States. It started before the Revolutionary War as an ad hoc network of spies that then evolved into more elaborate agencies in the Civil War. However, that period showed deficiencies by not coordinating activities. By the late 19th century, the Secret Service became the central spy service and later the FBI emerged as the agency that conducted forms of domestic surveillance. During World War II, there was an increased need to develop better foreign espionage. This led to the emergence of the CIA. Modern spying has also increasingly led to closer coordination between foreign and domestic surveillance.
- For more on spying during the American Revolutionary War, see: Sulick, Michael J. 2014. Spying in America: Espionage from the Revolutionary War to the Dawn of the Cold War. Georgetown University Press.
- For more on Benjamin Franklin's work during the war, see: Isaacson, Walter. 2004. Benjamin Franklin: An American Life. New York, NY: Simon & Schuster Paperbacks.
- For more on US spying between the Revolutionary War and the Civil War, see: Mickolus, Edward F. 2015. The Counterintelligence Chronology: Spying by and against the United States from the 1700s through 2014. Jefferson, North Carolina: McFarland & Company, Inc., Publishers.
- For more on the rise of American spying after 1865 and until World War I, see: Hastedt, Glenn P., ed. 2011. Spies, Wiretaps, and Secret Operations: An Encyclopedia of American Espionage. Santa Barbara, Calif: ABC-CLIO.
- For more on post-World War I spying development and up to World War II, see: 2002. Studies in Intelligence. V. 46, No. 3. CIA.
- For more on post-World War II spy developments in the United States, see: Carnes, Mark C., ed. 2007. The Columbia History of Post-World War II America. Columbia Guides to American History and Cultures. New York: Columbia University Press.