Misplaced Pages

Venona project: Difference between revisions

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
Browse history interactively← Previous editNext edit →Content deleted Content addedVisualWikitext
Revision as of 15:42, 19 August 2005 editJnc (talk | contribs)Autopatrolled, Administrators17,591 edits External links: Remove incorrect statement - the books source notes reveal that CPUSA archives were use a lot less than VENONA material← Previous edit Revision as of 16:21, 19 August 2005 edit undoJnc (talk | contribs)Autopatrolled, Administrators17,591 edits Major cleanup and reorganization; properly credit the cryptanalysts who made the initial break-in; fix other errorsNext edit →
Line 1: Line 1:
{{totallydisputed}} {{totallydisputed}}


'''The VENONA project''' was a long-running and highly secret collaboration between the ] intelligence agencies and the ]'s ] that involved the ] of ] messages. '''The VENONA project''' was a long-running and highly secret collaboration between ] ] and the ]'s ] that involved the ] of messages sent by several ] intelligence agencies. The British codename for VENONA was '''Bride'''.

In the early years of the ], it was one of the West's most important sources on Soviet intelligence activity, and although unknown to the public, it was important in many famous events of the early Cold War, such as the case of ], and the ].


==Background== ==Background==


] (commonly called ]) codebreakers had intercepted large volumes of ] high-level ] diplomatic and intelligence traffic during and immediately after ]. The British had stopped intercepting Soviet traffic, at ]'s orders, shortly after Germany invaded the Soviet Union in 1941 and had no traffic to contribute to the project after that time. This traffic, some of which was thought to be encrypted with a ] system, was stored and analyzed in relative secrecy by hundreds of cryptanalysts over a 40-year period starting in the early ]. The ] reported that, according to the serial numbers of the Venona cables, thousands were sent but only a fraction were available to the cryptanalysts. Approximately 2,200 of the messages were decrypted and translated; some 50 percent of the 1943 GRU-Naval Washington to Moscow messages were broken, but none for any other year, although several thousand were sent between 1941 and 1945. The decryption rate of the KGB cables was: ] (commonly called ]) codebreakers had intercepted large volumes of ] high-level ] diplomatic and intelligence traffic during and immediately after ]. The British had stopped intercepting Soviet traffic, at ]'s orders, shortly after Germany invaded the Soviet Union in 1941, and had no traffic to contribute to the project after that time.
This traffic, some of which was thought to be encrypted with a ] system, was stored and analyzed in relative secrecy by hundreds of cryptanalysts over a 40-year period starting in the early ]. Due to what turned out to be a serious blunder on the part of the Soviets - re-using pages of some of the one-time pads in other pads, which were then used for other messages - this traffic was vulnerable to crypanalysis.

The Venona Project was initiated under orders from the Chief of Military Intelligence, Carter Clarke, who mistrusted ]. He feared that Stalin and ] would sign a peace treaty in order to focus Germany's military forces on the destruction of Great Britain and the U.S.

==The breakin==

The Soviet systems in general used a ] to convert words and letters into numbers, to which an additive ] (from one-time pads) were added, further disguising the content. Some brilliant ] by American and British codebreakers revealed that some of the one-time pad material had incorrectly been reused by the Soviets (specifically, entire pages, although not complete books), which allowed decryption (sometimes only partial) of a small part of the traffic.

It was Arlington Hall's Lt. ], working on Soviet "Trade" traffic, who first discovered that the Soviets were re-using pages. Hallock and his colleagues (including Genevieve Feinstein, ], Frank Lewis, Frank Wanat, and Lucille Campbell) went on to break into a significant amount of "Trade" traffic, recovering many one-time pad additive key tables in the process.

A very young ] (of what would become the ]) then used this material to break in to what turned out to be ] (and later ]) traffic, by reconstructing the code used to convert text to numbers. Samuel Chew and Cecil Phillips also made valuable contributions. Very slowly, using assorted techniques ranging from traffic analysis to defector information, more of the messages were decrypted.

Claims have been made that information from physical theft of code books (a partially burned one was recovered by the Finns) to bugging embassy rooms in which text was entered into encrypting devices (analyzing the keystrokes by listening to them being punched in), contributed to achieving as much plaintext as was recovered. These latter claims are less than fully supported in the open literature.

One significant aid (mentioned by the NSA) in the early stages may have been work done in co-operation between the Japanese and Finnish cryptanalytic organizations; when the Americans broke into Japanese codes during WWII, they gained access to this information.

There has been speculation that the reason for the key material duplication was the increase in work (including key pad generation) in the period after the German attack in June of 1941. Other suggestions have it that it was ]'s tanks just outside Moscow in early December that year which forced Moscow Centre to make such a fundamental error.

==Results==

The NSA reported that, according to the serial numbers of the Venona cables, thousands were sent, but only a fraction were available to the cryptanalysts. Approximately 2,200 of the messages were decrypted and translated; some 50 percent of the 1943 GRU-Naval Washington to Moscow messages were broken, but none for any other year, although several thousand were sent between 1941 and 1945. The decryption rate of the NKVD cables was:


* 1942 1.8% * 1942 1.8%
Line 12: Line 36:
* 1945 1.5% * 1945 1.5%


Out of some hundreds of thousands of intercepted cyphertexts, it is claimed that under 3000 have been partially or wholly decrypted.
The Venona Project was initiated under orders from the Chief of Military Intelligence, Carter Clarke, who mistrusted ]. He feared that Stalin and ] would sign a peace treaty in order to focus Germany's military forces on the destruction of Great Britain and the U.S.

The British codename for VENONA was '''Bride'''. Some brilliant ] by American and British codebreakers (the first steps were by a very young ] of what would become ]) revealed that some of the one-time pad ] material had incorrectly been reused by the Soviets, which allowed decryption (sometimes only partial) of a small part of the traffic. Very slowly, using assorted techniques ranging from traffic analysis to defector information, more of the messages were decrypted. Out of some hundreds of thousands of intercepted cyphertexts, it is claimed that under 3000 have been partially or wholly decrypted. Claims have been made that information from physical theft of encryption pads (a partially burned one is reported to have been recovered by the Finns) to bugging embassy rooms in which text was entered into encrypting devices (and analyzing the keystrokes by listening to them being punched in), contributed to achieving as much plaintext as was recovered. These latter claims are less than fully supported in the open literature.


The Soviets eventually stopped reusing key pad material, possibly after learning from their agent(s) of the US / British work, after which their one-time pad traffic reverted to completely unreadable. There has been speculation that the reason for the key material duplication was the increase in work (including key pad generation) in the period after the German attack in June of 1941. Other suggestions have it that it was ]'s tanks just outside Moscow in early December that year which forced Moscow Centre to make such a fundamental error. The Soviets eventually stopped reusing key pad material, possibly after learning of the US / British work from several of their agents, after which their one-time pad traffic reverted to completely unreadable.


==Significance== ==Significance==
<i>Main articles: ] <i>Main articles: ]


The Venona project was a thirty-eight year investigation conducted by the ] and ] ], and held classified for an additional fifteen years after the program ended. Researchers, historians, and the public, struggle to understand its significance and meaning. The NSA followed Soviet intelligence traffic for only a few years in World War II, and decrypted only a small portion of that traffic. The Venona project was a thirty-eight year investigation conducted by the NSA and ] ], and held classified for an additional fifteen years after the program ended. Researchers, historians, and the public, struggle to understand its significance and meaning.

Senator ], Chairman of the ] ] was responsible for securing the release of Venona project materials. He wrote "The Venona intercepts contained overwhelming ] of the activities of Soviet spy networks in America, complete with names, dates, places, and deeds." (*).<sup id="fn_2_back">]</sup>


==List of Americans in the Venona Papers== ==List of Americans in the Venona Papers==
: ''See also: ]'' : ''See also: ]''
Senator ], Chairman of the ] ], responsible for securing the release of Venona project materials, later authored a book entitled ''Secrecy: The American Experience'' (New Haven: Yale University Press 1998). The Introduction states, "The Venona intercepts contained overwhelming ] of the activities of Soviet spy networks in America, complete with names, dates, places, and deeds." (*).<sup id="fn_2_back">]</sup>


349 U.S. citizens, noncitizen immigrants, and permanent residents of the United States who had covert relationships with Soviet intelligence were confirmed in the Venona traffic. Of these 171 are identified by true names and 178 are known only by a cover name.<sup id="fn_3_back">]</sup> The persons identified represent only a partial list and many are listed below. Twenty-four persons targeted for recruitment remain uncorroborated as to it being accomplished. These individuals are marked with an asterisk (*).<sup id="fn_4_back">]</sup> The NSA followed Soviet intelligence traffic for only a few years in World War II and decrypted only a small portion of that traffic. 349 U.S. citizens, noncitizen immigrants, and permanent residents of the United States who had covert relationships with Soviet intelligence were confirmed in the Venona traffic. Of these 171 are identified by true names and 178 are known only by a cover name.<sup id="fn_3_back">]</sup> The persons identified represent only a partial list and many are listed below. Twenty-four persons targeted for recruitment remain uncorroborated as to it being accomplished. These individuals are marked with an asterisk (*).<sup id="fn_4_back">]</sup>


==Document Release Issues== ==Document Release Issues==
Line 33: Line 56:
The release of VENONA translations involved careful consideration of the privacy interests of individuals mentioned, referenced, or identified in the translations. Some names have not been released when to do so would constitute an invasion of privacy. <sup id="fn_5_back">]</sup> The release of VENONA translations involved careful consideration of the privacy interests of individuals mentioned, referenced, or identified in the translations. Some names have not been released when to do so would constitute an invasion of privacy. <sup id="fn_5_back">]</sup>


The NSA has failed to release the VENONA documents as a Unicode based PDF text file. Text processing technology could be used to extract information from the decrypts for historical research if the VENONA documents were released in PDF form. The NSA has failed to release all the VENONA documents as machine-readable text files. (Text processing technology could be used to extract information from the decrypts for historical research if the VENONA documents were released in this form.)


The NSA website states: The NSA website states:
Line 39: Line 62:


"individuals may request that the government provide auxiliary aids or services to ensure effective communication of the substance of the documents. For such requests, please contact the Public Affairs Office at 301-688-6524." "individuals may request that the government provide auxiliary aids or services to ensure effective communication of the substance of the documents. For such requests, please contact the Public Affairs Office at 301-688-6524."




==See also== ==See also==
Line 66: Line 87:


===Additional background material=== ===Additional background material===
* ], ''Secrecy: The American Experience'' (New Haven: Yale University Press 1998) ISBN 0300080794

* ''The Hidden Hand: Britain, America, and Cold War Secret Intelligence''; by Richard J. Aldrich. New York: Overlook Press, 2002. ISBN 1585672742. * Richard J. Aldrich, ''The Hidden Hand: Britain, America, and Cold War Secret Intelligence'' (New York: Overlook Press, 2002) ISBN 1585672742.
* ''Body of Secrets: Anatomy of the Ultra-Secret National Security Agency''; by James Bamford. Anchor Books. ISBN 0385499086. See also the same author's earlier, ''The Puzzle Palace'', also about the NSA. * ], ''Body of Secrets: Anatomy of the Ultra-Secret National Security Agency'' (Anchor Books) ISBN 0385499086. (See also the same author's earlier, ''The Puzzle Palace'', also about the NSA.)
* Albright and Kunstel, ''Bombshell'' - About Soviet WWII espionage in the US, including Venona.
* ''Secrecy: The American Experience''; by Daniel Patrick Moynihan, New Haven: Yale University Press 1998. ISBN 0300080794.
* Steven Budiansky, ''Battle of Wits'' - An overview in one volume of cryptography in WWII.
* ''Bombshell''; by Albright and Kunstel. About Soviet WWII espionage in the US, including Venona.
* ''Battle of Wits''; by Steven Budiansky. An overview in one volume of cryptography in WWII.


==External links== ==External links==

Revision as of 16:21, 19 August 2005

Template:Totallydisputed

The VENONA project was a long-running and highly secret collaboration between United States intelligence agencies and the United Kingdom's MI5 that involved the cryptanalysis of messages sent by several Soviet intelligence agencies. The British codename for VENONA was Bride.

In the early years of the Cold War, it was one of the West's most important sources on Soviet intelligence activity, and although unknown to the public, it was important in many famous events of the early Cold War, such as the case of Elizabeth Bentley, and the Rosenberg case.

Background

U.S. Army Signal Security Agency (commonly called Arlington Hall) codebreakers had intercepted large volumes of encrypted high-level Soviet diplomatic and intelligence traffic during and immediately after World War II. The British had stopped intercepting Soviet traffic, at Winston Churchill's orders, shortly after Germany invaded the Soviet Union in 1941, and had no traffic to contribute to the project after that time.

This traffic, some of which was thought to be encrypted with a one-time pad system, was stored and analyzed in relative secrecy by hundreds of cryptanalysts over a 40-year period starting in the early 1940s. Due to what turned out to be a serious blunder on the part of the Soviets - re-using pages of some of the one-time pads in other pads, which were then used for other messages - this traffic was vulnerable to crypanalysis.

The Venona Project was initiated under orders from the Chief of Military Intelligence, Carter Clarke, who mistrusted Joseph Stalin. He feared that Stalin and Hitler would sign a peace treaty in order to focus Germany's military forces on the destruction of Great Britain and the U.S.

The breakin

The Soviet systems in general used a code to convert words and letters into numbers, to which an additive key (from one-time pads) were added, further disguising the content. Some brilliant cryptanalysis by American and British codebreakers revealed that some of the one-time pad material had incorrectly been reused by the Soviets (specifically, entire pages, although not complete books), which allowed decryption (sometimes only partial) of a small part of the traffic.

It was Arlington Hall's Lt. Richard Hallock, working on Soviet "Trade" traffic, who first discovered that the Soviets were re-using pages. Hallock and his colleagues (including Genevieve Feinstein, Cecil Phillips, Frank Lewis, Frank Wanat, and Lucille Campbell) went on to break into a significant amount of "Trade" traffic, recovering many one-time pad additive key tables in the process.

A very young Meredith Gardner (of what would become the National Security Agency) then used this material to break in to what turned out to be NKVD (and later GRU) traffic, by reconstructing the code used to convert text to numbers. Samuel Chew and Cecil Phillips also made valuable contributions. Very slowly, using assorted techniques ranging from traffic analysis to defector information, more of the messages were decrypted.

Claims have been made that information from physical theft of code books (a partially burned one was recovered by the Finns) to bugging embassy rooms in which text was entered into encrypting devices (analyzing the keystrokes by listening to them being punched in), contributed to achieving as much plaintext as was recovered. These latter claims are less than fully supported in the open literature.

One significant aid (mentioned by the NSA) in the early stages may have been work done in co-operation between the Japanese and Finnish cryptanalytic organizations; when the Americans broke into Japanese codes during WWII, they gained access to this information.

There has been speculation that the reason for the key material duplication was the increase in work (including key pad generation) in the period after the German attack in June of 1941. Other suggestions have it that it was Guderian's tanks just outside Moscow in early December that year which forced Moscow Centre to make such a fundamental error.

Results

The NSA reported that, according to the serial numbers of the Venona cables, thousands were sent, but only a fraction were available to the cryptanalysts. Approximately 2,200 of the messages were decrypted and translated; some 50 percent of the 1943 GRU-Naval Washington to Moscow messages were broken, but none for any other year, although several thousand were sent between 1941 and 1945. The decryption rate of the NKVD cables was:

  • 1942 1.8%
  • 1943 15.0%
  • 1944 49.0%
  • 1945 1.5%

Out of some hundreds of thousands of intercepted cyphertexts, it is claimed that under 3000 have been partially or wholly decrypted.

The Soviets eventually stopped reusing key pad material, possibly after learning of the US / British work from several of their agents, after which their one-time pad traffic reverted to completely unreadable.

Significance

Main articles: Significance of Venona

The NSA followed Soviet intelligence traffic for only a few years in World War II, and decrypted only a small portion of that traffic. The Venona project was a thirty-eight year investigation conducted by the NSA and FBI counterintelligence, and held classified for an additional fifteen years after the program ended. Researchers, historians, and the public, struggle to understand its significance and meaning.

Senator Daniel Patrick Moynihan, Chairman of the bipartisan Commission on Government Secrecy was responsible for securing the release of Venona project materials. He wrote "The Venona intercepts contained overwhelming proof of the activities of Soviet spy networks in America, complete with names, dates, places, and deeds." (*).

List of Americans in the Venona Papers

See also: List of Americans in the Venona Papers

349 U.S. citizens, noncitizen immigrants, and permanent residents of the United States who had covert relationships with Soviet intelligence were confirmed in the Venona traffic. Of these 171 are identified by true names and 178 are known only by a cover name. The persons identified represent only a partial list and many are listed below. Twenty-four persons targeted for recruitment remain uncorroborated as to it being accomplished. These individuals are marked with an asterisk (*).

Document Release Issues

The release of VENONA translations involved careful consideration of the privacy interests of individuals mentioned, referenced, or identified in the translations. Some names have not been released when to do so would constitute an invasion of privacy.

The NSA has failed to release all the VENONA documents as machine-readable text files. (Text processing technology could be used to extract information from the decrypts for historical research if the VENONA documents were released in this form.)

The NSA website states: "These historical documents are GIF images of formerly classified carbon paper and reports that have been declassified. Due to the age and poor quality of some of the GIF images, a screen reader may not be able to process the images into word documents."

"individuals may request that the government provide auxiliary aids or services to ensure effective communication of the substance of the documents. For such requests, please contact the Public Affairs Office at 301-688-6524."

See also

Notes

References

Further reading

  • Robert Louis Benson, Michael Warner, Venona: Soviet Espionage and the American Response 1939-1957 (National Security Agency and Central Intelligence Agency, Washington D.C., 1996)
  • Robert Louis Benson, The Venona Story (National Security Agency, Center for Cryptologic History, 2001)
  • John Earl Haynes, Harvey Klehr, Venona: Decoding Soviet Espionage in America (Yale University, New Haven, 1999)
  • Nigel West, Venona: The Greatest Secret of the Cold War (HarperCollins, London, 1999)

Additional background material

  • Daniel Patrick Moynihan, Secrecy: The American Experience (New Haven: Yale University Press 1998) ISBN 0300080794
  • Richard J. Aldrich, The Hidden Hand: Britain, America, and Cold War Secret Intelligence (New York: Overlook Press, 2002) ISBN 1585672742.
  • James Bamford, Body of Secrets: Anatomy of the Ultra-Secret National Security Agency (Anchor Books) ISBN 0385499086. (See also the same author's earlier, The Puzzle Palace, also about the NSA.)
  • Albright and Kunstel, Bombshell - About Soviet WWII espionage in the US, including Venona.
  • Steven Budiansky, Battle of Wits - An overview in one volume of cryptography in WWII.

External links

Categories: