Misplaced Pages

Open-source intelligence: Difference between revisions

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
Browse history interactively← Previous editContent deleted Content addedVisualWikitext
Revision as of 09:08, 8 August 2006 editAlphachimpbot (talk | contribs)100,435 editsm BOT - moving cia links to https, Replaced: http://www.cia.gov → https://www.cia.gov,← Previous edit Latest revision as of 21:53, 3 January 2025 edit undoBlysse (talk | contribs)Extended confirmed users622 edits →Risks for practitioners: I don't see why that needed quotes and brackets?Tag: Visual edit 
(790 intermediate revisions by more than 100 users not shown)
Line 1: Line 1:
{{Short description|Data collected from publicly available sources to be used in an intelligence context}}{{Multiple issues|
{{Globalize/USA}}
{{Globalize|article|USA|2name=the United States|date=December 2010}}
{{Update | date=April 2017}}
{{More citations needed|date=October 2024}}
}}
'''Open source intelligence''' ('''OSINT''') is the collection and analysis of data gathered from open sources (overt sources and publicly available information) to produce actionable intelligence. OSINT is primarily used in ], ], and ] functions and is of value to analysts who use non-sensitive intelligence in answering ], ], or ] ] across the previous intelligence disciplines.<ref name="row-2022">{{cite web |last1=Schwartz |first1=Leo |title=Amateur open source researchers went viral unpacking the war in Ukraine |url=https://restofworld.org/2022/osint-viral-ukraine/ |website=Rest of World |access-date=8 March 2022 |date=March 7, 2022}}</ref>


== Categories ==
'''Open Source Intelligence''' ('''OSINT''') is an ] that involves collecting information from open sources and analyzing it to produce usable ]. In the intelligence community, the term "open" refers to overt, publicly available sources (as opposed to covert or classified sources); it is not related to ]. OSINT includes a wide variety of information and sources:
OSINT sources can be divided up into six different categories of information flow:<ref>{{Cite book|title=The US Intelligence Community |isbn=978-0813349183 |last1=Richelson |first1=Jeffrey |year=2016 |publisher=Avalon }}</ref>
:* Media - newspapers, magazines, radio, television, and computer-based information.
:* Public data - government reports, official data such as budgets and demographics, hearings, legislative debates, press conferences, speeches.
:* Professional and academic - conferences, symposia, professional associations, academic papers, and subject matter experts.<ref>Lowenthal, Mark M. "Intelligence: From Secrets to Policy," 2nd Ed. (Washington D.C.: CQ Press, 2003) p. 79.</ref>
:*Most information has geospatial dimensions, but many often overlook the geospatial side of OSINT: not all open source data is unstructured text. Examples of geospatial open source include hard and softcopy maps, atlases, gazetteers, port plans, gravity data, aeronautical data, navigation data, geodetic data, human terrain data (cultural and economic), environmental data, commercial imagery, ], hyper and multi-spectral data, airborne imagery, geo-names, geo-features, urban terrain, vertical obstruction data, boundary marker data, geospatial ], spatial databases, and ].


*Media: print ], ], ], and ] from across and between countries.
The caricature of an open source analyst or practitioner is often of one who crawls the net reading reports and news articles (unstructured data). Most of the geospatial data mentioned above is integrated, analyzed, and syndicated using geospatial software like a ] (GIS) not a browser ''per se''.
*]: ]s, ], ]s, citizen media (i.e. – cell phone ], and ]), ], and other ] websites (i.e. – ], ], ], etc.). This source also outpaces a variety of other sources due to its timeliness and ease of access.
*Public government data: public government reports, budgets, hearings, ], press conferences, websites, and speeches. Although this source comes from an official source they are publicly accessible and may be used openly and freely.
*Professional and academic publications: information acquired from ]s, conferences, symposia, ], dissertations, and theses.
*Commercial data: ], financial and industrial assessments, and databases.
*]: technical reports, ]s, patents, working papers, business documents, unpublished works, and ]s.


OSINT is distinguished from research in that it applies the ] to create tailored knowledge supportive of a specific decision by a specific individual or group. OSINT is distinguished from research in that it applies the ] to create tailored knowledge supportive of a specific decision by a specific individual or group.<ref>{{cite web |title=Spy Agencies Turn to Newspapers, NPR, and Misplaced Pages for Information: The intelligence community is learning to value 'open-source' information |url=https://www.usnews.com/articles/news/national/2008/09/12/spy-agencies-turn-to-newspapers-npr-and-wikipedia-for-information.html |access-date=2008-09-15 |archive-url=https://web.archive.org/web/20121023193900/http://www.usnews.com/news/national/articles/2008/09/12/spy-agencies-turn-to-newspapers-npr-and-wikipedia-for-information |archive-date=2012-10-23 |url-status=live }}</ref>


== OSINT collection methodologies ==
OSINT is defined by the ] (DoD), as "information of potential ] value that is available to the general public".<ref></ref>
Collecting open-source intelligence is achieved in a variety of different ways,<ref>{{Cite web |last=Leos |first=Devan |date=2023-02-28 |title=Thinking Like a Spy: How Open Source Intelligence Can Give You a Competitive Advantage |url=https://www.entrepreneur.com/growing-a-business/thinking-like-a-spy-how-open-source-intelligence-can-give/444634 |access-date=2023-11-08 |website=Entrepreneur |language=en}}</ref> such as:


* ] Intelligence, which is acquired from viewing or observing a subjects online social profile activity.
OSINT is, as of 2005, defined by the U.S. ] under the category of "Forces And Direct Support" and specifically for the DoD under Commercial Code M320 as <ref></ref>
* ] data mining or scraping.
:''Open Source Intelligence (OSINT) Collection/Processing''
* ] checking.
A wide variety of vendors sell information products specifically within this category.
* Information matching and verification from ] services.


==Value== ==Definition==
OSINT is defined in the United States of America by Public Law 109-163 as cited by both the U.S. ] and the U.S. ] (DoD), as intelligence "produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement."<ref>{{Cite web |url=http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=109_cong_public_laws&docid=f:publ163.109 |title=As defined in Sec. 931 of Public Law 109-163, entitled, "National Defense Authorization Act for Fiscal Year 2006." |access-date=2006-12-08 |archive-url=https://web.archive.org/web/20081112002134/http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=109_cong_public_laws&docid=f:publ163.109 |archive-date=2008-11-12 |url-status=live }}</ref> As defined by NATO, OSINT is intelligence "derived from publicly available information, as well as other unclassified information that has limited public distribution or access."<ref>{{Cite web|title=NATOTermOTAN|url=https://nso.nato.int/natoterm/content/nato/pages/home.html|access-date=2021-04-02|website=nso.nato.int}}</ref>
{{advert}}
The secret intelligence world, which has resisted any significant expenditures on OSINT for the past fifty years, is finally beginning to slowly adapt to the modern world. According to the ] report submitted in March 2005, OSINT must be included in the all-source SECRET intelligence process for the following reasons (as stated in the report):


According to political scientist ], “open source acquisition involves procuring verbal, written, or electronically transmitted material that can be obtained legally. In addition to documents and videos available via the ] or provided by a human source, others are obtained after U.S. or allied forces have taken control of a facility or site formerly operated by a foreign government or ] group.”<ref>{{cite book|last1=Richelson|first1=Jeffrey T|author-link1=Jeffrey T. Richelson|title=The U.S. Intelligence Community|publisher=]|isbn=9780813349190|url=https://books.google.com/books?id=4sxVDgAAQBAJ&pg=PT284|access-date=15 May 2017|language=en|date=2015-07-14}}</ref>
#The ever-shifting nature of our intelligence needs compels the IC to quickly and easily understand a wide range of foreign countries and cultures. - … today’s threats are rapidly changing and geographically diffuse; it is a fact of life that an intelligence analyst may be forced to shift rapidly from one topic to the next. Increasingly, IC professionals need to quickly assimilate social, economic, and cultural information about a country—information often detailed in open sources.
#Open source information provides a base for understanding classified materials. Despite large quantities of classified material produced by the IC, the amount of classified information produced on any one topic can be quite limited, and may be taken out of context if viewed only from a classified-source perspective. Perhaps the most important example today relates to terrorism, where open source information can fill gaps and create links that allow analysts to better understand fragmented intelligence, rumored terrorist plans, possible means of attack, and potential targets.
#Open source materials can protect sources and methods. Sometimes an intelligence judgment that is actually informed with sensitive, classified information can be defended on the basis of open source reporting. This can prove useful when policymakers need to explain policy decisions or communicate with foreign officials without compromising classified sources.
#Only open source can ''store history.'' A robust open source program can, in effect, gather data to monitor the world’s cultures and how they change with time. This is difficult, if not impossible, using the ''snapshots'' provided by classified collection methods.<ref> (The Commission on the Intelligence Capabilities, 378-379). ]</ref>


Former Assistant Director of Central Intelligence for Analysis ] defines OSINT as “any and all information that can be derived from overt collection: all types of media, government reports and other documents, scientific research and reports, commercial vendors of information, the Internet, and so on. The main qualifiers to open-source information are that it does not require any type of clandestine collection techniques to obtain it and that it must be obtained through means that entirely meet the copyright and commercial requirements of the vendors where applicable."<ref name="Lowenthal">{{Citation|last=Lowenthal|first=Mark M.|editor1-last=George|editor1-first=Roger Z|editor2-last=Kline|editor2-first=Robert D|title=Intelligence and the national security strategist : enduring issues and challenges|chapter=Open-Source Intelligence: New Myths, New Realities|year=2005|publisher=]|location=Lanham|isbn=9780742540392|chapter-url=https://books.google.com/books?id=jJxyIb2hN4MC&pg=PA273|language=en}}</ref>
==Process==
Information collection in OSINT is generally a different problem from collection in other intelligence disciplines where obtaining the raw information to be analyzed may be a major difficultly, particularly if it is to be obtained from non-cooperative targets. In OSINT, the chief difficulty is in identifying relevant, reliable sources from the vast amount of publicly available information. However, this is not as great a challenge for those who know how to access local knowledge and how to leverage human experts who can create new tailored knowledge on the fly.


==History== ==History==
]]]
{{POV-section}}
OSINT practices have been documented as early as the mid-19th century in the United States and early 20th century in the United Kingdom.<ref>{{Cite journal |last=Block |first=Ludo |date=2023 |title=The long history of OSINT |journal=Journal of Intelligence History |volume=23 |issue=2 |pages=95–109 |language=en |doi=10.1080/16161262.2023.2224091 |issn=1616-1262|doi-access=free |hdl=1887/3731669 |hdl-access=free }}</ref>
In the fall of 1992, Senator ], then Chairman of the ], sponsored the National Security Act of 1992, attempting to achieve modest reform in the U.S. Intelligence Community. His counterpart on the House Permanent Select Committee on Intelligence was Congressman ]. The House version of the legislation included a separate Open Source Office, at the suggestion of Larry Prior, a Marine Reservist familiar with the MCIC experience and then serving on the House Permanent Select Committee on Intelligence staff.


OSINT in the ] traces its origins to the 1941 creation of the ], an agency responsible for the monitoring of foreign broadcasts. An example of their work was the correlation of changes in the price of oranges in Paris with successful bombings of railway bridges during ].<ref name="BornnMil">{{cite web|last1=Bornn|first1=D Marshall|title=Service members, civilians learn to harness power of 'Open Source' information|url=https://www.army.mil/article/94007/Service_members__civilians_learn_to_harness_power_of__Open_Source__information|website=www.army.mil|access-date=14 May 2017|language=en|date=9 Jan 2013|archive-url=https://web.archive.org/web/20171209101817/https://www.army.mil/article/94007/Service_members__civilians_learn_to_harness_power_of__Open_Source__information|archive-date=9 December 2017|url-status=live}}</ref>
The Aspin-Brown Commission stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and DCI attention.


The ] stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and ] attention.<ref>{{Cite book|title=The Five Disciplines of Intelligence Collection|last1=Lowenthal|first1=Mark|last2=Clark|first2=Robert|publisher=CQ Press|year=2015|isbn=978-1483381114|page=18}}</ref>
In issuing its July 2004 report, the ] recommended the creation of an open-source intelligence agency, but without further detail or comment.<ref>See page 413 of the .</ref> Subsequently, the ] (also known as the Robb-Silberman Commission) report in March 2005 recommended the creation of an Open Source Directorate at the CIA.


In July 2004, following the ], the ] recommended the creation of an open-source intelligence agency.<ref>See page 413 of the {{Webarchive|url=https://web.archive.org/web/20070705012031/http://www.9-11commission.gov/report/911Report_FM.pdf |date=2007-07-05 }}.</ref> In March 2005, the ] recommended<ref name=":0">{{Cite news|last=McLaughlin|first=Michael|date=June 2012|title=Using open source intelligence for cybersecurity intelligence|language=en-GB|work=ComputerWeekly.com|url=https://www.computerweekly.com/tip/Using-open-source-intelligence-software-for-cybersecurity-intelligence|url-status=live|access-date=2018-06-29|archive-url=https://web.archive.org/web/20180629155103/https://www.computerweekly.com/tip/Using-open-source-intelligence-software-for-cybersecurity-intelligence|archive-date=2018-06-29}}</ref> the creation of an open-source directorate at the CIA.
Following these recommendations, in November 2005 the ] announced the creation of the DNI ]. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery."<ref>Office of the Director of National Intelligence. "". Press release, ] ].</ref> In addition to collecting openly available information, it would train analysts to make better use of this information. The Center absorbed the ]'s previously existing ] (FBIS), originally established in 1941, with FBIS head ] named as director of the Center.<ref>Ensor, David. "". ''CNN'', ] ].</ref>


Following these recommendations, in November 2005 the ] announced the creation of the DNI ]. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery."<ref>Office of the Director of National Intelligence. " {{webarchive|url=https://web.archive.org/web/20060623072458/http://dni.gov/press_releases/20051108_release.htm |date=2006-06-23 }}". Press release, 8 November 2005.</ref> In addition to collecting openly available information, it would train analysts to make better use of this information. The center absorbed the ]'s previously existing ] (FBIS), originally established in 1941, with FBIS head Douglas Naquin named as director of the center.<ref>Ensor, David. " {{Webarchive|url=https://web.archive.org/web/20070325142701/http://www.cnn.com/2005/POLITICS/11/08/sr.tues/ |date=2007-03-25 }}". ''CNN'', 8 November 2005.</ref> Then, following the events of ] the ] merged FBIS and other research elements into the ] creating the ].
In December 2005, the Director of National Intelligence appointed ] as the ] to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the ]. <ref>Office of the Director of National Intelligence "". Press release, ] ].</ref>


Furthermore, the private sector has invested in tools which aid in OSINT collection and analysis. Specifically, ], a ] supported venture capital firm in Arlington, VA assisted companies develop web-monitoring and predictive analysis tools.
The Director of National Intelligence's Chief Information Officer, ] Ret., has broken with the past and sponsored both an open forum on open standards for information sharing, and a major conference open to foreigners, to discuss all aspects of the U.S. Intelligence Community's ] program for sharing and making sense of all sources of information. Note: the original Open Source Information System (OSIS) has been re-named ].


In December 2005, the Director of National Intelligence appointed ] as the Assistant Deputy Director of National Intelligence for Open Source to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the ].<ref>Office of the Director of National Intelligence " {{webarchive|url=https://web.archive.org/web/20060623072305/http://dni.gov/press_releases/20051207_release.htm |date=2006-06-23 }}". Press release, 7 December 2005.</ref> Mr. Jardines has established the National Open Source Enterprise<ref> {{webarchive|url=https://web.archive.org/web/20070928150640/http://upload.wikimedia.org/wikipedia/en/b/b4/NationalOpenSourceEnterprise.pdf |date=2007-09-28 }} May 2006</ref> and authored ]. In 2008, Mr. Jardines returned to the private sector and was succeeded by ] who is ADDNI/OS<ref> {{webarchive|url=https://web.archive.org/web/20100417020406/http://www.dniopensource.org/Conference/Agenda.aspx |date=2010-04-17 }}</ref> and previously Mr. Jardines' Senior Advisor for Policy.<ref> {{webarchive|url=https://web.archive.org/web/20080801081412/http://www.dniopensource2007.com/sessions.cfm |date=2008-08-01 }}</ref>
In February 2006 speech at the ], ] ] seems to have acknowledged the importance of open media as a component of national security in the ].<ref>{{cite web| url=http://www.cfr.org/publication/9900/| title=New Realities in the Media Age: A Conversation with Donald Rumsfeld| year=February 17, 2006| publisher=Council on Foreign Relations| accessdate=2006-07-06}}</ref>


=== Tools ===
{{Cleanup-section|July 2006}}
], as part of its intelligence platform.</nowiki>|thumb|Open source intelligence may be ingested to battle management systems such as CPCE by ], which uses an open source feed from ].]]
The web browser is a powerful OSINT tool that provides access to numerous websites and both open source and proprietary software tools that are either purpose-built for open source information collection or which can be exploited for the purposes of either gathering of open source information or to facilitate analysis and validation to provide intelligence. A cottage industry of both for-profit and not-for-profit investigative and educational groups such as ], IntelTechniques SANS and others offer indices, books, podcasts and video training materials on OSINT tools and techniques. Books such as Michael Bazzell's ''Open Source Intelligence Techniques'' serve as indices to resources across multiple domains but according the author, due to the rapidly changing information landscape, some tools and techniques change or become obsolete frequently, hence it is imperative for OSINT researchers to study, train and survey the landscape of source material regularly.<ref>{{Cite web |title=Books by Michael Bazzell |url=https://inteltechniques.com/book1.html |access-date=2022-05-02 |website=inteltechniques.com}}</ref> A guide by Ryan Fedasiuk, an analyst at the ], lists six tools open-source analysts can use to stay safe and utilize operational security (]) when conducting online investigations. These include ], cached webpages, ] services, URL and file scanners, browser sandbox applications, and ].<ref>{{Cite web |last=Fedasiuk |first=Ryan |date=2022-04-06 |title=Into the Jungle: Best Practices for Open-Source Researchers |url=https://cset.georgetown.edu/article/into-the-jungle-best-practices-for-open-source-researchers/ |access-date=2022-04-22 |website=Center for Security and Emerging Technology |language=en-US}}</ref>


Numerous lists of aggregated OSINT content are available on the web. The OSINT Framework contains over 30 primary categories of tools and is maintained as an open source project on ].<ref>{{Citation |title=OSINT Framework |date=2022-05-02 |url=https://github.com/lockfale/OSINT-Framework |publisher=lockFALE |access-date=2022-05-02}}</ref>
===Government===


== Risks for practitioners ==
Activities distinct from research & development in that they focus on decision-support.
A main hindrance to practical OSINT is the volume of information it has to deal with ]. The amount of data being distributed increases at a rate that it becomes difficult to ] in ]. To a small degree the work has sometimes been done by amateur crowd-sourcing.<ref>{{Cite magazine |title=Bellingcat's Eliot Higgins Explains Why Ukraine Is Winning the Information War |url=https://time.com/6155869/bellingcat-eliot-higgins-ukraine-open-source-intelligence/ |access-date=2022-04-13 |magazine=Time |language=en}}</ref>


Private individuals illegally collecting data for a foreign military or intelligence agency is considered ] in most countries. Of course, espionage that is not ] (e.g. betraying one's country of citizenship) has been a tool of statecraft since ancient times.<ref>] (]), '']'', Chapter 13: "Hostile armies may face each other for years, striving for the victory which is decided in a single day. This being so, to remain in ignorance of the enemy's condition simply because one grudges the outlay of 2 hundred ounces of silver in honors and emoluments, is the height of inhumanity."</ref>
Generic monitoring and other activities associated with the ], to take one prominent example of open source information capabilities, are not considered OSINT unless--as FBIS has done--they are directly supportive of specific needs such as the hunt for Bin Laden.


==Professional association==
The ] sponsors the ] (FRD}.
The OSINT Foundation is a professional association for OSINT practitioners in the United States Intelligence Community.<ref>{{Citation |title=New OSINT foundation aims to 'professionalize' open source discipline across spy agencies |url=https://federalnewsnetwork.com/inside-ic/2022/07/new-osint-foundation-aims-to-professionalize-open-source-discipline-across-spy-agencies/ |date=2022-07-27 |language=en-US}}</ref> It is open to U.S. Citizens and seeks to raise the prominence of the open-source intelligence discipline.<ref>{{Citation |title=New Group to Promote Open-Source Intelligence, Seen as Vital in Ukraine War | newspaper=Wall Street Journal |url=https://www.wsj.com/articles/new-group-to-promote-open-source-intelligence-seen-as-vital-in-ukraine-war-11658926800?st=q5j3a3ymqekavuj&reflink=desktopwebshare_permalink |date=2022-07-27 |language=en-US | last1=Volz | first1=Dustin }}</ref>

The ] produces useful compendiums that support legislative decision-making.

===Military===
The ], Dr. ] told the Security Affairs Support Association (SASA) in that he required universal coverage, 24/7, and the sub-state level of granularity. Encouraged in part by the ] reports on ] and ], he created the Defense Open Source Program (DOSP). As of 5 July 2006 executive agency for this program has not been assigned.

Within the U.S. military, there are a number of important OSINT activities, including:

*US Army Foreign Military Studies Office
*US Army Asia Studies Detachment
*Open Source Branch, Joint Intelligence Center, ]
*Foreign Media Monitoring in Support of Information Operations, ]

===Law enforcement===
'''Law Enforcement OSINT''' applies ] (OSINT) to the prediction, prevention, investigation, and procecution of criminals including terrorists.

Examples of successful law enforcement OSINT include ]; ] (RCMP) OSINT.

] and ] experimented with OSINT units for a time, but they appear to have atropied with the departure of their individual champions.

New York Police Department (NYPD) is known to have an OSINT unit.

===Academia===
'''Academic OSINT''' is best represented by the at ] in Erie, Pennsyvania.

Increasingly universities are finding that in addition to traditional research into the frontiers of science and social science, they can compete for contracts and grants to produce ]. An example of one successful academic OSINT enterprise focused on agricultural insurance fraud and other detectable patterns of abuse, is the ] at ] in ].

===Business===
Business OSINT encompasses ], ], and ].

In the course of collecting and exploiting relevant information for business purposes, ] and ] may be used.


==See also== ==See also==
{{div col|colwidth=20em}}
*]
*{{annotated link|Ashley Feinberg}}
*]
*{{annotated link|Bellingcat}}
*]
*{{annotated link|Co-occurrence networks}}
*]
*{{annotated link|Dan Butler (civil servant)}}
*]
*{{annotated link|DARPA TIDES program}}
*]
*{{annotated link|Doxing}}
*]
*{{annotated link|Eliot A. Jardines}}
*{{annotated link|Eliot Higgins}}
*{{annotated link|Fusion center}}
*{{annotated link|ICWatch}}
*{{annotated link|Intellipedia}}
*{{annotated link|Investigative Data Warehouse}}
*{{annotated link|MiTAP}}
*{{annotated link|National Intelligence Open Source Committee}}
*{{annotated link|NATO Open Source Intelligence Handbook}}, ]
*{{annotated link|Open data}}
*{{annotated link|Open Source Center}}
*{{annotated link|Oryx (blog)}}
*{{annotated link|Private intelligence agency}}
*{{annotated link|Social cloud computing}}
*{{annotated link|Special Libraries Association}}
*{{annotated link|Strategic intelligence}}
*{{annotated link|Open-source intelligence in the 2022 Russian invasion of Ukraine}}
{{Div col end}}


==References== ==References==
{{Reflist|2}}
<references/>
*, Washington Times – CIA mines 'rich' content from blogs, 19 April 2006
*, Government Computer News – Intelligence units mine the benefits of public sources 20 March 2006
*, Military Intelligence Professional Bulletin October–December, 2005 by Barbara G. Fast
*, Congressional Testimony on OSINT and Homeland Security 21 June 2005
*, When Everyone Can Mine Your Data by Taylor Buley, 11.21.08]
* {{cite web | last=Thompson | first=Clive | title=Open-Source Spying | website=The New York Times | date=2006-12-03 | url=https://www.nytimes.com/2006/12/03/magazine/03intelligence.html | access-date=2018-05-29}}
* {{cite web | title=Open Source Intelligence (OSINT) | website=RIS Open Source Intelligence | date=2018-05-29 | url=http://arnoreuser.com/ | ref={{sfnref | RIS Open Source Intelligence | 2018}} | access-date=2018-05-29}}
* {{cite web | title=The Intelligence Network : I n t r o d u c t i o n | website=intellnet.org | date=2008-05-09 | url=http://www.intellnet.org/ | archive-url=https://web.archive.org/web/20080509085418/http://www.intellnet.org/ | archive-date=2008-05-09 | url-status=unfit | ref={{sfnref | intellnet.org | 2008}} | access-date=2018-05-29}}
== Further reading ==
*{{cite book |author=United Nations Office of the High Commissioner for Human Rights |author-link=United Nations Office of the High Commissioner for Human Rights |author2=University of California, Berkeley Human Rights Center |author2-link=University Human Rights Centers#Human Rights Center – University of California, Berkeley |title=Berkeley Protocol on Digital Open Source Investigations A Practical Guide on the Effective Use of Digital Open Source Information in Investigating Violations of International Criminal, Human Rights and Humanitarian Law |url=https://www.ohchr.org/sites/default/files/2024-01/OHCHR_BerkeleyProtocol.pdf |publication-place=New York, Geneva |publisher=UN Office of the High Commissioner for Human Rights; Human Rights Center, UC Berkeley School of Law |year=2022 |isbn=978-92-1-154233-2 |oclc=1334608062}}


===Scientific publications===
==Further reading==
* Deneuville, A., Hernández López, G. & Rasmi, J. (Eds.) 'Contre-enquêtes en sources ouvertes'. ''Multitudes'', 89, 2022.
;General
* Arthur S. Hulnick: '', pages 229–241, The Oxford Handbook of National Security Intelligence, 2010
*
* {{Webarchive|url=https://web.archive.org/web/20160817134042/http://epublications.bond.edu.au/cgi/viewcontent.cgi?article=1010&context=cewces_papers |date=2016-08-17 }}
*
* *
* {{Cite book|last=Hassan, Nihad A.|url=https://www.worldcat.org/oclc/1043830928|title=Open source intelligence methods and tools : a practical guide to online intelligence|others=Hijazi, Rami|date=30 June 2018|isbn=978-1-4842-3213-2|location=|oclc=1043830928}}
*
*
*
*
* at ]
* - U.S. government arm focusing on open source intelligence under the DNI
* by "International Broadcaster" and vendor ]
*
*
* - a U.S. government ]


== External links ==
;Advocacy and analysis of OSINT
*
*
* Military Intelligence Professional Bulletin, Oct-Dec, 2005 by Jacob W. Kipp


*
;News and commentary
*
* 21 April 2006
*
* 19 April 2006
*
* 20 March 2006
*, Congressional Research Service, December 5, 2007
* March 2006
*, Congressional Research Service, January 28, 2008
* Oct-Dec, 2005 by Barbara G. Fast
*, FMSO-JRIC and Open Source Intelligence: speaking prose in a world of verse, Military Intelligence Professional Bulletin, Oct–Dec, 2005 by Jacob W. Kipp
* 21 June 2005
* by Stalder and Hirsh, 15 May 2002
* article ''Why Spy? '' by ] (cofounder of the ]) on why the government needs more OSINT.
*


{{Intelligence cycle management}}
]
{{Authority control}}


{{DEFAULTSORT:Open Source Intelligence}}
]
]
]
]
]
]
]
]
]
]
]
]
]

Latest revision as of 21:53, 3 January 2025

Data collected from publicly available sources to be used in an intelligence context
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
Globe icon.The examples and perspective in this article deal primarily with the United States and do not represent a worldwide view of the subject. You may improve this article, discuss the issue on the talk page, or create a new article, as appropriate. (December 2010) (Learn how and when to remove this message)
This article needs to be updated. Please help update this article to reflect recent events or newly available information. (April 2017)
This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.
Find sources: "Open-source intelligence" – news · newspapers · books · scholar · JSTOR (October 2024) (Learn how and when to remove this message)
(Learn how and when to remove this message)

Open source intelligence (OSINT) is the collection and analysis of data gathered from open sources (overt sources and publicly available information) to produce actionable intelligence. OSINT is primarily used in national security, law enforcement, and business intelligence functions and is of value to analysts who use non-sensitive intelligence in answering classified, unclassified, or proprietary intelligence requirements across the previous intelligence disciplines.

Categories

OSINT sources can be divided up into six different categories of information flow:

OSINT is distinguished from research in that it applies the process of intelligence to create tailored knowledge supportive of a specific decision by a specific individual or group.

OSINT collection methodologies

Collecting open-source intelligence is achieved in a variety of different ways, such as:

Definition

OSINT is defined in the United States of America by Public Law 109-163 as cited by both the U.S. Director of National Intelligence and the U.S. Department of Defense (DoD), as intelligence "produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement." As defined by NATO, OSINT is intelligence "derived from publicly available information, as well as other unclassified information that has limited public distribution or access."

According to political scientist Jeffrey T. Richelson, “open source acquisition involves procuring verbal, written, or electronically transmitted material that can be obtained legally. In addition to documents and videos available via the Internet or provided by a human source, others are obtained after U.S. or allied forces have taken control of a facility or site formerly operated by a foreign government or terrorist group.”

Former Assistant Director of Central Intelligence for Analysis Mark M. Lowenthal defines OSINT as “any and all information that can be derived from overt collection: all types of media, government reports and other documents, scientific research and reports, commercial vendors of information, the Internet, and so on. The main qualifiers to open-source information are that it does not require any type of clandestine collection techniques to obtain it and that it must be obtained through means that entirely meet the copyright and commercial requirements of the vendors where applicable."

History

Seal of the 9/11 Commission

OSINT practices have been documented as early as the mid-19th century in the United States and early 20th century in the United Kingdom.

OSINT in the United States traces its origins to the 1941 creation of the Foreign Broadcast Monitoring Service (FBMS), an agency responsible for the monitoring of foreign broadcasts. An example of their work was the correlation of changes in the price of oranges in Paris with successful bombings of railway bridges during World War II.

The Aspin-Brown Commission stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and DCI attention.

In July 2004, following the September 11 attacks, the 9/11 Commission recommended the creation of an open-source intelligence agency. In March 2005, the Iraq Intelligence Commission recommended the creation of an open-source directorate at the CIA.

Following these recommendations, in November 2005 the Director of National Intelligence announced the creation of the DNI Open Source Center. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery." In addition to collecting openly available information, it would train analysts to make better use of this information. The center absorbed the CIA's previously existing Foreign Broadcast Information Service (FBIS), originally established in 1941, with FBIS head Douglas Naquin named as director of the center. Then, following the events of 9/11 the Intelligence Reform and Terrorism Prevention Act merged FBIS and other research elements into the Office of the Director of National Intelligence creating the Open Source Enterprise.

Furthermore, the private sector has invested in tools which aid in OSINT collection and analysis. Specifically, In-Q-Tel, a Central Intelligence Agency supported venture capital firm in Arlington, VA assisted companies develop web-monitoring and predictive analysis tools.

In December 2005, the Director of National Intelligence appointed Eliot A. Jardines as the Assistant Deputy Director of National Intelligence for Open Source to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the National Open Source Enterprise. Mr. Jardines has established the National Open Source Enterprise and authored intelligence community directive 301. In 2008, Mr. Jardines returned to the private sector and was succeeded by Dan Butler who is ADDNI/OS and previously Mr. Jardines' Senior Advisor for Policy.

Tools

Systematic Software Engineering uses open source intelligence, compiled by ], as part of its intelligence platform.
Open source intelligence may be ingested to battle management systems such as CPCE by Systematic, which uses an open source feed from Janes Information Services.

The web browser is a powerful OSINT tool that provides access to numerous websites and both open source and proprietary software tools that are either purpose-built for open source information collection or which can be exploited for the purposes of either gathering of open source information or to facilitate analysis and validation to provide intelligence. A cottage industry of both for-profit and not-for-profit investigative and educational groups such as Bellingcat, IntelTechniques SANS and others offer indices, books, podcasts and video training materials on OSINT tools and techniques. Books such as Michael Bazzell's Open Source Intelligence Techniques serve as indices to resources across multiple domains but according the author, due to the rapidly changing information landscape, some tools and techniques change or become obsolete frequently, hence it is imperative for OSINT researchers to study, train and survey the landscape of source material regularly. A guide by Ryan Fedasiuk, an analyst at the Center for Security and Emerging Technology, lists six tools open-source analysts can use to stay safe and utilize operational security (OPSEC) when conducting online investigations. These include VPNs, cached webpages, digital archive services, URL and file scanners, browser sandbox applications, and antivirus software.

Numerous lists of aggregated OSINT content are available on the web. The OSINT Framework contains over 30 primary categories of tools and is maintained as an open source project on GitHub.

Risks for practitioners

A main hindrance to practical OSINT is the volume of information it has to deal with information explosion. The amount of data being distributed increases at a rate that it becomes difficult to evaluate sources in intelligence analysis. To a small degree the work has sometimes been done by amateur crowd-sourcing.

Private individuals illegally collecting data for a foreign military or intelligence agency is considered espionage in most countries. Of course, espionage that is not treason (e.g. betraying one's country of citizenship) has been a tool of statecraft since ancient times.

Professional association

The OSINT Foundation is a professional association for OSINT practitioners in the United States Intelligence Community. It is open to U.S. Citizens and seeks to raise the prominence of the open-source intelligence discipline.

See also

References

  1. Schwartz, Leo (March 7, 2022). "Amateur open source researchers went viral unpacking the war in Ukraine". Rest of World. Retrieved 8 March 2022.
  2. Richelson, Jeffrey (2016). The US Intelligence Community. Avalon. ISBN 978-0813349183.
  3. "Spy Agencies Turn to Newspapers, NPR, and Misplaced Pages for Information: The intelligence community is learning to value 'open-source' information". Archived from the original on 2012-10-23. Retrieved 2008-09-15.
  4. Leos, Devan (2023-02-28). "Thinking Like a Spy: How Open Source Intelligence Can Give You a Competitive Advantage". Entrepreneur. Retrieved 2023-11-08.
  5. "As defined in Sec. 931 of Public Law 109-163, entitled, "National Defense Authorization Act for Fiscal Year 2006."". Archived from the original on 2008-11-12. Retrieved 2006-12-08.
  6. "NATOTermOTAN". nso.nato.int. Retrieved 2021-04-02.
  7. Richelson, Jeffrey T (2015-07-14). The U.S. Intelligence Community. Avalon Publishing. ISBN 9780813349190. Retrieved 15 May 2017.
  8. Lowenthal, Mark M. (2005), "Open-Source Intelligence: New Myths, New Realities", in George, Roger Z; Kline, Robert D (eds.), Intelligence and the national security strategist : enduring issues and challenges, Lanham: Rowman and Littlefield, ISBN 9780742540392
  9. Block, Ludo (2023). "The long history of OSINT". Journal of Intelligence History. 23 (2): 95–109. doi:10.1080/16161262.2023.2224091. hdl:1887/3731669. ISSN 1616-1262.
  10. Bornn, D Marshall (9 Jan 2013). "Service members, civilians learn to harness power of 'Open Source' information". www.army.mil. Archived from the original on 9 December 2017. Retrieved 14 May 2017.
  11. Lowenthal, Mark; Clark, Robert (2015). The Five Disciplines of Intelligence Collection. CQ Press. p. 18. ISBN 978-1483381114.
  12. See page 413 of the 9-11 Commission Report (pdf) Archived 2007-07-05 at the Wayback Machine.
  13. McLaughlin, Michael (June 2012). "Using open source intelligence for cybersecurity intelligence". ComputerWeekly.com. Archived from the original on 2018-06-29. Retrieved 2018-06-29.
  14. Office of the Director of National Intelligence. "ODNI Announces Establishment of Open Source Center Archived 2006-06-23 at the Wayback Machine". Press release, 8 November 2005.
  15. Ensor, David. "The Situation Report: Open source intelligence center Archived 2007-03-25 at the Wayback Machine". CNN, 8 November 2005.
  16. Office of the Director of National Intelligence "ODNI Senior Leadership Announcement Archived 2006-06-23 at the Wayback Machine". Press release, 7 December 2005.
  17. "National Open Source Entreprise Vision Statement" Archived 2007-09-28 at the Wayback Machine May 2006
  18. DNI Open Source Conference 2008 "Decision Advantage" agenda, Office of the Director of National Intelligence, July 2008. Archived 2010-04-17 at the Wayback Machine
  19. DNI Open Source Conference 2007 "Expanding the Horizons" agenda, Office of the Director of National Intelligence, July 2007. Archived 2008-08-01 at the Wayback Machine
  20. "Books by Michael Bazzell". inteltechniques.com. Retrieved 2022-05-02.
  21. Fedasiuk, Ryan (2022-04-06). "Into the Jungle: Best Practices for Open-Source Researchers". Center for Security and Emerging Technology. Retrieved 2022-04-22.
  22. OSINT Framework, lockFALE, 2022-05-02, retrieved 2022-05-02
  23. "Bellingcat's Eliot Higgins Explains Why Ukraine Is Winning the Information War". Time. Retrieved 2022-04-13.
  24. Sun Tzu (Warring States period), The Art of War, Chapter 13: "Hostile armies may face each other for years, striving for the victory which is decided in a single day. This being so, to remain in ignorance of the enemy's condition simply because one grudges the outlay of 2 hundred ounces of silver in honors and emoluments, is the height of inhumanity."
  25. New OSINT foundation aims to 'professionalize' open source discipline across spy agencies, 2022-07-27
  26. Volz, Dustin (2022-07-27), "New Group to Promote Open-Source Intelligence, Seen as Vital in Ukraine War", Wall Street Journal

Further reading

Scientific publications

External links

Intelligence management
Collection
Human (HUMINT)
Clandestine
Espionage
Signals (SIGINT)
Measurement and
signature (MASINT)
Other
Analysis
Dissemination
Categories: