What is Open-Source Intelligence?
In this blog, Fivecast CEO and Co-founder, Dr. Brenton Cooper, goes back to first principles to define open-source intelligence (OSINT) and outline the OSINT trends and innovations that are supporting successful intelligence missions worldwide.
REGISTER FOR OUR INTEL HUB TO ACCESS EXCLUSIVE OSINT EDUCATIONAL CONTENT
Open-Source Intelligence Defined
Open-Source Intelligence, often referred to as a subset of digital intelligence, is the process of collecting, analyzing, and extracting meaningful insights from publicly available data sources including social media, news feeds, blog sites, discussion forums, and more. The constant growth and dynamic nature of the World Wide Web means that open-source intelligence must span across the Surface Web, the Deep Web, and the Dark Web and the many different data feeds, content and platforms that these support. The growth of open-source data is exponential and will be for the foreseeable future. As a result, it is now essential for intelligence teams – both government and corporate – to leverage technical solutions that automate data collection and analysis in addition to their own analytical expertise and tradecraft to progress intelligence missions.
Use Cases:
OSINT solutions, methodologies, and tradecraft are used for an increasingly diverse range of use cases spanning but not limited to:
- Counter-terrorism
- Violent Extremist Threat Monitoring
- Security Vetting
- Insider Threat Detection
- Identifying Transnational Organized Crime
- Anti Drug, Human & Weapons Trafficking
- Monitoring Organized Crime
- Corporate Security
- Fraud Detection
- AML/CFT
The Big Data Challenge
Extracting useful intelligence from open-source data is increasingly challenging for analysts because of the constantly increasing volume, velocity, and variety of data available.
- Data Volume – The reality of our increasingly online lives has resulted in a tsunami of data feeds being available for intelligence purposes. While this means potentially more useful data is available to analysts at their fingertips, it makes the task of monitoring and filtering data to identify what’s important extremely difficult and time-consuming.
- Data Velocity – The warp speed at which topics of conversations arise, gain popularity and then die off again across platforms significantly increases the complexity of developing an intelligence picture.
- Data Variety – The unlimited variety of images, videos, posts, quotes, texts, all in multiple languages and posted across increasingly diverse and constantly changing platforms adds serious complexity to the challenge.
Open-source data collection and analysis is now a task beyond human scale. The processes supporting open-source intelligence must be automated and, increasingly, enhanced with Artificial Intelligence (AI) and machine learning to support the collection, monitoring, and analysis of masses of both structured and unstructured data.
Open-Source Intelligence Challenges
The increasing prevalence and complexity of global threats and risks across law enforcement, defense, national, and corporate security sectors demand more sophisticated and innovative intelligence solutions. The above trends in big data present many challenges to intelligence teams and analysts and these are exacerbated by the complexity of so many intelligence missions.
Let’s take the example of an analyst responsible for monitoring and evaluating the threat associated with an extremist group. This analyst needs to understand a myriad of contextual factors including:
- Key individuals in the group
- The group’s activities
- Capabilities & intents of individuals & the group as a whole
- The harm that group has the potential to inflict
- Other organizations interacting with the group
- Timelines of events
- The group leadership, recruiters, followers and motivators.
All of this intelligence needs to be extracted in an environment with hundreds of different data feeds across many different platforms and analysts also need to understand the changing activity over time. And, this is for just one extremist group. It’s an extremely challenging task!
The history of Fivecast can be traced to this very challenge – the capabilities of our flagship digital intelligence solution, Fivecast ONYX, were developed through a joint project which brought together government departments and agencies from Australia and the United States, with world-leading research institutions to tackle the big data challenges facing national security and law enforcement.
Trends and Applications
With Fivecast ONYX, the technical capabilities of OSINT solutions have evolved alongside the exponential increases in data sources and complexity. Forward-looking and innovative analyst teams are leveraging best practices in open-source intelligence technology and tradecraft to overcome the data tsunami and extract meaningful insights. Here are some of the important OSINT trends and capabilities that can be deployed to gain insight across the threat landscape to find that “needle in the haystack”:
Targeted & Detailed Data Collection:
Moving beyond keyword searches to proactively collect detailed ‘entity-centric’ data enables analysts to develop a rich & full picture of online activities related to an investigation. It also supports more granular targeting of data that is of direct relevance to an investigation, without drawing in masses of unrelated ‘noise’.
Change Detection and Trend Analysis:
Deploying scalable data collection techniques that are ongoing, repeatable, and consistent helps make sense of the data tsunami and leads to an understanding of trends and changes over time. Additionally, tracking the sentiment and emotion of online activity is another important capability and predictor of risk.
Contextualized Insights:
Network analysis capabilities that link individuals, groups, and behaviors from masses of unrelated data prove critical in enabling analysts to identify relationships and form an enriched intelligence picture. Being able to correlate and compare data from a broad range of platforms across the Surface, Deep, and Dark Web to uncover insights not available from a single data source is a game changer for quickly progressing intelligence missions.
AI and Machine Learning:
While AI and machine learning is now table stakes for open-source intelligence, it is being applied in new and innovative ways to identify risk. Customizable risk detector frameworks that apply AI across masses of multi-media data and learn and increase capabilities over time help to quickly zero in on the most important content.
Fivecast ONYX – Open-Source Intelligence
Fivecast works globally with many customers in law enforcement, defense, national security, and the corporate sector to address the above challenges with open-source intelligence best practices. Our signature solution, Fivecast ONYX, delivers advanced data collection across open-source platforms and AI-enabled risk analytics. Delivering more than monitoring, the unique power of the solution is in its capacity to not only explore unprecedented amounts of digital data but to provide deep and actionable insights.
Fivecast ONYX is the force multiplier your agency needs. It is a completely scalable solution, with the power to seamlessly and rapidly increase vision of your targets, without needing more skilled staff to carry the load.
