Source validation is the eternal battle of intelligence analysts worldwide, irrespective of your intelligence discipline and background. Whether you focus on Human Intelligence (HUMINT), Open-Source Intelligence (OSINT), Financial Intelligence (FININT), or anything in between, the challenge of validating our sources and identifying whether circular reporting is present is an ongoing challenge.
The validity and reliability of the intelligence source and information can be determined using the source validity and information credibility evaluation matrix – a relatively universal scale that most will love and know, with ratings from A1 (reliable and confirmed by independent sources) through F6 (reliability unknown and validity of the information cannot be determined). However, the process of getting to the information validity is not always fully understood by the end user who receives it.
Learn more:
Read our Industry Brief – Ethics & OSINT; Navigating Publicly Available Information
Misinformation instances we see in social media are often referred to as ‘fake news’ or ‘echo chambers.’ Identifying these instances as analysts is essential to ensure our data is accurate for intelligence products.
When evaluating the validity and reliability of sources, we need to develop a critical mindset. There are three rules to apply when evaluating sources:
- Firstly, we must be professional and unbiased, ensuring that our personal feelings do not influence our judgment and the evaluation of the source.
- Secondly, we need to evaluate the source separately from the data collected. This builds on the first principle and ensures we aren’t influenced by the data, either positively or negatively.
- Thirdly, when performing the validation, we need to do this as close to the source as possible to ensure the results are as accurate as possible.
Performing an evaluation of a source can be summarized by following the rules mentioned above and using a number of principles.
These principles can be used to help guide us when assessing sources and are helpful for intelligence analysts from any background, but I will relate this to OSINT in this instance. The first being to check the author if there is one. The lack of an author for OSINT should not be an instant dismissal as a valid source of information but should definitely cause us to pause and consider why they wouldn’t want their name associated with the information.
Identifying Fake news and misinformation
If the author is identifiable, we must assess the motivation behind the information. Are they trying to inform their audience of something, or are they trying to persuade them? If we can accurately identify their motivation, we can develop an inference of the information’s purpose.
The next step is to identify if the author has any other pieces of information they have published- and if they have, what is the nature of the sites it has been published on? Strange sounding extensions such as .infonet and .offer as opposed to .com could indicate the source itself is suspect.
Some authors who attempt to spread fake information have been known to create fake websites in an attempt to give the information more credit.
Another consideration is the credentials of the author. Are they known for their expertise or knowledge on the subject? Are they well-known in the industry? Or, conversely, has no one ever heard of them before? This, in particular, can relate back to the validity of the data in question.
The publication date of the information is also relevant. Given how fluid and dynamic online environments are, old and outdated information may be of concern. The lack of a date may also raise concerns that the information was not properly published. This can, in turn, make it difficult to verify its accuracy and when the information was reported.
Facts, Opinion or Propaganda- Uncovering the difference
It is also recommended to see who else is reporting on the story, has anyone else picked up on it or are we looking at single-source reporting? If others are commenting on it, what are they saying? This can also help us to understand the objectivity behind the author. If the data is based in fact, it can give the author more credit, however, if we are seeing elements of opinion it could well be propaganda. This can be readily seen in certain larger news outlets that have a particular political leaning on topics.
Objectivity can be assessed by looking at the language being used. If the language is free of emotion-rousing words or bias it can further help us to understand the intent of the content.
Websites dedicated to ‘fact checking’ can be used as a source of validation, but these should be reviewed as well to identify any evidence of bias. Even these fact-checkers will show obvious leans towards political bias, at times going out of its way to justify certain statements and conversely putting smaller amounts of effort into fact-checking other statements for legitimacy.
Many of these sites like Poynter and FactCheck can be analyzed by looking into the authors various articles and seeing the context behind what they have been reporting or fact checking. The same principles of validation can be applied to these fact-checkers in order to identify if they are worth relying on.
The importance of source validation
While the process of validating a source can be seen as exhaustive, it is an important part of the intelligence process and shouldn’t be overlooked or rushed. Following the rules by developing a critical mindset, analyzing the source separately from the data and evaluating as close to the source as possible will ensure analysts yield accurate and valid data for intelligence products.
These rules apply to evaluating OSINT data just as readily as HUMINT or other intelligence disciplines, the nuance sits in how we use these rules to ensure our intelligence products are the best provided to our stakeholders.
