Imagine that you are the head of US military intelligence services immediately prior to the invasion of Iraq in 2003. You have a wealth of intelligence to help you find out whether Saddam Hussein really has weapons of mass destruction or not. You have access to satellite photos and huge amounts of information from spies and defectors. Each of these sources is fraught with uncertainty to a greater or lesser extent. Some pieces of evidence are more reliable than others.
The amount of information is so huge that nobody is capable of establishing a total overview. You therefore need computational tools to interpret all the information..
One of your hypotheses is that Saddam has weapons of mass destruction. The other hypothesis says the opposite. It is your job to determine which hypothesis appears to be the most correct one.
The head of the US intelligence services got it wrong. He erroneously determined that Saddam had access to weapons of mass destruction.
‘Current intelligence analyses are often based on information with a considerable degree of uncertainty. Intelligence analysts are constantly struggling with the reliability of circumstantial evidence. The sources may be unreliable or directly misleading. When intelligence services in one country attempt to find out what another country is planning to do, they need to take into account the credibility of the information, says Audun Jøsang, professor at the Department of Informatics, University of Oslo.
The fact that the information tends to be incomplete and the circumstantial evidence often is contradictory does not make things any easier.
‘It is therefore essential to assess all information, evidence, facts and circumstances in a way that reflects this situation.’
The new Logic
US ARMY: Professor Audun Jøsang has described a completely new logic that may improve intelligence services worldwide. The US Army is providing NOK 2 million in support for his research. Photo: Yngve Vogt
Audun Jøsang has developed a completely new type of mathematical logic that may improve the ability of intelligence services to deal with unclear evidence while pointing out intelligence areas that merit further investigation.
This new form of intelligence analysis is based on subjective logic. This type of logic can explicitly handle degrees of uncertainty, and this produces intelligence analysis models that are far more realistic than those produced by current interpretation methods.
In current intelligence analysis models, all circumstantial evidence must be weighted by a specific probability. Jøsang points out that this is not enough. He claims that an estimate of the certainty of this probability also must be included.
‘Most people are unaccustomed to the fact that a probability in itself may be uncertain.’
Certainty
Let us assume that an intelligence agent estimates the probability of two different events to be one half. Even though the probability is the same, their certainty and uncertainty may vary.
If you flip a coin, the probability of heads is one half. There is a very high certainty that this probability estimate is correct. Unless the coin is biased or manipulated, this certainty is one hundred per cent.
The probability that Oswald shot Kennedy in 1962 may also be estimated to one half, but this probability is fraught with a large margin of uncertainty. Even though the probability is set at one half here as well, there is a large uncertainty that the probability estimate might be totally wrong.
Quantifying uncertainty
Intelligence people struggle precisely with such things. They need to take uncertainty into account, but there are no tools available for undertaking intelligence analyses that consider the degree of uncertainty in each probability estimate.
With the aid of subjective logic, the certainty and uncertainty of all probability estimates can be quantified.
‘Unless this is done, the uncertainty is swept under the carpet. We humans are stuck in our preconceived notions and follow the beaten track. We are unable to see things objectively. If the Americans had used this new mathematical logic, they would have seen that the uncertainty as to whether Saddam had access to weapons of mass destruction or not was too large,’ Jøsang points out.
To express the uncertainty of their probability estimates, statisticians use a tool called the confidence interval. A confidence interval describes the likelihood that an event will occur with a probability within a certain interval. However, there are no tools that can simply handle confidence intervals as input arguments in complex models. As a rule, confidence intervals are used only for presentation of results.
Today, all input arguments in intelligence models must be entered with a specific probability, even if this figure is uncertain.
‘What they really ought to say is “we don’t know this”. However, such input arguments are not permitted in traditional analytical tools. With subjective logic, the input arguments may be completely uncertain and estimates can be made with these probabilities, even though they are fraught with uncertainty.’
Mathematical repertoire
The certainty and uncertainty of each probability can be represented by triangles, where the horizontal line describes the magnitude of the probability and the vertical height represents the uncertainty of the probability estimate.
In order to describe situations, Audun Jøsang is now working to expand the repertoire of mathematical operators, such as deduction and abduction, and how these mathematical operators can be combined.
By emphasizing the uncertainty of the input arguments, decision-makers can visualize the degree of uncertainty of their analyses in order to make better decisions. If the result is fraught with large uncertainty, one will be reluctant to make significant, important decisions, but perhaps call for more intelligence work instead.
‘When we implement this logic, we can see the aspects of the theory that are incomplete and need to be straightened out. If the Americans had been in possession of this tool, perhaps they would have found that there was too much uncertainty with regard to the hypothesis saying that Saddam had weapons of mass destruction before taking such a momentous decision as to invade Iraq. Unless the uncertainty of the circumstantial evidence is taken into account, the analysis tool may erroneously conclude that there was a clear probability that Saddam had access to weapons of mass destruction.’
The gist of the matter is:
‘In order to balance all circumstantial evidence it is crucial to describe the uncertainty of each individual piece of information. If this is not done, there is a risk that evidence with a large degree of uncertainty is compared to evidence that is basically certain. One should be cautious in taking large, momentous decisions if the results are fraught with a large degree of uncertainty.’
Professor Jøsang emphasizes that his theory is sufficiently developed to be adapted to analytical tools.
Intelligence services: Applicable
The Norwegian military intelligence service has expressed an interest in this new, mathematical logic, but the intelligence officer who had familiarized himself with the theory did not wish to make a statement to Apollon.
The intelligence officer referred Apollon to Dean Tore Pedersen who is head of intelligence studies at the Norwegian Intelligence College. He made it clear to Apollon that his statement was of a general, academic nature:
‘Subjective logic can be used to explore complex problems that include a large element of subjectivity and uncertainty,’ Tore Pedersen replied to Apollon in an email.
The US Army is interested
American intelligence services are currently using an analytical framework called ACH. The US Army Research Lab, the American equivalent of the Norwegian Defence Research Establishment at Kjeller, supports this research at UiO with NOK 2 million to explore how subjective logic can be implemented in their intelligence analyses.
‘By incorporating uncertainty, subjective logic has the potential to revolutionize automated probability reasoning and improve intelligence operations. The method may enable the decision-maker to realize when the responses are too uncertain and that more information needs to be collected. We still need answers to a number of fundamental questions. The US Army Research Lab is therefore collaborating with Professor Jøsang through the project “Advanced Belief Reasoning in Intelligence” to determine whether and how his idea can be realized,’ says Dr Lance Kaplan in the Networked Sensing & Fusion Branch, US Army Research Laboratory.