Risk matrices are an invaluable tool for organizations seeking fast, effective and practical risk assessment processes but they cannot be used in isolation. Much of the criticism regarding risk matrices has been related to attempting to use them without establishing the broader parameters of the risks being considered.
You’ll find more articles like this and excerpts from my upcoming book “Get the benefits of ISO 31000:2009 Risk Management fast!” at http://31000risk.blogspot.com/
For very good reason risk matrices have been widely promoted in risk management standards and reference books and adopted by many organizations. They are a practical and easy to use tool which can help most organizations in most circumstances to:
- promote robust discussion (the discussion often being more useful than the actual rating)
- provide some consistency to prioritizing risks1
- help keep participants in a facilitated risk workshop on track
- focus decision makers on the highest priority risks
- present complex risk data in a concise visual fashion (eg: bubble charts)
As a general rule with experienced practitioners leading the way they can be very effective for getting timely results in a facilitated risk workshop and for presenting data.
They are not without flaws however and are definitely not a panacea for all ills. In the hands of the inexperienced, the biased or individuals with an agenda, they can of course generate misleading ratings. In his article “What’s Wrong with Risk Matrices?” Tony Coxi suggests that they have the following limitations:
- They can correctly and unambiguously compare only a small fraction of randomly selected pairs of hazards and can assign identical ratings to quantitatively different risks.
- They can mistakenly assign higher qualitative ratings to quantitatively smaller risks to the point where with risks that have negatively correlated frequencies and severities, they can lead to worse-than-random decisions.
- The can result in suboptimal resource allocation as effective allocation of resources to risk treatments cannot be based on the categories provided by risk matrices
- Categorizations of severity cannot be made objectively for uncertain consequences. Assessment of likelihood and consequence and resulting risk ratings require subjective interpretation, and different users may obtain opposite ratings of the same quantitative risks.
To this list I could also add that risk matrices:
- don’t include any assessment of timeframes (eg: The risk of a terrorist attack in the next 2 weeks might be very different from the risk of a terrorist attack in the next 2 years)
- they can oversimplify the complexity or volatility of a risk insomuch as some risks are relatively static over time while others can change for better or worse almost overnight
Limitations of the limitations
All of these points are true but they omit the following fundamental issues:
- risk matrices are still one of the best practical tools that we have
- “the use of risk matrices is too widespread (and convenient) to make cessation of use an attractive option” ii
- prioritizing the allocation of resources is not the role of the risk matrix – that role belongs to the selection of risk treatments2
- any risk assessment tool can assign identical ratings to quantitatively different risks
- no tool can consistently correctly and unambiguously compare more than a small fraction of randomly selected pairs of hazards
- risk matrices are designed to provide qualitative or semi-quantitative ordinal information (relative priority) not mathematically precise data
- if a risk is in the ‘High’ or the ‘Top 10’ list it requires attention and whether it is third or fourth on the list is not likely to be significant
- the inherent limitations of decision making under uncertainty, the nature of political decision making and the fundamental processes of human risk perception mean that subjective decision making will always be a part of the risk assessment process no matter what tool is used
- risk matrices are a tool which support risk informed decisions, not a tool for making decisions
- last but not least, most of the flaws listed above only exist if risk matrices are used in isolation, which is rarely the case
Overcoming the limitations
The last point above is the most significant of all. If you use a risk matrix in conjunction with at least the following tools, they can be highly effective in supporting quality decision-making:
- A clearly defined risk statement
- Robust likelihood and consequence definitions
- A hierarchy of controls to prioritise risk treatments
- Expected monetary value (EMV) or equivalent cost/benefit of risk treatments
In addition it is important to have a process for considering all risks and risk treatments collectively. Each treatment is likely to mitigate several risks, albeit to differing degrees, therefore optimal allocation of resources is likely to be a complex decision making process. The last two tools on the above list are not really specific to risk matrices as they are about prioritising risk treatments. A hierarchy of controls (also known as ESIEAP3) enables an optimized approach for selecting the relative effectiveness of controls but does not consider cost/benefit which is a separate although linked process.iii
The really critical issues for successfully using risk matrices to assess risks however are the first two items in this list. If sufficient rigor has been put into defining the risk statement and the likelihood/consequence definitions then meaningful risk ratings can be quickly and consistently obtained from a risk matrix. If these 2 items have been adequately defined then you are likely to get similar if not identical risk ratings from knowledgeable people conducting independent assessments.
Many risk matrices have inadequate likelihood and consequence definitions and even more commonly, users attempt to use them to assess poorly defined risks. Without these two things in place a risk matrix will provide meaningless if any information.
CASE is one tool for clearly defining a risk statement but is the best I know of because if you want to clearly articulate a risk you need to consider the following four characteristics:
- C onsequence – what is the impact of this risk?
- A sset – what asset(s) are at risk?
- S ource – what are the hazards or threat actors behind this risk?
- E vent – what particular type of incident is being considered?
Why do you need these four items to define a risk statement? Consider if you will, the risks of “terrorism”, “climate change” or “compromise of sensitive information”. Each of these are actually multiple risks. It is very difficult if not impossible to analyze and rate these risks if we only have the event and the asset. Or, more particularly it is impossible to achieve consensus on the risks involved because everyone will have their own context and perception of what ‘terrorism’ or ‘climate change’ means. For example, the consequences and likelihood are very different if your organizations information was compromised through:
- industrial espionage by competitors
- theft by criminals seeking to sell it back to you
- espionage by foreign intelligence services
- computer user security access errors
- theft of a briefcase from a car by petty criminals
- staff inadvertently releasing the information to the corporate website
- premature distribution of a media release
- accidental emailing of a sensitive document to the wrong party
Not only would consequences and likelihood vary considerably and hence the risk but perhaps more importantly the countermeasures would be very different.
Consider however if the risks were written to include CASE:
- Compromise of sensitive information (Asset) due to untrained (Source) staff inadvertently posting incorrect files (Event) to corporate website resulting in competitive disadvantage, reputation damage or financial loss (Consequence).
- Financial loss (Consequence) due to espionage (Event) by competitors (Source) seeking sensitive information (Asset).
- Failure to protect information (Asset) from theft (Event) by opportunistic criminal (Source) elements while in transit leading to potential compromise (Consequence) of sensitive information.
These are much easier (and very different) risks to assess using a simple risk matrix and much easier to define specific countermeasures for.
Human beings have great difficulty in making accurate judgements under uncertainty.iv Accordingly, our ability to select from a range of likelihoods and consequence ratings is moderate at best and at worst, severely tempered by our biases and heuristics. In the real world however, we do not have a lot of options so the art and adventure of making good risk decisions is an ongoing challenge.
Often we are forced to make difficult decisions under conditions of extreme uncertainty. Hard statistical data is usually lacking and even when we have quantitative data it is limited in application. Insurance companies for example, can tell you how many houses are statistically likely to burn down each year in your city but they can’t tell you how likely it is that your house will be one of them. Equally a simple change such as mandating smoke detectors can make 10 years of statistical information virtually meaningless overnight. Equally, tools such as Monte Carlo modelling can help us to understand the breadth and depth of our risk exposures when we have data of sufficient quality but will still require subjective interpretation before we can apply those insights.
So what can we do to help ourselves make better decisions under conditions of uncertainty? Firstly we need to find a consistent approach so that we can compare apples with apples. Risk matrices can be invaluable and practical decision support tools in this respect if we frame the word pictures for their use in such a manner that individuals working in isolation would generate broadly consistent risk ratings with them.
When describing a risk it is essential to make some sort of determination regarding the type of consequences that you want to consider and the likely extent of those consequences if the risk eventuates. Not the worst-case consequence necessarily, but the worst credible scenario. For example, it is possible for ‘slips, trips and falls’ to cause death if someone lands on a sharp stick or hits their head on concrete but it is not a particularly credible outcome. The most credible outcome is that a person will get up with bruises, perhaps a graze or even a sprain. Death is possible but incredibly rare. If appropriate to circumstances, an organization might like to consider two or more risks. For example:
- Minor injury as a result of a staff member slipping on water in the kitchen due to a leaking pipe.
- Major injury as a result of a staff member slipping on water in the kitchen due to a leaking pipe.
- Death due as a result of a staff member slipping on water in the kitchen due to a leaking pipe.
The likelihood ratings and hence risk ratings, of these are so different as to make them essentially different risks. These however are not the only consequences of slipping over and others might include, lost time, capability impacts, financial costs and reputation damage. It’s important to note that there may well be downstream impacts of a particular risk which would exceed the immediately obvious.
For example, when rating “Minor injury as a result of a staff member slipping on water in the kitchen due to a leaking pipe” the consequence of the injury would be likely to rate as INSIGNIFICANT in terms of the People, Economic and Capability consequences in Table 1 and would not be rated at all in terms of property or information. At the same time, depending on the context, it might rate as NEGLIGIBLE in terms of Reputation. The overall risk rating would therefore be based on the higher of these two (Negligible).
|People||Minor injury or first aid treatment||Injury requiring treatment by medical practitioner and/or lost time from workplace.||Major injury / hospitalization||Single death and/or multiple major injuries||Multiple deaths|
|Information||Compromise of information otherwise available in the public domain.||Minor compromise of information sensitive to internal or sub-unit interests.||Compromise of information sensitive to the organizations operations.||Compromise of information sensitive to organizational interests.||Compromise of information with significant ongoing impact.|
|Property||Minor damage or vandalism to asset.||Minor damage or loss of <5% of total assets||Damage or loss of <20% of total assets||Extensive damage or loss <50% of total assets||Destruction or complete loss of >50% of assets|
|Ecomonic||1% of budget (organizational, division or project budget as relevant)||2-5% of annual budget||5-10 % of annual budget||> 10% of budget||> 30% of project or organizational annual budget|
|Reputation||Local mention only. Quickly forgotten.
Freedom to operate unaffected. Self-improvement review required
|Scrutiny by Executive, internal committees or internal audit to prevent escalation Short term local media concern.
Some impact on local level activities
|Persistent national concern. Scrutiny required by external agencies. Long term ‘brand’ impact.||Persistent intense national public, political and media scrutiny.
Long term ‘brand’ impact. Major operations severely restricted.
|International concern, Governmental Inquiry or sustained adverse national/international media. ‘Brand’ significantly affects organizational abilities.|
|Capability||Minor skills impact. Minimal impact on non-core operations. The impact can be dealt with by routine operations.||Some impact on organizational capability in terms of delays, systems quality but able to be dealt with at operational level||Impact on the organization resulting in reduced performance such that targets are not met. Organizations existence is not threatened, but could be subject to significant review.||Breakdown of key activities leading to reduction in performance (eg. service delays, revenue loss, client dissatisfaction, legislative breaches).||Protracted unavailability of critical skills/people. Critical failure(s) preventing core activities from being performed. Survival of the project/activity/organization is threatened.|
Likelihood can be framed in quantitative, semi-quantitative or qualitative fashions. Where we don’t have sufficient data for quantitative analysis and would like something more granular than simply ‘likely’ or ‘unlikely’, risk matrices are ideally suited for semi-quantitative analysis. There are many ways of representing likelihood, however in the example below I have elected to use the following terms:
- Chance: a qualitative assessment of likelihood.
- Probability: a statistical or actuarial assessment of likelihood.
- Frequency: the rate at which something occurs or is repeated over a given sample.
Once you have clearly identified the risk to be considered.
|Almost Certain||Is expected to occur in most circumstances||Has occurred 9 or 10 times in the past 10 years in this organization or circumstances are in train that will almost certainly cause it to happen||>95%|
|Likely||Will probably occur in most circumstances||Occurred more than 7 times over 10 years in this organization or in other similar organizations or circumstances have such that it is likely to happen in the next few years||>65%|
|Possible||Might occur at some time||Has occurred in this organization more than 3 times in the past 10 years or occurs regularly in similar organizations or is considered to have a reasonable likelihood of occurring in the next few years||>35%|
|Unlikely||Could occur at some time||Has occurred 2 or 3 times over 10 years in this organization or similar organizations||<35%|
|Rare||May occur only in exceptional circumstances||Has occurred or can reasonably be considered to occur only a few times in 100 years.||<5%|
Strictly speaking frequency is another way to express probability data however Gerd Gigerenzer in his book Calculated Risks cites any number of examples of highly educated professionals who are unable to correctly interpret probability data. He goes on to show that the most effective way for people to understand likelihood is to state it in terms of frequency.v In a 1998 study of counselors and medical professionals the overwhelming majority were unable to correctly answer the following question:
About 0.01 percent of men with no known risk behavior are infected with HIV. If such a man has the virus, there is a 99.99 percent chance that the test result will be positive. If a man is not infected, there is a 99.99 percent chance that the test result will be negative. What is the chance that a man with no known risk behavior who tests positive actually has the virus?
Most of the professionals and most people think that it is 99.99 percent or higher. Now consider the same question worded using natural frequencies:
Imagine 10,000 men who are not in any known risk category. One is infected and will test positive with practical certainty. Of the 9,999 men who are not infected, one will test positive. So we can expect that two men will test positive.
By presenting the data in natural frequencies, you can easily see that the odds are roughly 1 in 2 (50%) that someone from a low-risk category who has a positive test result is actually HIV positive.
Similarly Table 2 provides options for selecting the optimal expression of likelihood options but natural frequencies (the third column) will typically provide individuals and groups with an option which will be most meaningful to them and hence is likely to deliver the optimal results.
Using a risk matrix
It is important to remember the purpose of a risk matrix. We’re usually not trying to obtain a precise estimate of the risk, or to determine the potential impact on objectives in great detail and any such attempts are rarely useful. When we use a risk matrix we are usually trying to assess and prioritize a list of risks. Where there are too many risks for us to give them all the same level of attention we need to aggregate them into a few groups or identify the most significant risks, so that we can focus first on those requiring urgent management, then deal with other important risks, and merely monitor the remainder. The use of red-yellow-green types of categorization reflects this broad classification of risks into high-medium-low priority.
In some cases it may be enough merely to rank risks against each other to determine relative prioritisation. All ‘red’ risks should be treated as high priority and we may not need to worry about whether some are more red than others.
For the purposes of illustration I’ve used a 5×5 risk matrix with 5 levels of risk (Very Low, Low, Medium, High, Very High) in Figure 1 however no specific level of granularity is better than any other. As long as a matrix provides sufficient granularity for the purpose to which it is being applied, it has the right number of squares. A 2×2 matrix may be suitable for comparing 3 risks or you may choose to use a 4×8 matrix to compare 25 risks in your organization. The numbers in the squares in the risk matrix (2 to 10) are of course optional and purely intended to provide some level of granularity within specific risk ratings. In this instance the likelihood and consequence have simply been summed but multiplying them would essentially deliver the same functional results when prioritizing risks.
Figure 1 also illustrates the complexities of assessing a relatively minor risk and the potential perils of inadequate data. It considers the risk of “Minor injury as a result of a staff member slipping on water in the kitchen due to a leaking pipe” and illustrates the importance of considering historical data and downstream impacts. Even with a well defined risk the likelihood and consequence may not be as they appear. In the hypothetical but realistic example provided the risk assessors:
- initially downplayed the likelihood as ‘Unlikely’ (‘Could occur at some time’ or ‘<35%’) because people typically assess probability of an event by the ease with which instances or occurrences can be brought to mindvI however on considering the Frequency column and the historical incident reports they realised that it had occurred more than 7 times in the past 10 years and was therefore ‘Possible’4
- first considering only the consequence of ‘minor injury’ which provided a rating of ‘Insignificant’ but when they considered downstream impacts (‘scrutiny by internal committee’) the consequence was upgraded to ‘Negligible’
Although purely an illustrative example, the initial low ratings are a credible outcome given that people downplay risks which are pedestrian, common, familiar to them or well understood and exaggerate risks that are spectacular, personified or highly publicized.vII, vIII, Ix
FIgure 1: Example Risk Matrix
As illustrated in Figure 1 the fuller consideration of the risks prompted by the likelihood and consequence descriptors resulted in the risk being revised from ‘Very Low’ to ‘Low’. This is minor of itself and doesn’t fundamentally change the risk however it can significantly change the management attention this risk receives when it is prioritized among other risks. It is also a good example of the folly of relying solely on risk matrices to make resource allocation decisions. Although it is a relatively low risk, the cost of having a plumber repair the leak is likely to be insignificant.
Three points are important to clarify at this juncture:
- to obtain a realistic risk rating it is more useful to consistently use the highest (or worst) likelihood and consequence to rate the risk
- this example is an example only and other risks will have their own characteristics – individual assessments may provide identical or widely varying likelihood and consequence ratings
- a risk matrix which considers only one category of consequence and/or only one estimation of likelihood is likely to be of limited value and yield inconsistent results
Using risk matrices to present data
Even if an organization chooses to use another method of assessing risks, the humble risk matrix is one of the most effectively tools to quickly convey risk information to an audience.
Most people will be familiar with the use of bubble charts. A simple example is presented in Figure 2 but there are many more pieces of information which can be swiftly conveyed by a well crafted chart based on a commonly understood tool – the risk matrix.
3: Example of risk matrix used to present complex data
FIgure 3: Example of risk matrix used to present complex data
In Figure 3 at least 15 pieces of information are conveyed regarding an organizations risks using a risk matrix bubble chart including:
2. Inherent risk rating if no controls were in place (Position ‘A’)
3. Past risk ratings (Position ‘B’)
4. Changes in risk ratings over time (Delta between positions ‘B’ & ‘C’)
5. Expected residual risk after implementation of treatments (Position ‘D’)
6. Likelihood (Vertical positions on matrix)
7. Consequence (Horizontal position on matrix)
8. Timeframe of assessment (Title)
9. Rough order of magnitude cost of current spend on risk treatments (number of ‘$’ symbols on arrows between ‘A’ & ‘B’ and ‘B’ & ‘C’)
10. Comparative benefit and costs of proposed risk treatments (Delta – expressed by length of arrows – and number of ‘$’ between risk positions)
11. Volatility. ie. Whether risk is relatively static over time or can change dramatically at short notice (shape of the symbol)
12. Level of confidence in the quality of the risk rating (size of the symbol)
13. Whether or not the risk has occurred in this organization in the past (Risk number in plain text or Bold Italic)
14. Comparative priority of one risk to another (position on matrix)
15. Level of management intervention and responsibility required to address the risk (Colour of the grid square in which the risk is located)
This is just a sampling of the way in which risk information can be presented using a risk matrix and the only limitations regarding the amount information that can be transmitted is one’s imagination.
Risk matrices are an invaluable tool for organizations seeking fast, effective and practical risk assessment processes but they cannot be used in isolation. Any assumptions or embedded judgments need to be clearly articulated and in particular (a) the risk description must be clearly defined and (b) the likelihood and consequence descriptors need to be clearly articulated using a variety of parameters.
Risk matrices are not suited for every circumstance and they do have limitations but they also have a clear place in the toolbox of every risk manager who wants to:
- provide consistency and granularity to risk prioritization
- encourage and facilitate robust discussion
- provide a point of focus when assessing risks
- present complex data concisely
 Risk treatments will typically treat more than one risk in a basket of risks (risk register) therefore selection of risk treatments cannot be based simply on the priority of risks nor on their effectiveness against one single risk.
 ESIEAP stands for Elimination, Substitution, Isolation, Engineering, Administrative controls and Protective measures. It is a decision making tool for evaluating which of a bundle of risk treatments would be most effective. For example the best way to mitigate the risk of Malaria on holiday is to eliminate the risk by not travelling. Second best option if you still want to travel for holiday is to substitute another location which has no malaria. Third best option if you must visit a malarial region is to isolate yourself in parts of the country where it is not prevalent, and so on with engineering controls such as flyscreens and bed nets being less effective, administrative controls such as not going out at dawn or dusk in turn less effective and the least effective being measures such as protecting yourself with long sleeves and insect repellant.
 In the absence of historical data, the equivalent experiences of similar organizations would be likely to produce a similar result, however even without this the overall process would still yield valuable discussion and at the very least an assessment of risk which would provide prioritization (however inexact) and could be compared against any incidents or future data.
About the Author
Julian is an international risk management consultant and lead author of the Security Risk Management Body of Knowledge He is a Fellow of the Risk Management Institution of Australasia and Research Associate with the Australian Homeland Security Research Centre. Julian holds a Master of Risk Management and his background includes roles as Senior Risk Adviser for the Australian Department of Health and Ageing, Head of Security for the Australian Governments most extensive international network (the Australian Trade Commission) with offices in 60 nations and as Security Manager for Australia’s largest resources project (the $22 billion North West Shelf Project), Chairman of the $60 million Citadel Group, Director of the Risk Management Institution of Australasia (RMIA), Director of the Australian Institute of Professional Intelligence Officers and Assistant Regional Vice President with ASIS International. Julian has conducted risk analysis and prepared multi-year enterprise risk plans for multi-billion dollar international organizations and $300 million dollar projects. He is a popular conference speaker and is recognised internationally as a leading thinker and practitioner in the risk field, who has made several innovative contributions to improving risk management. This article is based on excerpts from his latest book “Get the benefits of ISO 31000:2009 Risk Management fast!” which is in final editing. You can find more excerpts from the book at http://31000risk.blogspot.com/.
[I] Cox, L.A. (2008), ‘What’s Wrong with Risk Matrices?’, Risk Analysis, Vol. 28, No. 2, DOI: 10.1111/j.1539-6924.2008.01030.x
[II] Cox, L.A. (2008), ‘What’s Wrong with Risk Matrices?’, Risk Analysis, Vol. 28, No. 2, DOI: 10.1111/j.1539-6924.2008.01030.x
[III] Talbot, J. & Jakeman, M. (2009), Security Risk Management Body of Knowledge, Wiley Interscience, NY, USA.
[Iv] Plous, S. (1993), The Psychology of Judgment and Decision Making, McGraw-Hill, NY, USA.
[v] Gigerenzer, G. (2002), Calculated Risks, Simon & Schuster, NY, USA.
[vI] Tversky, A. and Kahneman, D. (1974), ‘Judgment under Uncertainty: Heuristics and Biases’, Science, 1974, 185:1124–1130
[vII] Glassner, B. (1999), The Culture of Fear: Why Americans are Afraid of the Wrong Things, Basic Books, NY, USA.
[vIII] Slovic, P. (2000), The Perception of Risk, Earthscan Publications Ltd, London, UK.
[IX] Kluger, J. (2006), “How Americans Are Living Dangerously,” Time, 26 Nov 2006, NY, USA