Model vs Scenario: investment risk modelling in a crisis
Updated: May 22, 2020
If investment risk professionals spent 2019 preparing for volatile, uncertain, complex and ambiguous markets, 2020 is proving to be the year in which that "VUCA" environment has materialised. In these “unprecedented” times, how well equipped are risk models to help navigate volatility and predict a reliable distribution of possible future returns?
Strictly speaking, market conditions shouldn’t matter: the basic purpose of a risk model is to identify the risks and drivers of risk for an investment portfolio, regardless of the operating environment. Broadly speaking, that means you need it to answer the following questions: What is the total level of risk currently in the portfolio? How does it compare over time? What are the drivers of that particular portfolio and what are the particular market events that will lead to an undesirable outcome?
While those questions are the same no matter the market environment you’re in, in reality there are a few other considerations that could impact the performance of a model. For one, models that derive their input from historical data will, by definition, struggle with “unprecedented” conditions.
“If you only use historical data, you have a 100% chance of not being able to model something that hasn’t happened yet,” says Dr Laura Ryan, Head of Research at Ardea.
For another, there is always going to be a time delay – reflecting the models’ ability to pick up on the changed circumstances. That means that when faced with sudden spikes in volatility, a backward-looking risk model is likely to under forecast risk, which is hardly ideal when markets drop 150 points over lunch. While risk models are always one of many tools in a risk manager’s armoury, at times like these it is important to use all the tools in the box, and some more than others.
“While we’ve be using our risk models, we’re focusing on stress-testing using scenario analyses,” James Thompson, Head of Investment Analytics and Risk at Perpetual told Fund Business. “We find that while risk models are a very useful guide in the lead up to a crisis, at times of extreme volatility the fact that it takes a day or two of market turbulence before the models start picking up on increased volatility means that they’re really not telling me anything I don’t already know, or couldn’t rough guess anyway,” he says.
That sentiment is echoed by Tom Gillespie, Senior Manager Portfolio Risk at NSW Treasury Corporation (Tcorp), who says that while risk models continue to perform their basic functions, they are not the main tool to rely on in a crisis situation.
“A risk model gives you a measure of the broad spectrum of outcomes and that works well in 95% of market outcomes. It’s never going to be perfect, in much the same way that you can never predict with certainty whether a horse will win a race, but it will give you a probability of outcomes. When you are talking about stress events though, I would argue that stress-testing and scenario analysis are more useful tools to use,” Gillespie says.
That’s not to say that models are useless in a crisis – some numbers are still better than no numbers, but it’s important to look beyond the numbers. “My perspective on risk models is that all models are wrong, but it is better to have a model than no model and it is better to have two models than one model,” Ryan says. “Even if the type of crisis you anticipated doesn’t happen, at least you have thought about some of the operational implications of what you are going to do in a crisis, and then maybe these questions about how do you rebalance in a liquidity crisis would have already been considered and not been done on the fly,” she says. Most risk models come with in-built scenarios and the ability to tweak different inputs, though some provide more flexibility than others to think outside of the box.
“This is where it is up to us as fund managers to model our own scenarios and come up with plausible outcomes,” says Thompson, who says that while there are clear advantages to running a risk model in the background, getting the investment team together to brain storm scenarios is just as important.
“It’s about getting everyone together, making up a scenario and getting the team to think about how they’d work through that, what stocks they’d sell, how much cash they’d hold, how they’d manage downside risk. That’s when you start to think about other things that could happen, or the interrelationship between things, rather than blindly following the risk model and it’s certainly something we’ll be doing more of going forward,” Thompson says.
Ardea’s Ryan also stresses the importance of tailoring stress tests to the fund strategy in question.
“It’s important for us to incorporate adverse scenarios that are specific to our strategy, not just adverse markets events in general. For example, we need liquidity across global rates markets to implement our RV strategy. Because of this, our risk model scenario tests included severe liquidity crises,” she says, pointing out that the type of liquidity crisis experienced across March saw even the highest quality government bond markets’ liquidity challenged. “It’s not something we would have said was a high probability, but that’s the whole point of modelling a tail event,” she says.
While risk models continue to play an essential role in investment risk management, the latest crisis could have significant implications for some models. “Given how far off some of the models have been, once the numbers are backtested and comparisons are drawn, I reckon it could make or break a couple of them,” one practitioner told Fund Business.