Why more is not always better in the world of credit
Why more is not always better in the world of credit
Advanced analytics and AI promise revolutions everywhere, but real-world constraints abound. This is notably true in the world of credit scoring, which needs to be understandable, is often slow to give out real-world results, and can be muddied by the addition of unnecessary factors.
Philip Mackenzie ([email protected]s-research.com)
Big Data and new credit
The Big Data revolution emerged as the result of a wave of digitization and the wholesale shifting of business processes onto the internet, and it promised many things. Primary among these was the idea that more is better. Financial institutions (FIs) in particular, the thinking went, would be able to scoop up the vast oceans of structured and unstructured data washing around in their systems, and use them to more accurately determine how customers and counterparties were acting. Whether FIs were determining risk of default, or potential fraud or money-laundering, more detailed information would allow them to build three-dimensional, ‘holistic’ images of their counterparties that would far outstrip their old-fashioned, simplistic models.
Now that the first wave of hype is dying down, we are seeing that more is not necessarily better. Adding more factors, which govern the results of a model, may possibly make it more accurate, but it will also make it progressively more difficult to tell how the additions are affecting results. Each addition represents a constraint that must be effectively modeled, and their efficacy can only be proven in the real world. This becomes significantly challenging when the ‘real-world’ feedback from a process is delivered slowly. This is most notable in the case of credit risk, where it is often slow indeed: the most obvious example being people who default on their mortgages.
More complex doesn’t mean better
Credit scores tend to be fairly straightforward for this reason, and this is in part why those such as the FICO score, VantageScore, CE Score and the Credit Optics score have gained market share in the US. These tend not to be vastly complicated in terms of the factors they take account of (largely payment history and total debt, for example), but the FICO one in particular has been established for over 60 years, enabling these relatively simple factors to be tuned and tested over multiple credit cycles, and expanding them to cover different businesses. Attempts to enrich the FICO score with additional factors or advanced analytics such as machine learning techniques have had limited success.
The US offers an example of a vast experiment in credit scoring analytics, but there are even larger ones out there. There has been much talk of China taking the artificial intelligence (AI) lead with its governmental investment in complex analytics. The most well-known initiative is perhaps the Social Credit System. This is intended to standardize the assessment of citizens' and businesses' economic and social reputation, and does so by combining credit scoring with numerous other factors. These include those as ‘esoteric’ as playing too many video games and indulging in anti-government online speech.
This may represent an Orwellian intrusion into citizens’ lives, or an unyielding universal framework upon which individuals can build trust and trade, or something somewhere between the two. However, there is one relatively certain factor about it: it doesn’t represent an optimal credit scoring system.
This is because it has a dual purpose: it is not simply trying to determine who is likely to default on loans, it is also clearly attempting to enforce an image of what a model citizen looks like. It would take a lot of time and effort to determine the limits of excessive video games and online speech on probability of default, and any correlation would likely be weak to nonexistent. Throwing lots of these factors into the mechanism is likely to add significant noise and degrade the effect of the parts of it that actually work. The effects could vary, from a range of unexpected defaults, to the harder-to-measure (but equally negative) effect of innocent people being denied access to credit and economic stability.
Increasing complexity is not always the answer
It remains important to look at the underlying motivations and reasons for applying advanced analytics, as well as how exactly the underlying factors work, and over what timescales. Technology is not necessarily best served by an onward march toward increasing complexity. Those designing analytics should consider what the timescales are for validation, how easy it will be able to test outcomes, and whether there are bright lines between the questions the analytics are asking and the expected outcomes (‘risk of default’, for example, versus ‘good citizen’). More is not necessarily better.
Points of View are short articles in which members of the Chartis team express their opinions on relevant topics in the risk technology marketplace. Chartis is a trading name of Infopro Digital Services Limited, whose branded publications consist of the opinions of its research analysts and should not be construed as advice.
If you have any comments or queries on Chartis Points of View, you can email the individual author, or email Chartis at [email protected].