The impact of Covid on machine learning and data science in UK banking

Overview

This article reports findings from a survey conducted in August 2020 of banks headquartered or operating in the UK. The survey sought to understand how Covid has affected the adoption and use of ML and DS. Half of the banks surveyed reported an increase in the importance of ML and DS as a result of the pandemic. None of the banks believe that Covid will reduce the importance of ML and DS for them.

The fact that investment and interest in ML and DS has remained broadly stable, and has actually increased for some banks, is significant given wider market volatility and the economic downturn, which might be expected to constrain budgets for investment in innovation. This may reflect the nature of the shock, which has affected financial services less than other sectors so far. We will continue to monitor how these technologies are used, including their benefits and risks, and explore how we can best support their safe adoption.

The use of ML and DS in UK banking before Covid

Recent trends in ML and DS

The digitalisation of society and the economy over the past two decades has generated vast amounts of data. As a result, DS has become an increasingly important function for businesses seeking to capitalise on data-driven insights. This has also led to the increased use of ML across a range of sectors.

Finance is one of the sectors that has seen widespread adoption of ML and DS in recent years. In 2019, we conducted a joint survey with the Financial Conduct Authority (FCA) to understand how ML was being used in UK financial services. The survey showed that ML was already being used by a majority of firms across a range of financial subsectors and business lines (Chart 1).

Chart 1: Two thirds of respondents have live ML applications in use

Footnotes

• Source: Bank of England and FCA (2019), ‘Machine learning in UK financial services’.

Banking was the subsector with the highest number of ML applications and second highest share of ML applications relative to the number of survey respondents. The two most prominent uses were customer engagement and risk management. Among a majority of banks surveyed in 2019, their use of ML had matured to the point that they were deploying it in the regular run of operations. Moreover, the majority of banks expected the number of ML applications to triple by 2021 (Chart 2).

The increased use of ML in financial services is not limited to the UK. In 2019, the regulatory authorities in Canada and Hong Kong reported similar increases in the importance of ML and its adoption by banks in their respective jurisdictions.

These pre-Covid patterns in banks’ use of ML and DS were backed up by a survey conducted by the Economist Intelligence Unit in February and March 2020, as well as a study published in January 2020 by the Cambridge Centre for Alternative Finance (CCAF) and the World Economic Forum (WEF). The CCAF and WEF surveyed 151 fintech start-ups and incumbent firms across 33 countries. They found that 85% of respondents already used some form of AI, most commonly in risk management, 64% expected to use AI in three or more business areas within two years, and 77% anticipated that AI would have high or very high overall importance to their business by 2022 (Chart 3).

Chart 3: Before Covid financial firms expected AI to become strategically more important by 2022

Current and expected strategic importance of AI to firms (surveyed pre-Covid)

Footnotes

• Source: CCAF and WEF (2020), ‘Transforming Paradigms: A Global AI in Financial Services Survey’.

Benefits for households, banks and the economy

ML and DS have wide-ranging applications in financial services, which can bring benefits to consumers, businesses and the economy. For example, many banks use ML and DS for anti-money laundering (AML) processes. In many instances, this has reduced the rate of false positives in money laundering detection,^{footnote [2]} with one large UK bank lowering its false positives by 70%. For consumers, this helps reduce the number of erroneously blocked or delayed payments. For banks, this frees up scarce resources and speeds up internal processes. For the economy as a whole, this can help banks and authorities more precisely identify illicit financial activity.

ML and DS also have the potential to provide more inclusive and tailored products to consumers. For example, ML is already being used by banks and fintech companies around the world to analyse newer data sources (such as social media data) to provide risk assessments of individuals with limited credit histories, which might help underserved or unbanked customers access financial services. Some UK fintechs and banks are using new data sources for consumer and business risk assessments. This trend looks set to continue with one credit rating agency announcing plans to offer UK banks access to a broader range of transactional data for consumer credit scores, including money earned and spent, council tax payments, savings and investments, and subscription payments.

Risks and challenges

At the same time, existing risks may be amplified or new risks may emerge from the use of ML and DS in financial services. Respondents to the Bank of England and FCA survey and a similar report from the UK’s Centre for Data Ethics and Innovation note that risks could be amplified by ML’s lack of explainability (the so-called ‘black box’ problem), meaning the outputs cannot always be easily understood. In addition, ML models may perform poorly when applied to a situation they have not encountered before in the training data. This is particularly relevant in the context of the Covid pandemic when the underlying data may have changed (data drift) or the statistical properties of the data may have changed (concept drift).

These risks could materialise at an individual bank or system-wide level. Systemic risks are particularly concerning as they can create financial instability, which can adversely affect the real economy and the prosperity of households and businesses. Therefore we have a keen interest in understanding how ML and DS are being deployed. This includes continuing to monitor adoption, assess benefits and risks, and identify ways to support safe application in ways that align with our mission of monetary and financial stability.

The impact of Covid on ML and DS in UK banking

To help us understand the impact of Covid on ML and DS in the UK banking sector, we conducted a further survey of Prudential Regulation Authority (PRA) regulated banks in August 2020.^{footnote [3]} The survey focused on banks’ perception of ML and DS, as well as the resourcing for current and planned ML and DS projects (see annex for the survey questions).

Around 40% of respondents reported an increase in the importance of ML and DS for future operations, and a further 10% of banks reported a large increase. None of the banks reported a decrease in the importance of ML and DS. This is a striking finding that runs counter to the suggestion of some commentators that a new ‘AI winter’^{footnote [4]} might be setting in as a result of reduced investment budgets due to the economic impact of Covid or because pre-pandemic models may no longer be relevant. With the latter, this may be the case since the vast amount of training data required for ML does not account for recent societal and economic changes.

Chart 4: Half of banks view ML and DS as more important for future operations since Covid

Impact of Covid on banks’ plans for, and current use of, ML and DS

Footnotes

• Source: Bank of England (2020), ‘ML, DS and Covid survey’.

About a third of banks said there was an increase in the number of ongoing ML and DS applications. Yet only 16% of banks reported an increase in funding and/or resourcing for existing applications and a similar number reported a decrease. Similarly, around 35% of banks reported an increase in the number of planned applications. But only 23% of banks reported an increase in funding and/or resourcing for planned applications and 11.5% of banks reported a decrease.

Market contacts suggest that, as banks manage the cost and revenue impact of Covid, they are looking to use ML and DS to increase efficiency and improve digital customer channels. Covid has accelerated banks’ use of ML-powered tools to deal with an unprecedented uptick in customer enquiries. Half of the banks in our survey reported a ‘positive’ impact on plans for customer engagement applications. Around a third of banks also reported a ‘positive’ impact on planned investment in internal operations and financial crime applications. As the 2019 Bank of England and FCA survey found, ML models have already been used in all three of these areas.

Chart 5: Banks plan to invest more in ML and DS across a range of business areas due to Covid

Impact of Covid on planned investment by use case

Footnotes

• Source: Bank of England (2020), ‘ML, DS and Covid survey’.

The overall planned investment picture is largely similar for all banks in our survey, with UK headquartered banks having slightly more positive expectations. More specifically, nearly 60% of banks headquartered in the UK reported that Covid has had a ‘positive’ impact on planned investment in customer engagement applications. Similarly, almost half of these banks noted the ‘positive’ impact on planned investment in DS and ML applications in credit (including origination and pricing), with 29% of the banks reporting a large ‘positive’ impact. Market intelligence suggests this is due to UK banks using ML to deal with the high volume of customer enquiries and government-guaranteed loan applications.^{footnote [5]} These banks may also use ML and DS as they look to refine expected credit loss calculations in line with the IFRS 9 accounting regulation.^{footnote [6]}

Our survey shows that around 35% of banks reported that ML and DS had a ‘positive’ impact on technologies that support remote working among employees. The same percentage also reported a ‘positive’ impact on their overall risk appetite for ML projects, meaning these banks are more willing to use these techniques. At the same time, around 35% of banks reported a negative impact on ML model performance. This is likely because the pandemic has created major movements in macroeconomic variables, such as rising unemployment and mortgage forbearance, which required ML (as well as traditional) models to be recalibrated. Other areas where banks noted a negative impact were in ‘resourcing’ and in ‘hiring/retention of skilled staff’.

Chart 6: Covid had a negative impact on model performance

Issues (opportunities and risks) that existing ML and DS applications have encountered as a result of Covid

Footnotes

• Source: Bank of England (2020), ‘ML, DS and Covid survey’.

It is important to note that while Chart 6 indicates where ‘positive’ or negative effects are felt, the numbers do not tell us the extent of these effects, beyond small or large, nor indeed how they may impact banks’ business models or financial performance. More research is needed to gauge how material the affected ML/DS models are to banks’ overall performance, operations and risk profile, and hence the overall impact of the crisis.

Finally, there were marked differences between small and large UK banks^{footnote [7]} with respect to their use of third-party vendor products and services. Chart 7 shows that smaller banks reported a ‘positive’ impact (eg in terms of performance, impact, use) of Covid on all categories of DS and ML, with data collection, and model testing and validation being the areas with the largest ‘positive’ impact. Large UK banks reported a ‘positive’ impact on use of outsourced platforms and infrastructure. These findings are in line with market intelligence that smaller banks are looking to increase their use of off-the-shelf ML products. This stands to reason given the generally more substantial in-house data and analytical capabilities of large banks.

Alongside the usual risks associated with outsourcing, the use of ML and DS can pose additional risks and challenges. For example, outsourced ML models may be more difficult to interpret because detailed knowledge in terms of how they were developed resides outside the bank. This can make it more difficult for banks to understand how the model works and to monitor performance, which could result in unexpected or unexplained performance, and risks materialising. If multiple banks use the same third-party provider and ML model, this could also potentially lead to an increase in herding, concentration and even the possibility of systemic risks where methodologies are common.

Chart 7: Covid had a ‘positive’ impact on outsourcing and the use of third-party providers by large banks

Impact of Covid on the use of third-party providers across different parts of the ML and DS pipeline

Footnotes

• Source: Bank of England (2020), ‘ML, DS and Covid survey’.

Explaining our survey findings

Prior to the survey, we expected that UK banks’ investment in ML and DS in response to Covid might follow the same historical pattern as other business investment during an economic downturn. Most businesses tend to respond to negative macroeconomic shocks by reducing expenditure, including spending on investment and innovation. In this way, business investment is typically procyclical, rising in upswings and falling in downturns.

A major reason businesses reduce investment in innovation during economic downturns is the need to prioritise near-term cash flow rather than long-term technology projects. Businesses may become increasingly hesitant to invest in long-term capabilities when revenues are declining, and when there is higher uncertainty around future profits. There is plenty of evidence to suggest that uncertainty has increased during the pandemic. Our Decision Maker Panel survey, designed to be representative of the population of UK businesses, found that 70% of firms viewed overall economic uncertainty as high or very high in August 2020 when we conducted our survey.

Yet our Covid survey shows that banks’ investment and interest in ML and DS has held up. The strategic imperative to drive efficiency through automation and has perhaps been reinforced by the low interest rate environment. Furthermore, the nature of this shock means that demand for banking and other financial services may not have suffered to the same degree as other industries like hospitality, given the extent and impact of lockdown measures.

The pandemic has also catalysed more extensive use of computers and smartphones for commerce, remote working and socialising. This has likely increased the amount of data businesses have available to them. This in turn is likely to increase demand for data scientists, data engineers and other IT professionals. Ultimately, if necessity is the mother of all invention, then Covid has arguably accelerated demand for data and technical innovation.

Banks have benefited from ML during the pandemic

In March 2020, the Bank of England put in place a package of measures to help mitigate the economic shock resulting from Covid. The UK Government also provided a range of financial support for businesses, including government-guaranteed loan schemes. As noted earlier, some UK banks used ML to process the high volume of government-guaranteed loan applications, resulting in increased operational efficiency.

As the emphasis was on providing finance to businesses quickly during the early stages of the pandemic, lenders were given a 100% government guarantee on Bounce Back Loans, and borrowers could apply in a streamlined process with no assessment of their creditworthiness. Market intelligence suggests that banks are now using ML to enhance their credit risk management and to help identify and manage higher risk loans within certain portfolios, some of which may be expected to have higher default rates compared to other loan portfolios.

Covid may amplify certain risks associated with ML

As our survey highlights, Covid has had a negative impact on the performance of some ML models. This is linked to the fact that ML models’ performance can change or deteriorate under conditions different to those displayed in the data on which they were originally trained. This can occur either when the underlying data changes (data drift) or the statistical properties of the data change (concept drift). The Covid crisis has led to both data and concept drift, which has challenged models in unusual and unexpected ways. Therefore, monitoring for data drift and concept drift is one of the key challenges for firms to ensure appropriate risk management.

Our survey also showed that small banks have increased their use of third-party providers of data, infrastructure, and off-the-shelf or bespoke ML models as a result of Covid. As previously mentioned, while there are many advantages to outsourcing and third-party provider models, they can carry additional operational risks that may be amplified as banks seek to integrate new ML applications into existing legacy IT systems.

How Covid has impacted the Bank of England’s approach to ML and DS

It is important that we understand how ML and DS is being used by the firms we supervise and regulate. We must also understand the micro or macroprudential policy implications of their use, and how these could evolve as the techniques and technologies develop and become more widely adopted. Ultimately, we aim to support the safe and productive deployment of these technologies across the financial sector.

AI Public-Private Forum

In October 2020, we launched a new AI Public-Private Forum (AIPPF) along with the FCA. The AIPPF, a year-long project that was conceived before the pandemic, brings together a range of experts from financial services, the technology sector, and academia, to:

• share information and understand the practical challenges of using AI and ML within financial services, any barriers to deployment, and any potential risks or trade-offs;
• gather views on areas where principles, guidance, regulation or good practice examples could support safe adoption of these technologies; and
• consider how continued engagement following the end of the project could be useful in supporting safe adoption and if so, what form this could take.

At its first meeting, the AIPPF discussed several topics, including, but not limited to, the impact of Covid. Members of the forum highlighted some of the ways existing risk management and governance frameworks could be adapted to accommodate ML and AI. They identified some of the key barriers to deployment, such as access to data, a lack of standardised definitions for key terms, and skill gaps. The members also explored the merits of practical guidance versus high-level principles, and how financial services can look to other sectors for good practice examples. These are some of the issues the AIPPF will examine in further detail in 2021.

Further research on AI

In September, we published our new research agenda, which included the use of AI in finance as a key topic. Research on this topic will help us to understand, for example, if algorithmic trading could result in collusive pricing or pose potential risks to financial stability. Throughout 2020, we also published research using ML and DS, for example, showing the promise of ML models to outperform conventional forecasts of macroeconomic indicators such as inflation and GDP, and identifying meaningful but previously unknown correlations in the pattern of financial crises.

In addition to publishing research, we regularly host conferences and webinars with leading experts from around the world. In March 2020, we hosted a conference titled ‘The impact of ML and AI on the UK economy’, which covered everything from the impact of AI on employment to the ethical issues raised by this technology. The conference was followed by a webinar in August examining the impact of Covid on financial firms’ adoption of AI.

Internal application of ML and DS

The pandemic has also prompted an increase in the number of DS applications used within the Bank of England. This is largely because the speed and depth of the downturn has created demand from policymakers for higher-frequency indicators of economic activity.

In response, we have assembled a range of new informational assets to inform policymaking, often using DS techniques to do so. These include scraping websites and mining large transactional databases. Our researchers are also using ML to understand how the various trade-offs made by households in response to the pandemic might impact the economy under different scenarios.

Our pre-Covid commitments to invest in ML and DS have continued. For example, we have procured technology to improve the discoverability of documents, and patterns within documents, to speed up supervisory review of material received from firms.

Conclusion

Covid has required businesses to rapidly rethink their strategies and investment plans. Our survey suggests that it has continued to stimulate interest in and adoption of ML and DS in UK banking. And ML and DS now also sit higher on the priority list for policymakers because of their increasing use and, alongside their benefits, their potential risks.

While we recognise that the increasing use of ML and DS will bring benefits to banks, consumers and the economy, we must also consider the associated risks and challenges, and how to mitigate these. Banks and other financial services firms have already highlighted that risk management and governance frameworks will have to evolve in line with increasing maturity and sophistication of ML techniques. As our previous work and the first AIPPF meeting highlight, banks and other firms are aware of a widening gap between innovation and governance, and between the demand for data science skills and their supply. These gaps may persist in the near term as the pace of adoption increases.

We will continue to monitor these developments to identify potential risks, and work closely with a wide range of stakeholders to promote the safe adoption of these new technologies. We will also continue to consider whether new policy initiatives may help firms to realise the benefits and more effectively manage the risks of AI, ML and DS.

Annex

• ML and DS survey questions

  To help us understand the impact of Covid on ML and DS in the UK banking sector, we conducted a voluntary survey of Prudential Regulation Authority (PRA) regulated banks and insurers in August 2020.

  The survey consists of 32 submissions in total, with 17 from UK banks, nine from foreign banks with operations in the UK, and six from insurers. The sample of insurers was too small to be judged representative of the sector and the results are not included in this article. Note that, although the survey only covers 26 banks, the assets of those banks account for around 88% of all UK bank assets.

  The following questions were sent to banks and insurers via PRA supervisors.

  1. To what extent has Covid-19 changed the number of ongoing applications now relative to before the pandemic?

  2. To what extent has Covid-19 affected the funding and resourcing of existing applications underway before Covid-19?

  3. To what extent has Covid-19 affected the number of planned but not yet started applications?

  4. To what extent has Covid-19 affected the funding and resourcing for planned but not yet started applications?

  5. Compared to the pre-Covid-19 period, how has your view changed about the importance of machine learning and data science for the future operations of your firm?

  6. To what extent has Covid-19 impacted your planned investment and applications in the following areas?

  • Asset management
  • Commercial
  • Corporate & institutional
  • General insurance
  • Investment banking
  • Life insurance
  • Reinsurance
  • Retail
  • Treasury
  • Wealth management
  • Algorithmic trading
  • Anti-money laundering, countering terrorism financing and fraud detection
  • Credit (origination, pricing, provision, etc.)
  • Customer/client engagement
  • Cybersecurity and technology function
  • Data management
  • Internal operations and strategy
  • Payments, clearing, custody & settlement
  • Regulatory reporting and compliance
  • Risk management

  7. As a result of Covid-19, what issues (opportunities and challenges) have your existing machine learning and other data science application encountered?

  • Business volume eg loan applications or claims management
  • Customer engagement
  • Data supply issues
  • Financial crime
  • Funding
  • Hiring and retention of skilled staff
  • Model performance
  • Resourcing eg redundancies or redeployment of staff
  • Risk appetite for machine learning and data science projects
  • Technology issues related to virtual working
  • Other - please specify

  8. To what extent has Covid-19 impacted your use of third-party providers/vendors?

  • Financial data
  • Alternative data
  • Data preparation / management
  • Data collection
  • Models (deployment, training, selection)
  • Models (testing and validation)
  • Underlying platform (including operating system, middleware, runtime)
  • Underlying infrastructure (including networking, storage, servers)