The adoption of Accounting Standards Update (ASU) 2016-13, as amended, which introduced the current expected credit loss (CECL) methodology for estimating loan losses, has made it necessary for many banks and other financial services organizations to implement entirely new financial models to calculate the allowance for credit losses (ACL) for financial reporting.
As part of the CECL implementation plan, financial institutions that follow the supervisory guidance on model risk management likely will execute a model validation. The validation likely will be conducted by an independent party that can effectively challenge the model data, design, processing, outcome, and alignment to the prescribed CECL methodology.
With implementation of the new standard now nearly complete at many Securities and Exchange Commission filers that are not smaller reporting companies, model validation is identifying areas of concern that financial institutions need to address to successfully manage the transition.
Qualitative factors
During the Sept. 3, 2019, webinar “Ask the Regulators: Applying Model Risk Management to CECL Models at Large Banks,” hosted by the Office of the Comptroller of the Currency, Federal Reserve, and Federal Deposit Insurance Corporation,1 it was communicated that qualitative aspects of CECL outside of modeling (and therefore model risk management) should be subject to appropriate governance and controls but not necessarily validation. However, models that are used to estimate qualitative aspects of CECL are subject to model risk management and should be subject to validation. ASU 2016-13 and the proposed “Interagency Policy Statement on Allowances for Credit Losses,”2 detail qualitative factor considerations that are similar to, but expand upon, the factors included in the “Interagency Policy Statement on the Allowance for Loan and Lease Losses.”3
Many financial institutions are choosing to make use of the existing qualitative factor frameworks they had used under the previous incurred loss methodology for estimating an allowance for loan and lease losses. In doing this, financial institutions should be careful to take two important steps:
1. Identify and exclude the qualitative factors that are explicitly captured in the historical loss rate calculation. Certain economic and credit quality trends can be explicitly captured by the historical lifetime expected loss rate estimation methodology. For example, when a logistic regression probability of default model is used to estimate lifetime expected credit loss, the unemployment rate is typically an important independent variable. In this case, the model itself accounts for any differences in unemployment rate as of the measurement date and the reasonable and supportable forecast period compared to the historical period. As another example, a risk rating migration approach eliminates the need for a qualitative factor adjustment for credit quality, as it should be captured by the financial institution’s risk rating process.
2. Modify the adjustment range. Under the incurred loss estimation methodology, historical loss rate calculations were commonly based on an annualized loss rate. The calculations that were used to adjust historical loss rates to reflect various qualitative factors were anchored to a range of these annualized rates. This scale likely will need to be adjusted for CECL purposes because the loss rate calculation is now a lifetime loss rate estimate, rather than the annualized loss rate that was used under the incurred loss approach.
Model risk management
The 2011 joint regulatory publication, “Supervisory Guidance on Model Risk Management” (OCC 2011-12, SR 11-7, FIL-22-2017),4 is applicable to financial institutions of varying asset sizes5 and has been adopted by organizations to varying levels of maturity. One major section of this guidance is devoted to governance, policy, and controls issues related to the use of models. In addition to challenging the model’s data, design, assumptions, and estimates, a comprehensive CECL model validation also assesses if the proper model risk management governance has been established.
During validation processes, it frequently is observed that appropriate governance is lacking. This shortcoming could be due to insufficient model risk management policies and procedures, or it might simply reflect a lack of attention due to the intense focus given to establishing the estimate itself.
Among the important model risk management governance items banks should have in place are:
- Model documentation that details key model components including (but not limited to) portfolio segmentation, model design, assumptions and limitations, alternative approaches, data sourcing and preparation, and outcome analysis.
- A change control procedure and log to track model changes. The procedure and log should outline how changes are documented, tested, reviewed, and approved.
- Model procedures that detail the steps needed to operate the model.
- An ongoing monitoring plan which, at a minimum, specifies the frequency of monitoring, tests to be performed, metrics to be tracked with associated thresholds and triggers, and corrective actions in case of breaches.
Sensitivity analysis
The model risk management regulatory guidance states, “Banks should employ sensitivity analysis in model development and validation to check the impact of small changes in inputs and parameter values on model outputs to make sure they fall within an expected range.”6 Depending on the CECL model deployed, there are several inputs and design decisions that drive the CECL estimate.
These inputs might include prepayment rates, assumptions regarding the probability of default and loss given default, forecast scenarios and horizons, and the reversion approach that is used. Those in charge of model development should conduct sensitivity testing as appropriate to understand the impacts these variables will have on the CECL estimate.
Origination and renewal dates
CECL requires institutions to estimate the expected credit loss based on the contractual term without considering renewal or extensions, unless a troubled debt restructuring is expected. In reviewing the historical lifetime loss estimates, the calculation logic often does not compute the appropriate contractual life. For example, a working capital line of credit that originated in 2013 and renewed in each of the next five years generally should be identified as six separate one-year contracts, not one contract for six years.
However, some financial institutions retain the same note number throughout each of the renewal periods. If the calculation logic does not consider this practice, it might inaccurately assess the contractual term of the loan by looking at the “origination date” field, instead of the “last renewal date” field. In the example of the line of credit originated in 2013, this approach would result in the calculation logic inaccurately computing the life of the loan at six years. Additionally, the historical loss experience of five one-year loans having been originated and repaid with no loss would not be captured in the loss estimate.
Appropriate procedures should be in place to be certain both date fields are properly maintained and that calculation logic is applied consistent with the institution’s use of the fields. Without doing so, it could be impossible to determine the correct vintage or term. This shortcoming could lead to reporting inaccuracies in both the estimation models and the financial disclosures.
Vendor models and underlying assumptions
In implementing the CECL methodology, many financial institutions use vendor models to help them estimate their allowance for credit losses. The “Supervisory Guidance on Model Risk Management” states that “Banks are expected to validate their own use of vendor products.”
While using a vendor for the calculation can be appropriate, model owners should fully understand the selected model’s parameters and clearly document their approach’s underlying assumptions.
Depending on the specific modeling methodology being used, the needed documentation should answer questions such as these:
- Is default risk considered in the period of maturity?
- If using a discounted cash flow model, are cash flows re-amortized after each payment or treated as fixed payments?
- Are various payment structures (such as fully amortizing principal and interest payments, fixed payments, balloon loans, or interest-only payments) handled differently by the model? If so, are they treated in alignment with management’s expectations or are there modeling limitations?
- How does the model handle changing payment structures, such as a loan with an interest-only period that then converts to a term loan with fixed principal and interest payments?
Forecasting and statistical testing
Many financial institutions turn to statistical modeling techniques, most often linear regression models, to incorporate the required macroeconomic forecast adjustments into their CECL estimates. Ordinary least squares regression, a commonly used approach, typically includes regression assumptions, including, among others:
- Linear parameters. Assumes the dependent variable can be approximated by a linear combination of independent variables. If this assumption does not hold, the model might not be robust or adequate.
- Multicollinearity. The independent variables should not be collinear or highly correlated with each other. Multicollinearity, when the variables are collinear, causes unstable coefficients and inflated t-statistics or p-values of the coefficients potentially biasing the economic and statistical inference.
- Serial correlation. Assumes the model’s residuals are uncorrelated. Autocorrelated residuals might lead to biases in the t-statistics or p-values of the coefficients, and inflated R-squared, potentially biasing the economic and statistical inference.
- Homoscedasticity. Assumes the errors in the regression have a constant variance. Heteroscedasticity is the opposite, when the errors have nonconstant variance. Heteroscedasticity might lead to biases in the t-statistics or p-values of the coefficients, and inflated R-squared, biasing the economic and statistical inference.
- Residual normality. Assumes the residuals of the regression equation are normally distributed. Nonnormality might lead to biased t-statistics or p-values.
When the model development team is selecting the final models and supporting their robustness, the team often does not focus on statistical testing of these underlying regression model assumptions. Validation of the appropriate regression assumptions should be performed for any statistical model being used. Failing to test the regression assumptions and to understand the results exposes organizations to the risk of incorrect inferences on the robustness of the forecasting models.
Finally, bear in mind that even after the transition to the new CECL methodology has been completed, issues such as these will continue to be ongoing concerns for financial institutions. As banks, model developers, and regulators become more familiar with the effects of the new CECL methodology, the financial models used to calculate the ACL will undoubtedly undergo revisions. Moreover, changes in markets, products, customer base, or other factors also can affect a model’s performance.
As a result, the various assumptions, analytical tools, governance structures, and implementation practices discussed here will require continuing and ongoing validation to verify the models used for estimating expected credit losses are still working as intended.
1 Office of the Comptroller of the Currency, “Ask the Regulators: Applying Model Risk Management to CECL Models at Large Banks,” webinar announcement, https://www.occ.treas.gov/topics/supervision-and-examination/bank-operations/accounting/current-expected-credit-losses/mrm-cecl-webinar.pdf
2 Office of the Federal Register, “Interagency Policy Statement on Allowances for Credit Losses,” Federal Register: The Daily Journal of the United States Government, Oct. 17, 2019, https://www.federalregister.gov/documents/2019/10/17/2019-22655/interagency-policy-statement-on-allowances-for-credit-losses
3 Board of Governors of the Federal Reserve System, “Interagency Policy Statement on the Allowance for Loan and Lease Losses,” https://www.federalreserve.gov/boarddocs/srletters/2006/SR0617a1.pdf
4 "Supervisory Guidance on Model Risk Management,” Office of the Comptroller of the Currency online publication, April 4, 2011, https://www.occ.gov/news-issuances/bulletins/2011/bulletin-2011-12a.pdf
5 FIL-22-2017 is applicable to FDIC-supervised institutions with less than $1 billion in total assets.
6 "Supervisory Guidance on Model Risk Management,” Office of the Comptroller of the Currency online publication, April 4, 2011, https://www.occ.gov/news-issuances/bulletins/2011/bulletin-2011-12a.pdf
