Enterprise-Wide Model Risk Management Guideline - Letter (2017)
Information
Table of contents
To: Banks, Bank Holding Companies, Federally Regulated Trust and Loan Companies, Cooperative Retail Associations
OSFI is publishing the final version of Guideline E-23, which establishes OSFI’s expectations for institutions in managing and controlling the use of models, whether for regulatory capital determination, internal risk management, valuation/pricing, business decision-making or stress testing purposes. OSFI believes all Canadian deposit-taking institutions should have model risk management practices that reflect their size and degree of sophistication.
The final Guideline incorporates several revisions based on comments received from industry stakeholders during the public consultation period, which began in December 2016. This consultation included targeted discussions with several foreign bank branches on the proposed scope of coverage. Those discussions have led OSFI to exclude foreign bank branches from the scope of the Guideline, since OSFI believes that it has the capacity to assess model risk management of these institutions within existing supervisory processes. A footnote to the Guideline clarifies the Principal Officer of a foreign bank branch is accountable for ensuring there are appropriate risk controls over model risk, where material. Annex 1 summarizes comments of substance that were received from stakeholders and explains how they have been addressed.
Additionally, OSFI has decided that standardized institutions, as defined in the Guideline, will have until January 1, 2019 to become compliant with this Guideline. All other institutions (i.e. internal models approved institutions) are expected to comply with the Guideline by November 1, 2017.
Questions on the Guideline should be sent to Greg Caldwell, Capital Specialist, Capital Division by email at Greg.Caldwell@osfi-bsif.gc.ca.
Yours truly,
Carolyn Rogers
Assistant Superintendent
Regulation Sector
ANNEX – Summary of Comments Received and OSFI’s Response
| Comment | OSFI Response |
|---|---|
| Overall | |
| One commenter was supportive of the guideline as long as it remains scalable and flexible for smaller standardized institutions (SIs). The commenter believes there are several stages involved in demonstrating compliance that are a challenge for SIs with more limited resources. Clarity was sought on whether OSFI’s expectations and timelines would appropriately take this into account. |
OSFI is aware of these concerns and has decided to take additional time to develop a supervisory strategy for SIs that is in line with the proportionality principle articulated in the guideline. The timeline for implementation of the guideline by SIs has thus been set at January 2019. |
| Materiality | |
| One institution suggested that the requirement to develop contingency plans for all vendor models in use was too prescriptive. The commenter proposed either a softening to material models or deletion of a sentence in Section 6 referring to contingency plans. Another institution suggested that in Section 9 documentation should apply more specificity to material models. |
OSFI believes there should be a risk sensitive application of this guideline as a principle. As such, contingency plans are important for any material model that is used. Hence the wording in Section 6 has been modified but there is a minimum expectation that institutions have contingency plans for material models if they intend to rely on vendors. Likewise, OSFI wants to ensure that institutions, at a minimum, have appropriate documentation for their most material models. |
| Section 1: Scope of Coverage | |
| A number of foreign bank branches (FBBs) suggested that FBBs should not be within the scope of the guideline. They felt inclusion would create duplication and additional burden already faced by firms from their head office by their home regulator. It was suggested that compliance with approval status by the domestic regulator of group risk models would be difficult and violate national confidentiality laws. Finally, it was suggested that FBBs could work with regulators to develop some comfort language and compliance. |
OSFI does not believe the guideline would create duplication but believes it has the capacity and means to ensure a level playing field regarding model risk management. Consequently FBBs have been removed from the scope of application for the final guideline; however, a reference has been incorporated to guideline E-4B Role of the Principal Officer and Record Keeping Requirements and the responsibilities of the Principal Officer to be accountable for appropriate risk controls relating to models. |
| The new accounting models related to expected loss provisioning (i.e. IFRS 9) that integrate accounting with credit risk management will be subject to model risk. It was suggested that it would be helpful to include those models used for accounting models purposes into the scope in footnote 1. | OSFI believes such models should be in scope and the wording of footnote 1 has been modified for greater clarity. |
| Section 2: Definitions | |
| One commenter noted that appropriate model usage for a particular product is typically the domain of model developers (e.g. Quantitative Research) with the model reviewer providing guidance on whether the scope of the proposed model includes the product/trade/purpose in question. They proposed to replace in the definition of model reviewer the wording "recommendations on the usage" with "guidance on the appropriateness." Additional commentary suggested that the robustness of controls are expected to be commensurate with the risks associated with model usage, and proposed to modify the reference in the model approver definition from "on the size and complexity of the institution" to "materiality of model being reviewed." |
OSFI agrees with greater flexibility in the context of internal models approved institutions (IMAIs) but will retain reference to size and complexity since this is relevant for SIs. |
| Section 5.3: Independent review (vetting) | |
| A commenter suggested that this section implied that all models, regardless of materiality or purpose, are subject to the same minimum requirements. If so, such guidance would be very prescriptive. Instead, they proposed that secondary review include appraisal of conceptual soundness and model performance against a success criteria reflective of model purpose and product scope. That might involve evaluation against alternative benchmark models, assessment of model's predictive capacity over a range of assumptions, back testing, etc. | OSFI agrees that the language should be flexible and not be perceived to employ a one-size-fits-all approach to all models. The process at a high level is similar across model categories but each should be individualized to the context of use and the materiality for the institution. |
| Section 5.6: Modifications and decommission | |
| A commenter suggested the guidance could be restrictive and would create bottlenecks for changes and/or updates that have immaterial impact on model output. Provided existence of a robust change control process and availability of procedures for determining materiality of change, they suggested modifying the current guidance so as to allow the model governance function the latitude to validate changes deemed immaterial ex-post vs ex-ante. |
OSFI agrees that some flexibility should be enabled for institutions to prioritize their oversight of model modifications depending on materiality. |
| Section 7: Foreign Banks | |
FBB responses to the draft guideline varied with some indicating that:
|
OSFI agrees that taking a light touch approach may not be appropriate for all FBBs. Further, it would be challenging to uniformly impose extra-territorial requirements on foreign regulators. In light of this, OSFI will continue to rely on requirements on the Principal Officer outlined in guideline E-4B to address any concerns with model risk management processes. The final Guideline E-23 will alert FBBs that OSFI considers it the responsibility of the Principal Officer to be accountable for having appropriate controls of model risk, where material, as described in Guideline E-4B. |
| Section 8: Internal Audit | |
Commenters suggested the removal of the word “accuracy” and its replacement with “completeness” so that the bullet would read:
|
OSFI agrees that an expectation of “accuracy” in documentation creates unnecessary burden and has adopted the proposed wording change. |