Audit of Supervisory Processes – Systemically Important Banks Group

1. Background

OSFI supervises federally registered banks and insurers, trust, and loan companies with the primary goal of safeguarding depositors and policyholders from loss. The focus of supervisory work is to determine the impact of current and emerging risks, both from internal and external environment, on a Federally Regulated Financial Institution (FRFI)’s risk profile.

Supervisory Framework and Core Process

Supervisory work is governed by the Supervisory Framework that describes the principles, concepts, and core process that OSFI uses to guide its supervision of FRFIs. The supervision of FRFIs is principle-based, which requires application of sound judgment in the identification and assessment of risks. The intensity of supervision depends on the nature, size, complexity, and risk profile of the FRFI, and the potential consequences of the FRFI’s failure.

Foundational standards to support the Supervisory Framework have been developed by a central group, which apply across the supervision teams to ensure consistency of supervisory work. Complementing these standards, sector-specific policies and procedures have been established, which are tailored to the supervisory work performed by the respective sector. The Vu platform was implemented in recent years as OSFI’s new system of record for all supervisory work. Supervisory work assessing a FRFI’s risk profile is documented in Vu Assessment Topics (ATs), using pre-defined topics to better organize the assessments and facilitate comparison.

OSFI uses a defined process to guide its FRFI-specific supervisory work, which is a dynamic and continuous process. The core supervisory process includes the following key supervisory activities:

Text description - Core Supervisory Process

OSFI uses a dynamic and continuous process to guide its FRFI-specific supervisory work, which includes supervisory strategy & planning, monitoring & on-site reviews, reporting and intervention, issues management, etc.

• Annual planning consists of the preparation of Supervisory Strategies which outline the supervisory work planned for the next three years.
• Monitoring refers to the regular review of information on the FRFI and its industry and environment, to keep abreast of internal and external changes. Enhanced monitoring includes ad-hoc information requests beyond regular monitoring on additional risks.
• Review refers to more extensive supervisory work than monitoring. It can be onsite which has an extensive scope and provides an in-depth analysis of various risks and risk management of the FRFI. It can also be offsite which shares the same features and objectives as an onsite review but with a limited scope.
• Reporting and intervention is the process OSFI uses to communicate the assessment results to the FRFI through various formal, written reports (i.e., Annual Supervisory Letter, Interim Supervisory Letter, Staging/De-Staging Letter).
• Issues management is how OSFI monitors and follows up on FRFI action plans and closure evidence against findings (referred to as “issues” since the implementation of Vu), to ensure risks are adequately addressed by the FRFI in a timely fashion.

Systemically Important Banks (SIBs) Group

During the audit scope period, the SIBs Group in the Toronto office was a subset under the Deposit-Taking Supervision Sector (DTSS), which was responsible for supervising five of the six Systemically Important Banks in Canada to determine whether they were in sound financial condition and were complying with their governing statute law and supervisory requirements. The Lead Supervision (LS) teams acted as the relationship managers with the FRFI and conduct the core supervisory activities mentioned above, supported by specialists from the various RSS teams. The Banking Central Office team also supported DTSS’s risk assessment accountabilities through its work on a variety of liaison activities and operational functions.

As of April 1, 2022, OSFI’s supervisory activities have been re-organized. The audit report refers to DTSS as the home of the SIBs Lead Supervision team, supported by teams of specialists within the Risk Support Sector (RSS) and the Common Supervisory Services (CSS) group. Many of these functions reside within the new supervisory sector; while the findings describe the activities as they took place during the scope of the audit, the recommendations and management action plans reflect the new organizational structure and accountabilities.

Previous Audit Engagements

No audits have been performed on DTSS in the past six years. Two prior audits were conducted for Deposit-Taking Group Conglomerates and Non-Conglomerates in 2012 and 2014 respectively before the sectors were re-structured.

2. About the Audit

2.1 Objective

The objective of the engagement was to assess the adequacy, effectiveness and efficiency of control activities related to OSFI’s supervisory processes.

2.2 Scope

The audit covered supervisory activities performed by the SIBs Group between April 1, 2019 and March 31, 2021, and focused on assessing the following:

• Design and operating effectiveness of key supervisory activities and controls, including annual planning, review, monitoring, issues management, intervention, annual approval, and annual letter process; and
• Compliance with applicable supervisory framework, standards, and policies.

The audit covered the periods before and after COVID-19, which required the SIBs Group to adjust their planned supervisory work. It also covered the periods before and after the implementation of Vu.

2.3 Approach and Methodology

The audit was conducted through performance of the following procedures:

• Reviews of applicable frameworks, standards, and policies, including CSS and sector-specific standards and guidance;
• Walkthroughs and interviews with senior management and supervision team members from each of the SIBs teams; and
• Sample testing using both statistical and judgmental sampling for all key supervisory activities.

2.4 Statement of Conformance

This audit was conducted in conformance with the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing, consistent with the Treasury Board’s Policy on Internal Audit, and as supported by the results of the Quality Assurance and Improvement Program.

3. Overview of Audit Results

3.1 Summary of Results

Supervisory activities conducted by the SIBs Group were generally in adherence to existing standards and formalized processes, and were documented with adequate support. Some opportunities for improvement were identified by the audit, including formalizing aspects of the annual planning and issues management processes, strengthening data validation controls to support performance monitoring and reporting, and improving review and approval evidence of key supervisory documents.

While the improvement opportunities in this audit report are directed to supervision of the SIBs, all OSFI supervision teams are encouraged to review the findings for applicability.

3.2 Management Response

Management accepts the findings and has identified Management Action Plans for each recommendation as outlined in the relevant sections, with all recommendations to be addressed by Q2 2024-25.

4. Observations and Recommendations

4.1 Annual Planning

 

The annual planning process establishes supervisory work for the upcoming year and includes a risk-based estimate of the overall resources required to meet the sector objectives. It consists of planning for FRFI-specific work (e.g., reviews, monitoring, issues follow-up, etc.), as well as non-FRFI specific work (e.g., OSFI projects, sector initiatives, etc.).

Alignment with Sector RTF

The Risk Tolerance Framework (RTF) is a document that establishes the type and degree of risk OSFI is willing to assume in its supervision of FRFIs and identifies the required intensity of supervision. The Supervisory Strategy, which identifies the FRFI-specific activities needed to keep the FRFI’s risk profile up-to-date, is updated each year during annual planning, and is expected to align with the sector RTF.

While all five Supervisory Strategies sampled were updated and reviewed during the annual planning process, one sample was not fully aligned with the sector RTF. According to the DTSS RTF, review of a FRFI’s high/moderate significant activity - which is considered a fundamental element of a FRFI’s business model - should be performed approximately every five years. Within the sample, two reviews for moderate significant activities are currently planned for greater than the five-year cycle from the last review date (nine and ten years) due to resource constraints identified by management.

As the sector RTF establishes general expectations for the nature and frequency of the application of supervisory activities, planning supervisory work outside of the sector’s expectations can result in increased information decay and an inaccurate assessment of the FRFI’s risk profile.

Resource Planning

Along with the Supervisory Strategy, two key documents are used in the planning process: the Supervisory Intensity Model (SIM) leverages the sector RTF to develop key assumptions and expected days for FRFI-specific supervisory work and calculates the required resources for each Lead Supervision (LS) team; and the Planning Sheets determines the planned number of days at the more detailed activity level based on actual resources available.

However, the current planning tools may not be effective in supporting management’s resource planning decisions. For instance, the calculated resources in SIM are an incomplete projection of total resources, as they do not encompass non-FRFI specific work. Audit sample testing indicated a significant number of days required for non-FRFI specific work, ranging from 180 to 544 days, was not accounted for in the resource needs. Consequently, when the SIM is shared with senior management to inform the resource planning, the true resource needs are not shown. Moreover, as planned days at the detailed activity level in the Planning Sheets are driven by actual resources available, they always equal to the approved resources and do not highlight any resource gaps.

Without a full and clear picture of how many resources are required to perform all planned activities, the resourcing constraints, and how those constraints align with established risk tolerances, management may not be in a position to direct resources to the highest priorities or highest risks throughout the year.

Tracking Plan Changes

The DTSS Internal Control Policy requires that a Change Request Form be filled out for any significant change to planned reviews, and an approval be obtained. The Banking Central Office team supports the sector by maintaining a Work Movement Report that tracks changes to the approved planned supervisory work.

For all five samples tested, Change Request Forms were completed and approved for any significant changes to the planned reviews, as required by the Internal Control Policy. However, there are currently no requirements to track significant changes to the planned enhanced monitoring work in a similar manner to planned reviews. While enhanced monitoring by design is a fluid process which allows flexibility in adding enhanced monitoring work throughout the year, there is some enhanced monitoring work that is planned in advance, such as additional work for staged FRFIs, thematic studies, and monitoring of regulatory changes implementation. Incomplete tracking of changes to the plan could result in misestimations of resources required and unsupported resource allocation decisions.

It was also found during testing that while some enhanced monitoring work was set up as separate risk assessments, most was embedded within the regular monitoring work. This approach hinders clear identification and tracking of enhanced monitoring work, and makes it difficult to ensure work has been conducted as intended to address emerging or additional risks.

Risks Monitored at OSFI Risk Committees

One key principle in the Supervisory Framework is that the assessment of risk should be continuous and dynamic to reflect changes to risks in both the FRFI and its operating environment. Thus, the supervisory plan is expected to be adjusted throughout the year for significant changes in current and emerging risks.

To support the governance of cross-sector industry-wide risks, OSFI has established risk committees to discuss these risks and to help inform supervisory work. For much of the scope of the audit, these risks were discussed at the Emerging Risk Committee, before it was replaced by the Business Risk Committee in March 2021. While these risk discussions were regularly held, there was no formal mechanism to ensure that FRFI plans considered the direction of the risk committee meetings, or that risk prioritizations were incorporated into supervisory activities. Management has indicated that the incorporation of Business Risk Committee priorities has become more formalized in the current annual planning cycle, which fell outside of the audit scope and could not be validated.

Without a formal mechanism, risks monitored at OSFI Risk Committees could be missed or not fully incorporated in supervisory work, which could lead to limited resources being allocated to lower risk areas and misaligned FRFI risk assessments.

4.2 Issues Management

 

Through supervisory work, specific concerns about a FRFI’s governance, risk management, or control activities are identified and formally communicated to the FRFI as “issues.” Recommendations are used to formally communicate OSFI’s expectations that FRFIs will remediate the issues. It is critical that recommendations are followed up in a timely manner by the responsible supervision teams to ensure the supervisory concerns are properly addressed.

Action Plans

According to the Issues Management Standard, to address OSFI recommendations, FRFIs are required to provide action plans and timeframes for corrective actions, which are then assessed by the LS and specialists for adequacy. When an assessment of the FRFI action plan is complete, the LS should acknowledge receipt, and identify concerns or deficiencies, as appropriate.

For 81% (21 of 26) of samples tested, documentation could not be located to demonstrate that the associated action plan had been assessed for sufficiency. Where documentation could be located, 60% (3 of 5) could not demonstrate approval and 80% (4 of 5) could not demonstrate communication to the FRFI. Interviews with various LS teams suggested that there was no common understanding on how action plans should be assessed, the approval required, and whether assessment of satisfactory results need to be communicated back to the FRFI. The ambiguity of the current Standard on the expectations of action plan assessment and communication can result in inappropriate action plans being implemented by the FRFIs, leading to issues not being adequately addressed.

Follow-Up of Recommendations

The Issues Management Standard requires that the supervisors follow up on recommendations and receive evidence of closure from FRFIs, with the support of specialists from RSS as needed. The five FRFIs supervised by the SIBs Group proactively provide updates of issues status to OSFI on a quarterly basis.
The SIBs Group currently relies on the FRFIs’ quarterly reporting for issues updates, supplemented by regular touchpoints with the FRFIs. However, the FRFIs’ quarterly reporting dates do not necessarily align to the issue target dates. Consequently, the supervision team may overlook overdue issues between quarters or receive updates after target completion dates. In sample testing, 69% (9 of 13) of the closure evidence was submitted by the FRFIs after the agreed target date (on average 52 days late), and there was no evidence of follow-up conducted by the LS teams. Sole reliance on FRFIs’ proactive quarterly updates and informal touchpoints is not sufficient to ensure that issues are closed prior to the agreed-upon dates.

Without formalized monitoring of target completion dates and follow-up process, issues might not be closed in a timely manner, which could lead to FRFIs’ prolonged exposure to risks.

Target Date Extension

FRFIs are responsible for providing closure evidence to OSFI to support the execution of action plans based on the agreed-upon timelines (i.e., target date). Where the FRFI cannot meet the originally agreed-upon timeline, they can request an extension and a new target date can be set.

There is currently no guidance or formalized process for assessing, escalating, and approving the target date extension. In sample testing, 75% (9 of the 12) of the issues that had target date extensions, documentation to support the request and approval of extension could not be located. Also, based on the samples, the average days of extension was 437, ranging from 76 to 1,490 days from the original target date. Without a formal assessment and approval process, it is difficult to assess whether the long extension granted was reasonable and well supported.

Issue Closure

The Issues Management Standardrequires supervisors to specify the nature of work required to assess closure of an issue and lists three primary means of closure, i.e., desk review, effectiveness testing, and third-party confirmation. It also requires that the assessment of closure should be well supported to demonstrate satisfactory resolution.

As “means to close” are indicative of the level of effort needed to close issues, there is currently no clear guidance or structured approach to align the level of closure effort to issue impact. Sample testing showed inconsistencies in the ways issues were closed, in that different “means to close” were used when the issue impact was the same; conversely, the same “means to close” was used when the issue impacts were different. While some judgment is needed to select a method for closing an issue, without sufficient documentation of the rationale, it is difficult to assess whether the method selected to close an issue was appropriate.

Furthermore, it is unclear whether there is a minimum level of documentation required to support issue closure assessment and whether the extent of documentation should be based on issue impact and complexity. In our samples reviewed, there were various mechanisms used to record assessment results and the level of details varied between issues.

4.3 Performance Monitoring

 

OSFI has established Key Performance Indicators (KPIs) to formally track and monitor the performance of certain supervisory activities. The Vu application, supported by the CSS Technology and Tools team, was launched over three releases from November 2019 to October 2020. It is now the source system for KPI reporting as the supervisory work for all supervision teams is recorded in the system.

One KPI relates to the timely assessment and communication of issue closure evidence. Based on sample testing, 17% (3 of 18) of the RSS-owned issues did not demonstrate response time within the required 150 days, which is below its 90% target. IA’s sample testing also showed that some key Vu fields (e.g., “FRFI to LS/Specialist re Remediation” date, “LS to FRFI re Completion” date) were not filled out consistently or accurately by supervision teams according to the supporting documentation. While several tools (e.g., Vu Glossary) are available to support use of Vu, as the system is relatively new, the supervision teams have not fully utilized these tools, and may not have been aware of them. As some of these Vu fields are used in KPI reporting, inaccurate data would have led to inaccurate KPI reporting, which would have prevented management from effectively monitoring performance. The supervision teams have not established supplementary internal controls to validate the accuracy and completeness of Vu data input.

The inaccurate data input and inconsistent use of Vu key fields could result in inaccurate KPI reporting which limits management oversight over its team’s performance and restrain the sector’s ability to compare performance across teams.

4.4 Review and Approval of Key Supervisory Documents

Evidence of Review and Approval

Management’s review and approval of supervisory work ensures that OSFI’s assessment of the FRFIs’ risk profile is adequate and in adherence to established standards, guidance, and management expectations. The Approval Authorities Standard requires that the evidence of review be documented. Since implementation of Vu, evidence is required to be documented directly in Vu or linked to Vu.

However, for review methods not directly documented in Vu, there is no specific guidance on the baseline evidence of review. Multiple instances were found in sample testing where reviews and approvals of different key supervisory documents were either undocumented or without formal sign-off. The exceptions were applicable to both the SIBs Group and RSS teams, for various supervisory activities including annual planning, review, monitoring, and annual letter.

For the samples where review and approval were undocumented, the timeliness of approval could not be verified. For the samples that demonstrated formal approval, some approvals were not granted in a timely manner. Specifically, though FRFI’s risk matrix is reviewed quarterly during monitoring assessment, it must be formally approved at least annually as required by the Supervisory Documentation Standard. According to the dates recorded in Vu, 40% (2 of 5) of annual approvals sampled were not formally approved within one year when compared to the previous annual approval date.

Without clear guidance on what the baseline evidence of review should be, review and approval of key supervisory documents might not be fully demonstrated, which is necessary in ensuring the adequacy and timeliness of supervisory decisions.

Appendix A - Recommendation Ratings

Recommendations are ranked in order to assist management in allocating resources to address identified weaknesses and/or improve internal controls and/or operating efficiencies. These ratings are for guidance purposes only. Management must evaluate ratings in light of their own experience and risk appetite.

Recommendations are ranked according to the following definitions:

• High Risk: should be given immediate attention due to the existence of either a significant control weakness (i.e., control does not exist or is not adequately designed or not operating effectively) or a significant operational improvement opportunity.
• Medium Risk: a control weakness or operational improvement that should be addressed in the near term.
• Low Risk: non-critical recommendation that could be addressed to either strengthen internal control or enhance efficiency, normally with minimal cost and effort. Individual ratings should not be considered in isolation; and their effect on other objectives should be considered.