Internal Audit Report on Supervision Support Group - Operational Risk Division

1. Background

Introduction

Internal Audit conducts assurance work to determine whether the Office of the Superintendent of Financial Institutions Canada’s (OSFI’s) risk management, control, and governance processes, as designed and represented by management, are adequate and functioning in a manner to ensure risks are appropriately identified and managed, and to ensure compliance with such requirements as policies, plans, procedures and applicable laws and regulations.

An audit of the Supervision Support Group – Operational Risk Division (SSG - ORD) was recommended by the OSFI Audit Committee and approved by the Superintendent for inclusion in the OSFI 2016-17 Internal Audit Plan.

ORD management has reviewed this report and provided their response along with action plans. The report will be presented at the OSFI Audit Committee’s February 15, 2017 meeting for review and approval by the Superintendent.

Context

The Supervision Support Group (SSG) consists of seven distinct support groups that provide support in the form of specialized technical knowledge to the Lead Supervisors’ teams. The Operational Risk Division (ORD) is one of these specialized groups.

ORD’s mandate is to:

• Provide advice and support to Lead Supervisors carrying out monitoring, on-site and early intervention activities at deposit taking and insurance federally regulated financial institutions (FRFIs), with respect to operational risk;
• Support supervisory teams’ efforts to monitor and evaluate system-wide or sectorial issues related to operational risk that may impact institutions negatively; and
• Provide support to OSFI’s Regulation Sector to advance and administer a regulatory framework that promotes the adoption of policies and procedures related to operational risk management.

ORD’s activities are important as operational risk is a key risk in financial institutions. Given OSFI’s integrated supervisory process whereby several aspects of ORD’s activities directly contribute to the FRFI’s overall risk assessment, the potential impact on the institution and consequently on OSFI’s objectives could be material, if operational risks are not properly and timely identified and assessed.

OSFI uses a disciplined, risk-based methodology to supervise FRFIs, consistent with OSFI’s Supervisory Framework (Framework). The Framework describes the principles, concepts, and core processes OSFI uses to guide its supervision of all FRFIs. The Framework provides the conceptual framework to support an effective supervisory process that all supervisory groups, including SSG - ORD, must follow and apply.

2. About the Engagement

Engagement Objective

The objective of the engagement was to assess whether ORD’s supervisory process was risk-based and effectively contributed to OSFI’s supervisory risk assessment process. Specifically, the audit assessed whether:

• ORD’s monitoring and planning activities demonstrated their risk-based approach and allocation of resources;
• Sufficient and relevant evidential matter was available to support ORD’s operational risk related assessments, conclusions and supervisory actions taken; and,
• OSFI’s Supervisory Methodology was appropriately and consistently applied in ORD’s supervisory process followed to identify, assess and report on operational risk related matters at FRFIs.

Engagement Scope

The engagement covered ORD’s activities for supporting Lead Supervisors’ teams in risk assessing their institutions during the fiscal years 2014/15 and 2015/16.

Recognizing that the supervisory process is continuously evolving, IA reviewed documentation relating to events after the audit period chosen for evidence of improvements, as appropriate.

Engagement Approach

The approach to conducting the engagement included:

• A review of ORD’s operational manual and procedures;
• Discussions and walkthroughs with ORD’s staff to understand ORD’s supervisory process and practices followed during their monitoring, planning, assessment, reporting and follow-up activities;
• Examination of selected supervisory documentation prepared by ORD’s teams to assess their application of OSFI’s Methodology; and
• Discussions with Lead Supervisors’ teams and other OSFI groups, as required.

Statement of Conformance

The audit was conducted in conformance with the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing, consistent with the Treasury Board Secretariat (TBS) Policy on Internal Audit and the Internal Auditing Standards of the Government of Canada, as supported by the results of the Quality Assurance and Improvement Program.

3. Observation Ratings

Observation Ratings

Observations are ranked in order to assist management in allocating resources to address identified weaknesses and/or improve internal controls and/or operating efficiencies. These ratings are for guidance purposes only. Management must evaluate ratings in light of their own experience and risk appetite.

Observations are ranked according to the following:

High priority – should be given immediate attention due to the existence of either a significant control weakness (i.e. control does not exist or is not adequately designed or operating effectively) or a significant operational improvement opportunity.

Medium priority – a control weakness or operational improvement that should be addressed in the near term.

Low priority – non-critical observation that could be addressed to either strengthen internal control or enhance efficiency, normally with minimal cost and effort.

Individual ratings should not be considered in isolation and their effect on other objectives and areas should also be considered.

4. Results of the Engagement

Executive Summary

The Operational Risk Division (ORD) staff demonstrated a sound understanding of the operational risks inherent in the business activities of financial institutions. ORD, as a supervision support group, plays an important role in the overall effectiveness of the supervisory process at OSFI.

To facilitate a more effective integration of ORD’s efforts into Lead Supervisor’s supervisory processes, more emphasis can be applied to ensure Lead Supervisors are provided the full context of the operational risk characteristics and potential supervisory concerns that may impact their respective institutions. The two levels of working knowledge (i.e. Lead Supervisors and supervision support group) should be fully coordinated and aligned with OSFI’s Framework.

Positive change initiatives were introduced to enhance ORD’s contribution to OSFI’s supervisory processes, notably enhancements to planning and monitoring supervisory activities as well as staffing changes, including filling several vacancies. These changes appear to be directionally appropriate. However, ORD’s employee learning and development practices aimed at building technical knowledge, skills, and required competencies need further strengthening.

5. Management Response

Response

Management wishes to thank the audit team for the professional and transparent approach in conducting the audit.

Management is committed to addressing the specific recommendations as noted in the Action Plan

6. Observations and Recommendations

1. Management of ORD’s Resources, Training and Development

Medium Priority Observation

To achieve its mandate, ORD requires resources with the specialized skills, up-to-date knowledge and experience. These resources need to have the ability to perform in-depth analyses and apply judgement on a range of complex issues requiring specialized operational risk knowledge. These internal resources should be strategically planned and effectively managed.

The audit revealed that, although there were some training initiatives across ORD and staff had formal “Goal Commitment Documents” (GCD) and Learning Plans in place, these initiatives did not appear to have been designed around training, developing and addressing required knowledge levels for its staff at varying levels over a broader time horizon, to ensure OSFI is not vulnerable to key person dependency risk.

The audit also revealed that the type, the amount of time, and the quality of several learning and training initiatives at ORD did not adequately focus on building and/or updating the technical knowledge, skills and competencies that may be required to support OSFI’s future needs. Although ORD had a training budget, the rationale for its allocation was not always adequately supported, which could lead to a perception of unfairness and unequal opportunities among employees. In addition, ORD’s on-boarding training guidance to expedite the integration of new employees into ORD/OSFI was minimal and out-dated.

Recommendation

Enhancing ORD’s training approach will require the development of a strategy for learning and career advancement aimed at building and maintaining technical knowledge, skills, and required competencies for all staff that links to ORD’s overall business strategy and supports talent management.

In addition, ORD’s on-boarding training guidance for new employees should be periodically reviewed for content relevance and adequacy to help them adjust more effectively into ORD/OSFI work environment.

A skills development strategy is more likely to be successful when employees understand and support it. Sharing and communicating this development plan and soliciting employees input with respect to the knowledge and skills they need to do their work as well as their preferred learning styles could be integral to the plan’s success.

Managers should be formally assessed and accountable for developing their employees to ensure ORD has the required technical and supervisory skills and knowledge at varying levels over a broader time horizon.

ORD’s management should periodically assess skills, knowledge and expertise available in ORD’s pool of resources for its relevance and adequacy to quickly respond to potential emerging issues and/or changes in the external environment where financial institutions operate.

Management Action Plan

The current learning and development plan will be further developed and aligned to maintain currency in technical knowledge. Furthermore, the training and on-boarding will be coordinated with the overall OSFI new employee orientation/on-boarding program as and when implemented.

Management is committed to addressing the learning and development recommendation. The immediate plan is to develop technical learning strategy (September 2017) followed by skills inventory (December 2017) to identify skills requirements. In addition, staff is currently attending industry forums, conferences and business specific programs and workshops.

2. Alignment of Technical Guidance with OSFI’s Framework

Medium Priority Observation

ORD, as a support group, plays an important role in the overall effectiveness of the supervisory process at OSFI. This is particularly true given the potential negative impact operational risk could have on FRFIs, if not identified and assessed on a timely basis.

Lead Supervisors (LS) significantly rely on the knowledge, quality and timeliness of the work performed by the support groups, such as ORD. OSFI’s overall supervisory process is anchored on OSFI’s Supervisory Framework (SF), which requires supervisory and specialist groups to work closely, integrate their work and leverage their respective expertise, efficiently and effectively. In order to facilitate the effective integration of ORD’s work results into the LS supervisory process, it is important that OSFI’s Supervisory Framework (SF) be consistently applied across OSFI.

The audit revealed that ORD’s staff demonstrated a sound understanding of the operational risks inherent in the business activities of financial institutions; however, ORD’s operational risk assessments were not always consistent.

ORD is committed to using OSFI’s Framework as the overriding methodology driving its work processes and plans. ORD has developed its own specific working tools/criteria on how to assess FRFIs’ operational risks and the quality of the FRFIs’ respective risk management functions. These tools/ criteria demonstrated initiative and appeared to be useful to ORD. However, in the absence of a review of ORD’s guidance/tools by the Common Supervisory Services group (former Practices group) to ensure its alignment with the key principles of OSFI’s SF, there may be inconsistencies in the application of the Framework that may hamper the full integration of ORD’s work into the LS’ supervisory process.

Given increasing industry complexities, more emphasis is needed to ensure Lead Supervisors are provided the full context of the operational risk characteristics and potential supervisory concerns that may impact their respective institutions. Support groups such as ORD need to clearly demonstrate how their efforts fit into the LS supervisory process; otherwise, the two levels of work knowledge (i.e., LS and support groups) may not be fully coordinated and integrated, which may lead to gaps in supervisory coverage and/or ineffective use of OSFI’s supervisory resources.

To the extent Lead Supervisors do not have a base level understanding of operational risk characteristics, operational risks within the FRFIs’ activities may not be appropriately identified. Although ORD is in a position to share its specialized knowledge of operational risk related matters with the Lead Supervisors (who are generalists, not operational risk specialists), ORD’s approach to knowledge sharing on emerging operational risks/issues appeared to be informal.

Recommendation

Given the importance of OSFI’s supervisory process and to facilitate the effective coordination and integration of work efforts and conclusions of support groups, such as ORD, into the LS supervisory work, there needs to be a process in place to ensure that any operational technical criteria/tools issued by specialist groups are properly aligned with OSFI’s Supervisory Framework. This will likely require the attention and support of the recently formed Common Supervisory Services (CSS) group as this matter likely falls outside of ORD’s purview.

Management Action Plan

There is currently no requirement that Divisions in SSG seek approval of any operational technical criteria/tools issued by the support groups. It is the responsibility of the Division that develops the criteria/tools to ensure that they are consistent with the relevant Guidelines and with the Supervisory Framework.

The Common Supervisory Services unit (CSS), specifically through their methodology support, is best positioned to provide oversight and validation of internal assessment criteria developed within supervisory teams to ensure consistency with the Supervisory Framework. CSS priorities include the development of: 1) an approval framework for methodology governance and 2) governance for updates and revisions to the supervisory methodology. At present, CSS is primarily focused on approval framework for methodology governance. The CSS Steering Committee will endorse the approval framework and escalate to the Senior Operating Committee for discussion and consideration by September 30, 2017.

Governance for updates and revisions to the supervisory methodology which includes internal assessment criteria developed within supervisory teams, however, is subject to the methodology governance framework approval. Consideration for the creation of this protocol will occur before December 31, 2017.