Model Risk Management
Total Page:16
File Type:pdf, Size:1020Kb
Model Risk Management Nav Vaidhyanathan Director, Head of Model Risk Management Wintrust Financial Corporation Global Association of Risk Professionals August 2014 These materials reflect the views and opinions of the speaker and are not intended to reflect the views, policies or practices of Wintrust Financial or its affiliated companies. They are for informational purposes only. The views expressed in the following material are the author’s and do not necessarily represent the views of the Global Association of Risk Professionals (GARP), its Membership or its Management. 2 Agenda Model Risk Management is a broad, evolving, and an expanding topic. This presentation will cover only select components at a high level. Inherent risk (model tier) and residual risk of model . A framework for model monitoring . Model risk aggregation framework . Model validation for stress test (e.g., DFAST) models .Global Association of Risk Professionals. All rights reserved 2014 © | 3 Model Risk Axioms* The following can potentially be considered as self-evident truths for Model Risk. Given everything else equal, . All models have limitations . Model Risk is related to model limitations . Model Risk increases with more models (this might fall in the grey area) − E.g., two Tier 1 models potentially pose higher risk than 1 Tier 1 model (everything else being the same, i.e., similar limitations, similar validation status, similar materiality, similar findings, etc.) . Model Risk for Tier 1 model > Model Risk for Tier 2 model . Model Risk for validated model ≤ Model Risk for unvalidated model (for the same model) . More findings ⇒ more Model Risk (same category and level of findings) . Certain types of findings will have higher Model Risk than others . Models with deteriorating performance pose higher Model Risk . Model Risk is exhibited by the variance between model output and observed actual (this is just one of the ways that Model Risk exhibits itself) * may not be axioms in the purest sense of the term .Global Association of Risk Professionals. All rights reserved 2014 © | 4 Inherent Risk and Residual Risk of Model Residual Risk Rating could be High driven by: Rating • Model validation findings: Moderate • Category Residual Low • Level Risk • # of Open Findings Low (Tier 3) Moderate High (Tier 1) • Model performance / Inherent Risk Rating (Model Tier) monitoring Inherent Risk Rating could be driven by: Model owners can reduce residual • Materiality risk – by improving models, • Type of model improving controls, and/or • Used for regulatory / financial reporting or critical business remediating findings decisions • Feeds into another Tier 1 model? Model owners cannot (usually) reduce inherent risk .Global Association of Risk Professionals. All rights reserved 2014 © | 5 Residual Risk Rating Framework . Residual Risk Rating (High, Moderate, Low) is based on the Residual Risk Score range . E.g., RRS > 200 ⇒ High, RRS ≤ 100 ⇒ Low . If the model is deemed unfit for use, the RRR is High . The Residual Risk Score Range to Residual Risk Ratings mapping needs to be monitored periodically . The ranges can be determined by typical observations from model validation and model monitoring exercises . If a model is unvalidated, the residual risk rating is HIGH .Global Association of Risk Professionals. All rights reserved 2014 © | 6 Model Monitoring . Model monitoring should evaluate whether changes (including anticipated) in products, exposures, activities, clients, or market conditions necessitate adjustment, redevelopment, or replacement of the model . Ongoing activities include – review of input data and assumptions, outcome analysis, model stability checks, review of overlays and expert judgments, benchmarking, etc. Key considerations − Model performance monitoring › Pros and cons of model performance measures › Model type › Model use › Type of measure − Periodicity of monitoring − Thresholds for model recalibration or rebuild − From performance measure to model performance rating .Global Association of Risk Professionals. All rights reserved 2014 © | 7 A Bottom Up Framework For Model Risk Aggregation Impact of Failure (e.g., on Capital) Tier 1 Tier 2 Tier 3 Validated … … Yes No High Residual Moderate Low Residual As part of an RCSA Risk Residual Risk Risk process, the impact of the model failure is estimated by model owners The end nodes in the For DFAST projection models, for Severely Adverse Scenario, the impact on capital can be assessed decision tree can be using backtesting during Great Recession period for all models (adjust the variance by severity of the used for aggregating scenario compared to the Great Recession) model risk By looking at the distribution, a conservative starting point for capital can be chosen for each model Tier .Global Association of Risk Professionals. All rights reserved 2014 © | 8 Some Considerations in Validation of Stress Test Models . Are estimates conditioned on macroeconomic scenarios? . Do the models produce outcomes that show separation and order between the baseline, adverse, and severely adverse scenarios? . Are the scenario and assumptions consistent across all models? . Are exposures modeled using appropriate levels of segmentation? . Qualitative elements of the model . Relevance of third party vendor models to company specific characteristics, especially when using proxy portfolio . Sensitivity analysis around assumptions . Use of weak models vs. expert judgment . For models not validated, compensating controls and conservativeness . Documentation . Capital buffer for model risk .Global Association of Risk Professionals. All rights reserved 2014 © | 9 Creating a culture of risk awareness ® Global Association of Risk Professionals 111 Town Square Place 14th Floor Jersey City, New Jersey 07310 U.S.A. + 1 201.719.7210 2nd Floor Bengal Wing 9A Devonshire Square London, EC2M 4YN U.K. + 44 (0) 20 7397 9630 www.garp.org About GARP | The Global Association of Risk Professionals (GARP) is a not-for-profit global membership organization dedicated to preparing professionals and organizations to make better informed risk decisions. Membership represents over 150,000 risk management practitioners and researchers from banks, investment management firms, government agencies, academic institutions, and corporations from more than 195 countries and territories. GARP administers the Financial Risk Manager (FRM®) and the Energy Risk Professional (ERP®) Exams; certifications recognized by risk professionals worldwide. GARP also helps advance the role of risk management via comprehensive professional education and training for professionals of all levels. www.garp.org. .Global Association of Risk Professionals. All rights reserved 2014 © | 10 Model Risk Management for Non Banks Michelle McCarthy Managing Director, Risk Management Nuveen Investments Global Association of Risk Professionals August 2014 Disclaimers These materials reflect the views and opinions of the speaker and are not intended to reflect the views, policies or practices of Nuveen Investments or its affiliated companies. They are not intended to provide investment advice and are for informational purposes only. The views expressed in the following material are the author’s and do not necessarily represent the views of the Global Association of Risk Professionals (GARP), its Membership or its Management. 2 Contents . Differences between models and their uses at banks vs. the buy side . What should be in scope for model risk management activity in the asset management business? . Potential tiering of asset management models, and effect on model risk management activities .Global Association of Risk Professionals. All rights reserved 2014 © | 3 Most literature on model risk management has concerned banks . OCC 2000-16 . OCC 2011-12, FRB SR 11-7 . The case of AXA Rosenberg brought the conversation to the buy side, without much guidance on how bank practices can and should differ from asset managers’ practices .Global Association of Risk Professionals. All rights reserved 2014 © | 4 Different uses of models in asset managers vs. banks . Asset managers are not computing regulatory capital Bank-owned asset managers may need to provide operational risk capital calculations, but this is not in service of the asset management business . We rarely use valuation models as the sole source of valuation; we frequently use external sources . Many asset management models are an input to a decision, but not the sole driver of that decision .Global Association of Risk Professionals. All rights reserved 2014 © | 5 Potential scope of model management activity for asset managers . Includes models that are used for investing or hedging Drive investing activity to some extent Value key assets . Does not include calculators; model must provide estimates and involve uncertainty to require model risk management Good change controls always an important operational control, but calibrations and validations should be reserved for models that estimate uncertainty Would not include compliance systems Would not include price-to-yield calculations Would not include simple screens for fundamental stock characteristics Would not include pure index replication models without meaningful uncertainty . Does not include analyses that help develop an investment thesis—only the tools used to execute the thesis, or to value assets once purchased .Global Association of Risk Professionals. All rights reserved 2014 © | 6 6 Tiering for materiality . A potential tiering of asset management models: