EARLY BIRD FROM $1,199* Register by September 20 2ND ANNUAL MODEL USA OCTOBER 7-8, 2019 | CITY

Reviewing the evolution of model risk management and managing innovation with management and oversight

2019 HIGHLIGHTS INCLUDE: REGULATION VALIDATION Regulatory expectations as model risk management Evaluating and looking beyond conceptual continues to evolve across jurisdictions soundness of single use of models

AI & MACHINE LEARNING CECL Increasing efficiency, removing bias in output Early readings and approaches from and limiting reputational risks large vs. small institutions

QUALITATIVE MODELS FUTURE OF MODEL RISK MANAGEMENT Bringing effective validation and governance The future state of model risk management: into the scope of model risk management Evolution of model definition, uses and management

HEAR FROM 20+ MODEL RISK EXPERTS INCLUDING: Stephan Meili Katie Hysenbegasi Emre Balta Paul O’Donovan Snehal Kanakia Managing Director, Managing Director, Head Head of Financial, Market, Director, Model Director, Model Risk Risk Management of Credit Modelling and AML Model Validation Governance Capital One Citi Economic Forecasting U.S. (BMO) groups, Enterprise Capital Adequacy BNY Mellon

Dr. Agus Sudjianto Deniz Senturk Daniel Ward Kash Agrawal Jing Zou Head of Corporate Head of Model Risk Head of RISK IRC, Director, Quantitative Managing Director, Model Risk Management CIB Americas Analytics Model Risk Management State Street BNP Paribas Capital

CO-SPONSOR: ASSOCIATE SPONSOR: CPE ACCREDITATION: BRING THE TEAM 3RD ATTENDEE GETS 50% OFF when registering at the same time

#MRMUSA E: [email protected] T: +1 888 677 7007 www.cefpro.com/model-risk 2ND ANNUAL MODEL RISK MANAGEMENT USA | OCTOBER 7-8, 2019 |

WHY ATTEND:

Model Risk Management over the last few years has seen increased development and innovation. In this new diverse industry, many financial institutions are tasked with the challenge of keeping up with the speed of innovation and regulatory expectations.

The 2nd Annual Model Risk Management agenda is highly focused on these changes to financial institutions and how they are managing changes including regulation, AI & Machine Learning, qualitative models, validation, CECL, the future of model risk management and many more.

To explore these themes further The Center for Financial Professionals will host the 2nd Annual Model Risk Management USA Congress. The two-day Congress will review the latest updates, challenges and emerging technology. The Congress aims to bring together industry professionals and peers to provide a platform for thought -leadership, networking and idea sharing. Join us on October 7-8 in New York City for the main event, and October 9 for a 1-day intensive workshop led by Wells Fargo on Managing Machine Learning Model Risk.

CO-SPONSOR: Andrew Davidson & Co., Inc. (AD&Co) was founded in 1992 by Andy Davidson, an international leader in the development of financial research and analytics, mortgage-backed securities product development, valuation and hedging, housing policy and GSE reform and credit-risk transfer transactions.

Since its inception, the company has provided institutional fixed-income investors and risk managers with high quality models, applications, consulting services, research and thought leadership, aimed at yielding advanced, quantitative solutions to issues. AD&Co’s clients include some of the world’s largest and most successful financial institutions and investment managers. ASSOCIATE SPONSOR: BDO is the brand name for BDO USA, LLP, a U.S. professional services SPONSORSHIP & firm providing assurance, tax, and advisory services to a wide range of publicly traded and privately held companies. For more than 100 years, EXHIBITION BDO has provided quality service through the active involvement of experienced and Advance your branding, awareness, committed professionals. The firm serves clients through more than 60 offices and over 650 industry expertise, thought-leadership independent alliance firm locations nationwide. As an independent Member Firm of BDO and lead-generation at the upcoming International Limited, BDO serves multi-national clients through a global network of more 2nd Annual Model Risk Management than 80,000 people working out of nearly 1,600 offices across 162 countries and territories. USA.

EXHIBITOR: Sponsorship and exhibition with The Center for Financial Professionals offers unique networking, brand recognition and thought-leadership deliverance opportunities with senior risk professionals from around the EARN UP TO 21 CPE CREDITS world. Whether you want full branding Earn up to 14 CPE Credits for the two-day Congress and up across the event or simply a well- to 7 CPE Credits for also attending the Masterclass. positioned exhibition stand, our business Prerequisites: Knowledge of financial risk management development team will tailor the right Advanced Preparation: No advanced preparation is required package for you. We do everything we Program Level: Intermediate to advanced can to help you get your marketing Delivery Method: Group-live message across and also to benchmark the return on your investment. The Center For Financial Professionals is registered with the National Association of State Boards of Accountancy (NASBA) as a sponsor of continuing professional education on the National Registry of CPE Sponsors. State boards of accountancy have final authority on the acceptance of individual courses For more information for CPE credit. Complaints regarding registered sponsors may be submitted to the National Registry of please call us on +1 888 677 7007 CPE Sponsors through its website: www.learningmarket.org / +44 (0) 20 7164 6582 Please note these are subject to change as per the agenda and final credits will be available after the event. or email [email protected]

E: [email protected] T: +1 888 677 7007 #MRMUSA www.cefpro.com/model-risk 2ND ANNUAL MODEL RISK MANAGEMENT USA | OCTOBER 7-8, 2019 | NEW YORK CITY AGENDA DAY ONE | OCTOBER 7

08:00 REGISTRATION AND BREAKFAST 12:20 LUNCH BREAK AND NETWORKING

08:50 CHAIR’S OPENING REMARKS Richard Cooperstein, Director of Model Risk Management, RISK APPETITE 1.20 Setting an institution’s Model Risk Appetite Andrew Davidson & Co., Inc. • Model Risk related Key Risk Indicator identification and creation of Model Risk Index • Monitoring an institution’s Model Risk posture REGULATION - PANEL DISCUSSION • Model Inventory Governance 9:00 Reviewing regulatory expectations as model risk management Director of Enterprise Risk Management, Analytics and continues to evolve and management of requirements across Arindam Majumdar, Reporting, jurisdictions Bank OZK • SR11-7 changes and relation to governance of machine learning • Evolving model risk guidance towards more structured machine learning frameworks • Defining framework for management of machine learning models PERFORMANCE MONITORING – PANEL DISCUSSION 2:00 Developing robust performance monitoring plans and tracking reliability of models over time Wei Ma, Head of Model Risk Management, Sumitomo Mitsui Banking • Documentation and review of conceptual soundness Corporation • Ensuring model is performing to standard in between validations Emre Balta, Head of Financial, Market, AML Model Validation, U.S. Bank • Metrics on a live basis – move to actionable signals to provide a live view Manish Chakrabarti, Head of Model Governance, Americas, BNP Paribas o Tools and technology requirements • Ongoing performance analysis

Toks Adekoya, Director Model Risk Management, CIT AI & MACHINE LEARNING Head of Model Risk Governance, 9:50 Validation and Governance of the use of AI and machine learning Barbora Meunier, Société Générale Julia Litvinova, Head of Model Validation, Managing Director, models to increase efficiency State Street • Determining the most efficient way to validate and control machine Richard Cooperstein, Director, Model Risk Management, Andrew Davidson & learning models Co., Inc. • Handling evolving regulatory expectations • Ensuring oversight from model risk management and building an enhanced governance framework 2:50 AFTERNOON REFRESHMENT BREAK AND NETWORKING • Use of tools that have embedded AI • Bringing model risk management to the next level • Integrating MRM with other risk functions (supplier, technology, ALTERNATIVE METHODOLOGY information security) 3:20 Preparing and implementing alternative methodology adoption • Statistical models, automatic decision, AI and machine learning impact and evaluating conceptual soundness to companies • Engaging machine learning algorithms • Adoption of models in non-traditional areas Paul O’Donovan, Director, Model Governance, Bank of Montreal (BMO) • Expectation from regulators • Moving away from traditional statistics and econometrics • Understanding how to evaluate conceptual soundness 10:30 MORNING REFRESHMENT BREAK AND NETWORKING • Skill set requirements compared to traditional modelers and validators • Moving into a new frontier of modeling • Evaluating and ensuring conceptual soundness of models

11:00 Removing bias in output from AI and machine learning models Juan Salafranca, Head of Retail Credit Risk Models, BBVA Compass and limiting reputational risks • Risk to firm’s reputation and conduct o Managing within validation • Testing and validating for the purpose of fairness and bias RISK SENSITIVITY • Understanding decisions made by algorithms and machines 4:00 Developing risk sensitive model risk management practices • Binary outputs of models tailored to individual model requirements • Unintended consequences of bias in backward data • Cookie cutter approach to model risk management • Validating models intended use to avoid data and output bias • Scheduling validation frequency to determine risk sensitivity • Testing models before implementation to work as intended Chris Smigielski, Model Risk Director, Arvest Bank Lourenco Miranda, MD, Regional Head of Model Risk Management (Americas), Société Générale MATERIALITY THRESHOLD 4:40 Determining materiality thresholds and ensuring dynamic DATA monitoring 11:40 Managing increased emphasis on data and identifying quality • Triggering periodic revaluation issues with increased reliance for technology initiatives • Consistent data from formalization of MRM through SR11-7 • Balancing modeling need and data availability • Determining risk posed to institution and proportionate controls for risk • Integrating data used in modelling with broader data management processes/ • Allocation of effective risk oversight practices • Future adaptations to standard: Idiosyncratic reflection of processes • Understanding upstream data sources and downstream data uses • Increased use of alternate and unstructured data Petr Chovanec, Director, Business Modeling and Forecasting, UBS • Managing data used for new modelling technologies (such as AI/ML)

David Palmer, Senior Supervisory Financial Analyst, Board 5:20 CHAIR’S CLOSING REMARKS

5:30 END OF DAY ONE AND DRINKS RECEPTION

*PLEASE NOTE THIS AGENDA IS SUBJECT TO CHANGE*

E: [email protected] T: +1 888 677 7007 #MRMUSA www.cefpro.com/model-risk 2ND ANNUAL MODEL RISK MANAGEMENT USA | OCTOBER 7-8, 2019 | NEW YORK CITY AGENDA DAY TWO | OCTOBER 8

08:15 REGISTRATION AND BREAKFAST CECL – PANEL DISCUSSION 1:50 Getting ready for CECL live: modeling, implementation and impact 08:50 CHAIR’S OPENING REMARKS • Use of qualitative overlays and judgements Sudip Chatterjee, Managing Director, BDO o Impact on the reserved • Use of stress test models o Implementing one model to do both • Building a model that captures macro-economic variables and loss QUALITATIVE MODELS - INTRODUCTION • Approaches from large vs. small institutions 9:00 The what and why to qualitative models • Early readings of reserve impacts • Views on volatility evolution Ximena Zambrano, Head of Qualitative Model Validation, Wells Fargo • Impact on profitability on product offerings • Compensating controls and use of conservatism • Reasonable and supportable forecast period QUALITATIVE MODELS - EXTENDED INTRODUCTION AND PANEL DISCUSSION Katie Hysenbeasi, Managing Director, Head of Credit Modelling and Economic 9:20 Effective validation and governance of qualitative models and Forecasting groups, Enterprise Capital Adequacy, BNY Mellon adding them to the scope of model risk management • Expanded model definition from regulators • Managing assumptions and standard testing requirements 2:40 AFTERNOON REFRESHMENT BREAK AND NETWORKING • Tiering qualitative models • Scale required to bring in qualitative models • Reviewing, validating and monitoring complete inventory of qualitative models • Regulatory feedback on development and validation techniques QUANTIFYING MODEL RISK 3:10 Quantification of model risk: Defining and identifying model risk Ximena Zambrano, Head of Qualitative Model Validation, Wells Fargo limitations in performance Deniz Senturk, Head of Model Risk Management, State Street • Aggregate basis individually and across model risk types Kash Agrawal, Director, Quantitative Analytics, Barclays Capital • Regulatory framework for continuous model monitoring Chris Smigielski, Model Risk Director, Arvest Bank • Defining and monitoring metrics • Quantifying model risk related to a breach of thresholds • Definition of model risk appetite statement • Decisions on usage and redevelopment of models 10:20 MORNING REFRESHMENT BREAKS AND NETWORKING Jing Zou, Managing Director, Model Risk Management, Royal Bank of Canada

VALIDATION 10:50 Evaluating model use and ensuring validation looks beyond FUTURE OF MODEL RISK MANAGEMENT conceptual soundness of single use • Validating for specific uses of models used for multiple purposes - PANEL DISCUSSION • Performance and monitoring across use 3:50 The future state of model risk management: Evolution of model • Making model risk management a value adding partner definition, uses and management • Strategic planning for model risk management to fully plug into enterprise risk • Organisational design structure • Comprehensive monitoring to understand model uses • Model risk as a true risk management function • Aligning model development and user’s terminology • Migration to a holistic risk management and advisory function • New techniques of the future • o Investment levels to get there Head of Model Risk Management, Albert Chin, Signature Bank • Adapting frameworks for the future of model risk management • Monetizing model risk and adding value

CHANGE CONTROL Daniel Ward, Head of RISK IRC, CIB Americas, BNP Paribas 11:30 Managing change control to ensure visibility and tracking Stephan Meili, Manging Director, Risk Management, Citi across models • Restricting access to model output to users • Managing models in a controlled IT system 4:40 CHAIR’S CLOSING REMARKS • Independent validation unit visibility to changes made • Tracking changes made to model • Identifying full impact on a business • Regulatory expectation for access and change control 4:50 END OF CONGRESS o Tracking exposure to an individual model *PLEASE NOTE THIS AGENDA IS SUBJECT TO CHANGE*

Snehal Kanakia, Director, Model Risk, Capital One

12:10 LUNCH BREAK AND NETWORKING BRING THE TEAM 3RD ATTENDEE GETS 50% OFF STANDARDIZATION 1:10 Developing an enterprise standardization of development and when registering at the same time documentation standards across model population • Model teams spread out – ensuring all are following the same standards and templates • Getting all models to one standard 50% o Varying maturity of teams • Global regulatory agendas and varying expectations 1 2 OFF • Standardize across regions or bespoke across jurisdictions

E: [email protected] T: +1 888 677 7007 #MRMUSA www.cefpro.com/model-risk 2ND ANNUAL MODEL RISK MANAGEMENT USA | OCTOBER 7-8, 2019 | NEW YORK CITY

POST-EVENT MASTERCLASS | OCTOBER 9 Jie Chen Jie ChenHarsh is SinghalManaging DirectorHarsh Singhal in isthe Head of Decision Science and Artificial Agus Sudjianto Agus Sudjianto is an executive vice president and headAdvanced of Corporate Technologies for ModelingIntelligence (AToM) Validation within the Model Risk group. Model Risk forManaging Wells Fargo, Director where he is responsible for enterpriseManaging Director His team is responsible for validating and approving all Executive Vice President MANAGINGmodel risk management.Head of StatisticsMACHINE and Machine LEARNINGGroup Headof of CorporateDecision Science MODEL andModel Risk RISKat Wells Artificial Intelligence Validation retail Credit Decision, Commercial Credit Rating, Head of Corporate Model Risk Learning Fargo. WellsShe Fargo is& Companyleading the FinancialStatistics Crimes and & Fair Lending including Fraud and Wells Fargo & Company Corporate Model Risk Prior to his current position, Agus was the modelingLED andMachineBY: analytics Learning team, BSA/AML,focusing Operationson Risk, Marketing, and other director and chief model risk officer at in the artificial intelligence/machine learning models. United Kingdom. Before joining Lloyds, he was a seniordevelopment credit risk of cutting-edge models, executive and head of Quantitative Risk at .algorithms, and a computingPrior platform to his current to position, Harsh was responsible for advance the Bank’s practice innew the model areas development of for Wholesale Risk in Bank of Prior to his career in banking, he was a product design manager in America. Harsh also led the Retail IRB model the Powertrain Division of Ford Motor Company. credit, operational, and market risk qualification at Bank of America and contributed management. She has over ten year towards the development of first generation of deposit Agus holds several U.S. patents in both finance and engineering. He experience on machine learning,balance modelsartificial for Asset -Liability management. has published numerous technical papers and is a co-author of Design and Modeling for Computer Experiments. His intelligencetechnical and advanced statistics in the Prior to joining the financial industry, he worked on Dr. Agus Sudjianto expertise and interestsDr. includeJie Chen quantitative risk, particularlybanking creditDr. industry. Harsh Singhal Dr. Bernhard Hientzsch risk modeling, machine learning and computational statistics. quantitative modeling for pharmaceutical and telecom Head of Corporate Head of Statistics and Head of Decision industries.Head He of is Model, passionate Library, about developing Model Risk He holds mastersMachine and doctorate Learning degrees in engineeringJie and Science holds a andPh.D. Artificial in Statistics quantitative fromand Toolstalentthe Developmentand fostering a culture of responsibility and intellectual curiosity. His technical Wells Fargo management fromWellsWayne State Fargo University and the MassachusettsStewartIntelligenceSchool Validationof Industrial andWells Fargo Institute of Technology. Systems Engineering at the Georgexpertiseia Institute includes machine-learning, multivariate Wells Fargo analysis, commercial and consumer credit. of Technology, and a bachelor’s degree in Computational Mathematics Hefrom has Master’sNanjing degree in Electrical Engineering and a Ph.D. in Statistics. The adoption and ubiquity of machine learning in FinancialUniversity. Institutions pose a set of new model risks This workshop is intended to provide practitioners in model development, validation, and governance to take advantage the power of machine learning and to understand and manage their model risks. The following topics will be covered in the workshop:

Introduction Machine learning applications in banking Model risk in machine learning Explainable and trusted machine learning

wellsfargo.com wellsfargo.com Introduction to machine learning© 201 methodology8 Wells Fargo Bank, N.A. All rights reserved. © 2018 Wells Fargo Bank, N.A. All rights reserved. wellsfargo.com Supervised Learning:© 2018 Wells Random Fargo Bank, N.A. Forest, All rights reserved. Gradient Boosting Machine, and Neural Networks Unsupervised Learning and Reinforcement Learning Transfer Learning and Generative Adversarial Network

Examples of advanced machine learning applications in banking Credit Modeling and Stress Testing Deritative pricing and CVA using deep learning Value at Risk and Multivariate Time-Series Modeling using Generative Adversarial Network

Explainability and interpretation techniques for machine learning Diagnostic techniques Variable/feature importance: local (LIME and SHAP) and global importance (SHAP, Sobol, Permutation Test, ANOVA) Effects of Input/Outputs (PDP, ICE and Derivative-based approach) Model distillation (Tree, KLIME. LIME-SUP) Structured-Interpretable models: Explainable Neural Networks

Validation Framework Model suitability and conceptual soundness: model bias and explainability Model robustness and stability Model implementation and safety: cautious generalization and fail-safe mode Change control: model retraining and monitoring Managing vendor models

E: [email protected] T: +1 888 677 7007 #MRMUSA www.cefpro.com/model-risk 2ND ANNUAL MODEL RISK MANAGEMENT USA | OCTOBER 7-8, 2019 | NEW YORK CITY

2019 MODEL RISK MANAGEMENT SPEAKERS

Toks Adekoya Snehal Kanakia David Palmer Director Model Director, Model Risk Senior Supervisory Risk Management Capital One Financial Analyst CIT Federal Reserve Board

Kash Agrawal Nikolai Kukharkin Juan Salafranca Director, Quantitative Head of Model Head of Retail Credit Risk Analytics Risk Management Models Barclays Capital TIAA BBVA Compass

Emre Balta Julia Litvinova Deniz Senturk Head of Financial, Market, Head of Model Validation, Head of Model Risk AML Model Validation Managing Director Management U.S. Bank State Street State Street

Manish Chakrabarti Wei Ma Chris Smigielski Head of Model Head of Model Risk Model Risk Director Governance, Americas Management Arvest Bank BNP Paribas Sumitomo Mitsui Banking Corporation

Sudip Chatterjee Arindam Majumdar Daniel Ward Managing Director Director of Enterprise Risk Head of RISK IRC, Management, Analytics and CIB Americas BDO Reporting BNP Paribas Bank OZK

Albert Chin Stephan Meili Ximena Zambrano Head of Model Risk Manging Director, Head of Qualitative Model Management Risk Management Validation Signature Bank Citi Wells Fargo

Petr Chovanec Barbora Meunier Jing Zou Director, Business Head of Model Managing Director, Model Modeling and Forecasting Risk Governance Risk Management UBS Société Générale Royal Bank of Canada

Richard Cooperstein Lourenco Miranda Director, Model MD, Regional Head of Risk Management Model Risk Management BRING THE TEAM Andrew Davidson (Americas) 3RD ATTENDEE GETS & Co., Inc. Société Générale

Katie Hysenbeasi Managing Director, Head 50% OFF of Credit Modelling and Paul O’Donovan when registering at the same time Economic Forecasting Director, Model Governance Bank of Montreal (BMO) groups, Enterprise Capital 50% Adequacy 1 2 OFF BNY Mellon

E: [email protected] T: +1 888 677 7007 #MRMUSA www.cefpro.com/model-risk REGISTRATION FORM PLEASE REGISTER THE FOLLOWING DELEGATE(S) DELEGATE 1: Miss Ms Mrs Mr Dr Other 2ND ANNUAL MODEL RISK Name MANAGEMENT USA Position VENUE: MILLENNIUM TIMES Organization SQUARE, 145 WEST 44TH STREET, Address NEW YORK, NY 10036, USA OTHER METHODS TO REGISTER Country Zip/Postal Code By telephone: +1 888 677 7007 Telephone Fax Online: www.cefpro.com/model-risk Email: [email protected] E-mail DELEGATE 2: Miss Ms Mrs Mr Dr Other DELEGATE 3: Miss Ms Mrs Mr Dr Other Name Name Position Position Telephone Telephone E-mail E-mail DELEGATE 4: Miss Ms Mrs Mr Dr Other DELEGATE 5: Miss Ms Mrs Mr Dr Other Name Name Position Position Telephone Telephone E-mail E-mail

Please keep attendees updated on our Please keep attendees updated on our latest I agree to 3rd parties associated with this I agree to my details being passed to leading financial risk and regulation Risk Reports and Risk Insights (Articles, initiative receiving my contact details for relevant relevant third parties for offers and conferences and webinars. Newsletter, Quarterly Magazine and TV). knowledge sharing and thought leadership subscriptions SELECT EVENT OPTIONS

Representing a financial institution EARLY BIRD STANDARD RATE (E.g. Bank, company, Asset Manager, Regulator) Register by September 20 Registrations after September 20

$1,199 $1,499 MAIN EVENT ONLY (SAVE $300)

$699 $899 MASTERCLASS ONLY (SAVE $200)

$1,898 $2,398 MAIN EVENT & MASTERCLASS (SAVE $500)

Representing an information/service provider – (E.g. Consultant, Vendor, Executive Search Firm, Law Firm)

$1,799 $2,099 MAIN EVENT ONLY (SAVE $300)

$699 $899 MASTERCLASS ONLY (SAVE $200)

$2,498 $2,998 MAIN EVENT & MASTERCLASS (SAVE $500)

I am unable to attend the 2019 Congress, please send me the course documentation for $499 (All registered delegates will receive documentation)

DISCOUNT CODE: PLEASE READ PAYMENT OPTIONS In all cases payment prior to the event is required. Fee includes attendance at sessions, refreshments Invoice: Please send me an invoice with wire transfer information. and course documentation of registered event. Confirmation: you will receive confirmation and joining instructions from us within two working days of . Please debit my: VISA MASTERCARD AmEx. registering. If this is not the case, please telephone us to ensure we have received your booking. Card No: / / / Expiry Date: / Please note that credit cards will be debited within 7 days of your registration. Payment must be made within two weeks to ensure that discounted rate is CVV/Security number: retained. To ensure access to the event, payment must be made prior to the event. Name of Card Holder By proceeding and registering you Billing Address agree to our Terms of Services found at www.cefpro.com/terms-and-conditions. We take your privacy seriously and our data is held securely and held in compliance to EU and US data laws. If Country Zip/Postal Code you wish to unsubscribe to our communications you can do so at any time using the links provided in all Signature: of our emails. Please review our Privacy Policy at www.cefpro.com/privacy-policy for more information and if you have any queries please contact Please click submit to complete. If a draft email window does not open in [email protected]. your email client, please forward this completed form to [email protected]