Spring 2009

A quarterly newsletter from the National Institute of Statistical Sciences NISS and OCC Explorations Workshop: Financial Modeling and Banking Regulations The room was packed at the risk management in the banking identified the central issues NISS-OCC Explorations Workshop, context. He noted that as risk from a viewpoint. “Exploring Statistical Issues in modeling and management is Michael Kalkbrenner, head of the Financial Risk Modeling and now rapidly evolving, focus on modeling team within Banking Regulation,” held on the fundamentals is critical at the Risk Analytics and Instruments February 5-6 2009 at the Office of this juncture to provide the the Comptroller of the Currency necessary underpinning to (OCC) in Washington DC. The approach issues including risk OCC, a NISS-SAMSI affiliate, co- aggregation, interdependence sponsored the workshop. and concentration of , In response to the success of and propagation of impact of this workshop, a second workshop extreme events analytically is scheduled for 20-21 October and with an eye to policy 2009, also to be held at the OCC development. headquarters in Washington, DC. Speakers from the banking Registration will be available on the and regulatory community NISS website and pre-registration addressed both financial Paul Embrechts, Professor of Mathematics, ETH, is required as space is limited. risk and from Zurich (left) and Alan Karr, director of NISS (right) The keynote speaker, the regulatory, banking and chat during a break at the NISS-OCC Explorations Paul Embrechts, professor of monitoring perspectives. From Workshop. Mathematics, ETH, Zurich, the regulatory community, Emré department of Deutsche Bank delivered an overview of the Balta, senior financial economist outlined the internal risk modeling roles for statistics in quantitative in the Analysis and prediction process; Douglas Division at OCC and Patrick Dwyer, managing director of de Fontnouvelle, VP of the Moody’s KMV, articulated the Quantitative Analysis Unit challenges to forecasting and of the Federal Reserve Bank indexing from a ratings agency’s of Boston outlined crucial perspective. issues for operational risk; Academic considerations of Dennis Glennon, director these same issues were presented of the Credit Risk Analysis by David J. Hand, Chair and Division at OCC, and professor of Statistics at Imperial Min Qi, senior financial College, London, Katherine Ensor, economist also of the Credit professor and chair of Statistics at Everyone listened attentively to the presentations regarding Risk Analysis Division development of new risk models for the banking industry. Rice University, Nicholas Kiefer, (Continued on page 3) 1 A Note from the Director

The halls—both old and teams, each of which also includes new—of NISS are humming a graduate student and one of as we prepare for a very busy our new postdocs. A new TRB- summer. We have interviewed funded project on uncertainty several outstanding postdoctoral in travel time prediction and an candidates for the pleasingly NSF/DTRA-funded project on large number of positions we syndromic surveillance also will have available. To date, four offers start this summer. Finally, our have been accepted, and we shared postdoctoral fellowship look forward to completing the with the Hamner Institutes for recruiting Health Research, an RTP neighbor process of NISS, has been revived; Jessie soon. Xia is the current holder of this Starting fellowship. this Thanks to the efforts and summer, energy of communications director we will Jamie Nunnelly (among them have eight this newsletter!), we are nearly post- ready to debut our re-designed doctoral web site. And we now have, as the fellows, the result of a contest that drew more most in the than 20 entrants, a NISS tagline: history of The Statistics Community Serving NISS! You the Nation. No tagline can exist can read without a T-shirt, and we have more about our new hires in the a new one, pictured right. Over summer issue of NISS Parameters, time, it will be distributed to many and can meet current postdoc friends of NISS. Jessie Xia elsewhere in this edition. New postdocs mean new research activity. One major new project is underway—a very Alan Karr complex study with National Director Cancer Institute on the Clinical Proteomic Technology Assessment for Cancer (CPTAC). This summer, the NISS/NASS research in residence program begins, and three teams of researchers who NISS Picks a Tagline be working at NISS. Faculty NISS held a tagline contest Serving the Nation. members Balgobin Nandram of this past quarter. While Thanks to everyone who Worcester Polytechnic Institute, there were several very good helped come up with ideas. Scott Holan of the University of suggestions, none of the entries You will each get a t-shirt for Missouri, Linda Young of the seemed to capture just what contributing! University of Florida and Pamela NISS does. However, the Arroway, Barry Goodwin and Sujit suggestions helped lead the Ghosh of North Carolina State staff to the phrase that was University will lead the research developed. Our new tagline is: The Statistics Community

2 Research Profile:

NISS-NASS

NISS and NASS Collaborate NISS-OCC Workshop on Cross-Sector Research (Continued from page 1) NISS and the National opportunity to tackle some of the professor of Economics at Cornell Agricultural Statistics Service hard problems at USDA and NASS, University (and senior advisor (NASS), the survey and estimation specifically,” said Nell Sedransk, to the Enterprise Risk Analysis arm of the U.S. Department of associate director of NISS. Division at OCC), Don McLeish, Agriculture, have formed a program Some of the areas of research professor of Statistics and Actuarial called the Cross-Sector Research in that the team or teams will Science, University of Waterloo and Residence Program. The program work on include Multivariate John Liechty, associate professor consists of various teams, each Imputation Mechanisms and Valid of Statistics at Pennsylvania State comprised of a faculty researcher in Mean Squared Error Estimation: University. Agricultural Resource Management The first day was spent on the Survey – Phase exposition of issues from the three III; New Design different vantage points. Discussion and Estimation repeatedly returned to the problems Methodologies for of quantification of risk and of model Biased Self-Exclusion validation and model optimization, (under-coverage): and consequently the specific Estimation of Small requirements for data at each level Farms from Census of detail. From the risk modeling at Mail List; New the level of individual banks, these Statistical Editing and issues quickly expanded to questions Imputation Methods for the larger community of banks, that Preserve Data with all the attendant problems of statistics, a NASS researcher, a NISS Quality: Quarterly Agricultural interdependence. mentor, a postdoctoral fellow and Survey; and Statistical Multi- The second day, the participants a graduate student, who will work Source Predictive Models and divided into six working groups, each together at NISS during the two Error Estimates: Major USDA Crop to review one of the principal topics consecutive summers. The NASS Protection Forecasts and Estimates. that emerged and to develop outlines researcher and the postdoctoral NISS will host a workshop or initial drafts of white papers. fellow will work together during the June 1-2 to kick off the collaboration These white papers are intended to academic year in Washington DC. of the two organizations. The connect both fundamental statistical “NISS is particularly excited meeting will kick off this new concepts and formalizations useful about the NISS-NASS program collaboration. NASS will review the in developing methodology with because it gives statistical researcher latest methods in editing surveys the banking and/or regulatory risk at different stages of their that other organizations are using modeling objectives. careers, from graduate students and will highlight NASS’ developing The presentations are available to postdoctoral fellows to senior plans to improve its system. at http://www.niss.org/affiliates/ academic statisticians, an ideal financialrisk200902/financial_risk_ 3 home.html. Postdoc Profile: Jessie (Qing) Xia

Jessie (Qing) Xia is one of a collaborative network NISS’ more seasoned postdoctoral of five CPTAC teams to fellows. Jessie grew up in Nantong, address this critical need. also known as “The pearl of The goal is to enable all the Yangtze river.” She got her researchers conducting Bachelor’s degree from Tsinghua cancer-related protein University in Beijing and later research at different worked for a company there. laboratories to effectively Then she decided to come to use proteomic technologies the United States to pursue her and methodologies to advanced degrees. She attended directly compare and Duke University, and received both analyze their work. This her Master’s degree in statistics should lead in turn to and her Ph.D. in biomedical improved diagnostics, Jessie (Qing) Xia in her office at NISS engineering at Duke. Jessie joined therapies and even prevention NISS in 2007. of cancer. NISS was brought into University School of Medicine. Jessie is one of the people at the project to provide statistical NCI also has five working NISS who are working with the expertise and guidance on the groups underway including: National Cancer Institute on the project and to help coordinate the bioinformatics, digestion, Clinical Proteomic Technology efforts of the five centers. verification, protein standards and Assessment for Cancer (CPTAC). The five centers currently unbiased discovery. Current cancer proteomic involved in this research “We are glad we are actively research is hampered by a lack include: Broad Institute of contributing to NCI’s cancer of standardized technologies and MIT and Harvard, Memorial research through this important methodologies, which are critically Sloan-Kettering Cancer Center, project. For example, one group we needed in order to more effectively Purdue University, University of are working with is the digestion discover and validate proteins California, San Francisco/Lawrence working group. They are interested and peptides relevant to cancer, Berkeley National Laboratory/ in comparing the digestion or “biomarkers’. NCI established Buck Institute, and the Vanderbilt (Continued on page 6) Calendar of Events NISS-NASS Cooperative Research Conference June 1-2, 2009 at NISS in Research Triangle Park, NC. ARA ELIGIBLE.

ITSEW 2009: The Concept of Total Survey Error—Uses and Abuses June 14-17, 2009 in Tällberg, Sweden. ARA ELIGIBLE.

NISS-OCC Explorations Workshop October 20-21, 2009 in Washington, DC. ARA ELIGIBLE.

4 Affiliate Profile: Office of the Comptroller of the Currency

The Office of the Comptroller quantitative modeling expertise. as reflected in the variations in of the Currency (OCC) Economics This is generally done by the primary focus of the four divisions Department joined the NISS/ Risk Analysis Divisions (the within the RAD group: SAMSI affiliates program in RAD group). The economists, Compliance RAD specializes April 2008. The OCC Economics statisticians, and mathematicians in using statistical methods to Department delivers economic of the RAD group generally also model a bank’s compliance with and quantitative analysis to policy conduct their own independent applicable laws, primarily in the makers and bank supervisors research, for presentation area of fair lending. at the OCC. This encompasses at professional meetings or Credit RAD specializes in an active program of economic publication in journals, research models and methods used for research and analysis, using credit scoring, credit ratings, traditional tools of statistics and and credit risk-measurement in econometrics to bring data to bear banking. on policy issues. Enterprise RAD specializes The OCC was established in economic capital modeling, in 1863 as a bureau of the U.S. enterprise-wide stress testing, Department of the Treasury to operational risk modeling, and charter, regulate and supervise methods banks use to allocate all national banks, which hold 70 capital internally. percent of all commercial banking Market RAD specializes in assets. The OCC also supervises the risk-measurement and asset- the federal branches and agencies pricing models that banks use of foreign banks. Headquartered for over-the-counter derivative in Washington, D.C., the OCC’s dealing, portfolio management, nationwide staff conducts on-site asset-liability management, reviews of national banks and mortgage banking, and asset provides ongoing supervision of . bank operations. The agency also Statistical analysis has issues rules, legal interpretations, become increasingly important in and corporate decisions concerning banking and in bank regulation, banking, bank investments, and the department has evolved bank community development Emré Balta, Senior Financial Economist at OCC, to reflect that shift. The RAD activities and other aspects of bank speaks at the NISS-OCC Explorations Workshop group in particular has grown operations. The OCC is headed by in February. significantly and become more the Comptroller of the Currency, that often reflects the modeling central to bank supervision and who is appointed by the President, issues they confront in practice in regulation, due to the growing use with Senate confirmation, for a the banks. They also frequently of quantitative models by banks five-year term. design and deliver training on for risk management and decision The Economics Department quantitative topics for OCC making. Although the primary provides expert advice to OCC examiners. responsibility for assessing the bank examiners, sometimes The types of statistical work suitability of these models rests from the office but often out in conducted by staff in RAD, and with each individual bank as the field, to assess the quality the problems they confront in part of their validation processes, of banks’ risk measurement practice, vary depending on the (Continued on page 6) methods and generally provide specific needs of bank supervisors, 5 Jessie Xia Profile Continued (Continued from page 4) OCC Profile Continued protocols to see for certain what an (Continued from page 5) advantage or disadvantage might bank supervisors increasingly Basel II capital framework. be of using a certain protocol. are required to understand the Many of the techniques under One limiting factor is the machine technical details of these models consideration by banks are too time. It takes a lot of time for and, when warranted, recommend new or too specialized to have each protocol to run on the mass adjustments or enhancements been studied within the traditional spectrometry. Each specimen takes to model designs as part of the academic research framework. about one and a half hours to go supervisory process. The RAD Affiliating with NISS provides through the machine, plus some staff provides the needed expertise the opportunity to further OCC’s extra time for a blank washout run within the OCC through direct ability to review these models in-between. So, there is a limitation participation in examinations, and provide expert opinions, and on how many replicates we can do. construction of models and tools to facilitate cooperative research Once we optimize the design to for use by examiners, consultation across multiple disciplines and to that they can achieve their primary with examiners and policymakers, communicate the challenges faced study goals,” notes Jessie. educational outreach and training by practitioners to the academic NISS is also helping some of examiners, and preparation research community. other working groups with of written materials for use by The staff in the RAD group experimental designs. NISS will examiners and policymakers. and in other parts of the OCC assist in creating standards that OCC and NISS held an Economics Department have active will be used across the various Explorations Workshop, research programs of their own. labs and institutions to ensure “Exploring Statistical Issues in Discussing ideas and interacting that consistent replications can be Financial Risk Modeling and with other scholars through achieved. Banking Regulations,” at the OCC NISS helps foster new ideas and Jessie enjoys working on a headquarters in Washington DC additional research that ultimately variety of projects. “I have learned in February 2009. As Professor brings great value to the OCC. a lot since I started working here,” Paul Embrechts pointed out at the As a quantitative department she says, “Not only do I get to workshop, statistical quantities heavily engaged in statistical use my expertise in statistics and are “hardwired” into banking modeling, the mission of NISS to biomedical informatics, I also regulations. The Basel II Accord, identify, catalyze and foster high- understand more of the area of which formulates international impact cross-disciplinary research work each project is about.” In risk-based capital adequacy involving statistical sciences addition to the CPTAC project, standards for financial institutions, made affiliation highly attractive. Jessie played a leading role in QT specifies the use of an extreme The February workshop was a interval studies in collaboration quantile (known as the Value- perfect illustration of the value the with Eli Lilly and Company, high- at-Risk) as the measure of risk; OCC hopes to derive from NISS dimensional microarray data this makes minimum capital affiliation. NISS and OCC will hold analysis of fetal mouse gonad requirements one of the few areas another Explorations Workshop development and is currently of law with a significant and October 20-21. collaborating with The Hamner explicit reliance on statistics. Basel Institutes for Health Sciences II is not the only instance of this; on a toxicology project to aid in for example, statistical tests of toxicity testing of environmental significance are a key component chemicals. of the enforcement of laws and Jessie presented her QT work at regulations related to fair lending. the First International Symposium Advances in risk-modeling on Biopharmaceutical Statistics techniques are developing quickly held in Shanghai last year. This in the financial industry for many year she will present some of her reasons, including reliance on recent work at the First ToxCast risk-based pricing, the need to Data Analysis Summit in May, and value portfolios of securitized the 2009 Joint Statistical Meetings loans, and the inclusion of risk in August. Meanwhile, she is measurement tools under the writing several papers. 6 Young Asserts Earlier Study on Cereal Consumption is Flawed

Stan Young, assistant While Young et al. didn’t get director of Bioinformatics for the 50,000 Google hits that the NISS, published a paper in the original study received; the paper Proceedings of the Royal Society did generate a good amount of B this winter. Young believes publicity. Young was interviewed an earlier study published in by the Wall Street Journal, NPR, the same journal was incorrect ABC News Health Report, Web due to multiple testing errors. MD and Scientific American The original study by Mathews, to name a few. Young and his Johnson and Neil, “You are What colleagues’ findings were found in your Mother Eats” claimed that over 6,700 articles and blog entries, women who eat a lot of cereal at according to a Google search. the time of conception are more While this paper is complete, likely to have a boy than a girl. Young is still actively informing Young, along with his people about randomness and colleagues, Heejung Bang, Ph.D., multiple testing errors. This past of Cornell University and Kutluk winter, he presented lectures about Oktay, MD, FACOG, professor randomness to a statistics class at of Obstetrics & Gynecology and North Carolina State University director, Division of Reproductive and to the Science Communicators Medicine & Infertility Department of North Carolina (SCONC). of Obstetrics & Gynecology from Young also published a letter New York Medical College, in their in the Journal of American paper “Cereal-Induced Gender Medical Association (JAMA) Selection? Most Likely a Multiple that commented on the potential Stan Young speaks to the Science Communicators of Testing False Positive,” assert that statistical limitations of a study North Carolina (SCONC) meeting (Photo by Russ the result of the original study is claiming that bisphenol A is Campbell). easily explained as chance. Young, associated with cardiovascular Bang & Oktay examined the diagnoses, diabetes and abnormal data sets from the original study blood level liver enzyme levels. and noted that 132 food items According to Young, the earlier were tested for two time periods, study, published in JAMA totaling 264 statistical tests. With (September 16, 2008) by Dr. Ian this many tests, it is quite likely A. Lang and colleagues, did not that some apparent statistical adequately address the possibility significance will occur simply by that multiple testing could lead to a chance. false positive result. At the standard significance This spring he published a level of 5 percent, the 264 tests letter in theInternational Journal will yield approximately 13 false of Epidemiology about multiple positives unless the analysis is testing. adjusted to account for multiple testing. Young, Bang & Oakley argue that this is precisely what the authors of the original papers failed to do. 7 P.O. Box 14006 19 T.W. Alexander Drive Research Triangle Park, NC 27709 919.685.9300 (phone) 919.685.9310 (fax) www.niss.org

NISS/SAMSI Affiliates Industry University

AT&T Labs-Research, Florham Park, NJ University of California - Berkeley, Statistics and Biostatistics Avaya Labs, Basking Ridge, NJ Department of Statistics University of Missouri-Columbia, Bayer HealthCare Pharmaceuticals, West Carnegie Mellon University, Department of Department of Statistics Haven, CT Statistics North Carolina State University, Department of Bell Labs - Lucent Technologies, Columbia University, Department of Statistics Murray Hill, NJ Biostatistics North Carolina State University, Department of GlaxoSmithKline, Research Triangle Park, NC University of Connecticut, Department of Mathematics and Collegeville, PA Statistics University of North Carolina at Chapel Hill, Merck Research Laboratories, West Point, PA Duke University, Departments of Statistical Department of Statistics and Operations MetaMetrics, Inc., Durham, NC Science and Mathematics Research PNYLAB, LLC, Princeton, NJ Emory University, Department of University of North Carolina at Chapel Hill, RTI International, Research Triangle Park, NC Biostatistics Department of Biostatistics Sanofi-Aventis Pharmaceuticals, University of Florida, Department of University of North Carolina at Chapel Hill, Bridgewater, NJ Statistics Department of Mathematics SAS Institute, Cary, NC Florida State University, Department of Oakland University, Department of Mathematics SPSS, Chicago, IL Statistics and Statistics Wyeth, Collegeville, PA George Mason University, Department of Ohio State University, Department of Statistics Xerox Innovation Group, Webster, NY Statistics Pennsylvania State University, Department of Yahoo! Research Laboratory, Silicon Valley, Georgetown University Medical Center, Statistics CA Department of Biostatistics, Bioinformatics, Purdue University, Department of Statistics and Biomathematics Rice University, Department of Statistics University of Georgia, Department of Rutgers University, Department of Statistics Government Agencies & Statistics University of South Carolina, Department of National Laboratories University of Illinois Urbana-Champaign, Statistics Department of Statistics Southern Methodist University, Department of Bureau of Labor Statistics, Washington, DC Indiana University, Department of Statistics Statistical Science US Census Bureau, Washington, DC University of Iowa, Department of Statistics Stanford University, Department of Statistics Energy Information Administration, Iowa State University, Department of Texas A&M University, Department of Statistics Washington, DC Statistics Virginia Commonwealth University, National Agricultural Statistics Service, Johns Hopkins University, Department of Departments of Biostatistics and Statistical Fairfax, VA Applied Mathematics and Statistics Sciences National Center for Education Statistics, Medical University of South Carolina, Washington, DC Department of Biostatistics, Bioinformatics National Center for Health Statistics, and Epidemiology Hyattsville, MD University of Michigan, Departments of National Security Agency, Ft. George W. Meade, MD Office of the Comptroller of the Currency (Treasury Department), Washington, DC 8