November 1, 2003 to October 31, 2004

REPORT OF RESEARCH ACCOMPLISHMENTS AND FUTURE GOALS FOR FY 2004 HIGH ENERGY PHYSICS

***

Budget Period: November 1, 2003 to October 31, 2004

***

Grant DE-FG03-92-ER40701 . RESEARCH PROPOSAL SUBMITTED TO THE DEPARTMENT OF ENERGY

Annual Budget for FY 2004 and Report of Research Accomplishments and Future Goals Grant DE-FG03-92-ER40701 *** California Institute of Technology Department of Physics Pasadena, CA 91125 *** Budget Period November 1, 2003 to October 31, 2004 Amount Requested: $6,497,000

David Hitlin Thomas A. Tombrello Professor of Physics Chair, Division of Physics, (626) 395-6694 Mathematics and Astronomy (626) 395-4241

Date: 7/3/03 Date:

Richard P. Seligman Director, Sponsored Research (626) 395-6073

Date: . DOE F 4650.2 OMB Control No. (10-99) 1910-1401 (All Other Editions Are Obsolete) Office of Science (SC) (OMB Burden Disclosure Statement on Back) Face Page

TITLE OF PROPOSED RESEARCH: Report of Research Accomplishments and Future Goals for FY04 High Energy Physics

1. CATALOG OF FEDERAL DOMESTIC ASSISTANCE #: 8. ORGANIZATION TYPE: 81.049 _ Local Govt. _ State Govt. _ Non-Profit _ Hospital 2. CONGRESSIONAL DISTRICT: _ Indian Tribal Govt. _ Individual Applicant Organization's District: 27th _ Other X Inst. of Higher Educ. Project Site's District: 27th, SLAC, Fermilab, CERN _ For-Profit _ Small Business _ Disadvan. Business 3. I.R.S. ENTITY IDENTIFICATION OR SSN: _ Women-Owned _ 8(a) 95-1643303

9. CURRENT DOE AWARD # (IF APPLICABLE): 4. AREA OF RESEARCH OR ANNOUNCEMENT TITLE/#: DE-FG03-92-ER40701 High Energy Physics 10. WILL THIS RESEARCH INVOLVE: 10A .Human Subjects X No _ If yes 5. HAS THIS RESEARCH PROPOSAL BEEN SUBMITTED Exemption No. or TO ANY OTHER FEDERAL AGENCY? IRB Approval Date _ Yes X No Assurance of Compliance No: 10B .Vertebrate Animals X No _ If yes PLEASE LIST IACUC Approval Date or Animal Welfare Assurance No:

6. DOE/OER PROGRAM STAFF CONTACT (if known): 11. AMOUNT REQUESTED FROM DOE FOR ENTIRE P.K. Williams PROJECT PERIOD $ 6,497,000

12. DURATION OF ENTIRE PROJECT PERIOD: 7. TYPE OF APPLICATION: 11/01/03 to 10/31/04 _ New _ Renewal MM/DD/YY MM/DD/YY X Continuation _ Revision _ Supplement 13. REQUESTED AWARD START DATE 11/01/03 MM/DD/YY 14. IS APPLICANT DELINQUENT ON ANY FEDERAL DEBT? _ Yes (attach an explanation) X No

15. PRINCIPAL INVESTIGATOR/PROGRAM DIRECTOR 16. ORGANIZATION'S NAME California Institute of Technology * NAME David Hitlin ADDRESS Mail Code 213-6 TITLE Professor of Physics California Institute of Technology ADDRESS Mail Code 356-49 1200 E. California Blvd. California Institute of Technology Pasadena, CA 91125 1200 E. California Blvd. Pasadena, CA 91125 CERTIFYING REPRESENTATIVE'S PHONE NUMBER 626-395-6694 * NAME Lucy Molina TITLE Contract and Grant Analyst PHONE NUMBER 626 395 2372

* David Hitlin * Lucy Molina SIGNATURE OF PRINCIPAL INVESTIGATOR/PROGRAM DIRECTOR SIGNATURE OF ORGANIZATION'S CERTIFYING REPRESENTATIVE (please type in full name if electronically submitted) (please type in full name if electronically submitted) *Date July 1, 2003 *Date July 2, 2003 PI/PD ASSURANCE: I agree to accept responsibility for the scientific conduct of the project and CERTIFICATION & ACCEPTANCE: I certify that the statements herein are true and complete to the to provide the required progress reports if an award is made as a result of this submission. Willful best of my knowledge, and accept the obligation to comply with DOE terms and conditions if an provision of false information is a criminal offense. (U.S. Code, Title 18, Section 1001). award is made as the result of this submission. A willfully false certification is a criminal offense. (U.S. Code, Title 18, Section 1001). NOTICE FOR HANDLING PROPOSALS This submission is to be used only for DOE evaluation purposes and this notice shall be affixed to any reproduction or abstract thereof. All Government and non-Government personnel handling this submission shall exercise extreme care to ensure that the information contained herein is not duplicated, used, or disclosed in whole or in part for any purpose other than evaluation without written permission except that if an award is made based on this submission, the terms of the award shall control disclosure and use. This notice does not limit the Government’s right to use information contained in the submission if it is obtainable from another source without restriction. This is a Government notice, and shall not itself be construed to impose any liability upon the Government or Government personnel for any disclosure or use of data contained in this submission. PRIVACY ACT STATEMENT If applicable, you are requested, in accordance with 5 U.S.C., Sec. 562A, to voluntarily provide your Social Security Number (SSN). However, you will not be denied any right, benefit, or privilege provided by law because of a refusal to disclose your SSN. We request your SSN to aid in accurate identification, referral and review of applications for research/training support for efficient management of Office of Science grant/contract programs. . DOE F 4620.1 U.S. Department of Energy OMB Control No.

(04-93) Budget Page 1910-1400 All Other Editions Are Obsolete (See reverse for Instructions) OMB Burden Disclosure Statement on Reverse

ORGANIZATION Budget Page No: 1 of 1 California Institute of Technology FY2004/GY05 PRINCIPAL INVESTIGATOR/PROJECT DIRECTOR Requested Duration: 12 (Months) David Hitlin A. SENIOR PERSONNEL: PI/PD, Co-PI's, Faculty and Other Senior Associates DOE Funded (List each separately with title; A.6. show number in brackets) Person-mos. Funds Requested Funds Granted CAL ACAD SUMR by Applicant by DOE 1. 2. 3. 4. 5. 6. ( ) OTHERS (LIST INDIVIDUALLY ON BUDGET EXPLANATION PAGE) 7. ( ) TOTAL SENIOR PERSONNEL (1-6) 10.67 21 $311,598

B. OTHER PERSONNEL (SHOW NUMBERS IN BRACKETS) 1. ( ) POST DOCTORAL ASSOCIATES 11.83 142 $575,790 2. ( ) OTHER PROFESSIONAL (TECHNICIAN, PROGRAMMER, ETC.) 12.0 144 $843,648 3. ( ) GRADUATE STUDENTS 8.84 $172,104 4. ( ) UNDERGRADUATE STUDENTS 5. ( ) SECRETARIAL - CLERICAL 5.0 60 $162,240 6. ( ) OTHER TOTAL SALARIES AND WAGES (A+B) $2,065,380 C. FRINGE BENEFITS (IF CHARGED AS DIRECT COSTS) 27.0% $511,185 TOTAL SALARIES, WAGES AND FRINGE BENEFITS (A+B+C) Excluding GRA $2,576,565

D. PERMANENT EQUIPMENT (LIST ITEM AND DOLLAR AMOUNT FOR EACH ITEM.)

Computing Equipment $100,000

TOTAL PERMANENT EQUIPMENT $100,000 E. TRAVEL 1. DOMESTIC (INCL. CANADA AND U.S. POSSESSIONS) $285,471 2. FOREIGN $123,704

TOTAL TRAVEL $409,175

F. TRAINEE/PARTICIPANT COSTS 1. STIPENDS (Itemize levels, types + totals on budget justification page) 2. TUITION & FEES 3. TRAINEE TRAVEL 4. OTHER (fully explain on justification page) TOTAL PARTICIPANTS ( ) TOTAL COST

G. OTHER DIRECT COSTS 1. MATERIALS AND SUPPLIES $382,943 2. PUBLICATION COSTS/DOCUMENTATION/DISSEMINATION $24,785 3. CONSULTANT SERVICES 4. COMPUTER (ADPE) SERVICES 5. SUBCONTRACTS $1,350,000 6. OTHER (62% of GRA support) $106,704 TOTAL OTHER DIRECT COSTS $1,864,432 H. TOTAL DIRECT COSTS (A THROUGH G) $4,950,172

I. INDIRECT COSTS (SPECIFY RATE AND BASE) (Excludes D. and G. 4-6) 52.61% 75% TOTAL INDIRECT COSTS 24.50% 25% $1,546,828 J. TOTAL DIRECT AND INDIRECT COSTS (H+I) $6,497,000 K. AMOUNT OF ANY REQUIRED COST SHARING FROM NON-FEDERAL SOURCES L. TOTAL COST OF PROJECT (J+K) $6,497,000 . Senior Personnel

Barish, B.C. ( 9%) Hitlin, D.G. (18%) Kamionkowski, M. (18%) Kapustin, A. (18%) Newman, H.B. (18%) Peck, C.W. (18%) Ooguri, H. (18%) Politzer, H.D. (0%) Porter, F.C. (18%) Preskill, J.P. ( 9%) Schwarz, J.H. (18%) Weinstein, A.J. (12%) Wise, M.B. (18%) . Contents

Budget Request iii

HEP Personnel — FY 2003 xi

I Theory 1

1 Research in Theoretical Physics 3

2 Theoretical Particle Astrophysics 13

II Experimental Program 23

3 The Experimental Program 25

4 CLEO-II and CLEO-III 35

5 CMS at LHC and L3 at LEP2 55

6 BaBar 119

7 MINOS 155

III Technical Support 179

8 Experimental Computing 181

9 LHCNET: Wide Area Networking and Collaborative Systems for HENP 189

i ii CONTENTS

IV Curriculum Vitae 229

V Bibliography 245

A Published Papers — Theory (2002 - present) 247

B Published Papers — Experiment (2002 – present) 251

C Unpublished Papers — Theory (2002 - present) 257

D Preprints and Conference Proceedings — Experiment (2002 – Present) 259

E Theses (2002 – Present) 265 Budget Request

The proposed budget for the Caltech High Energy Physics Program for FY2004, the fifth year of a five year grant cycle, is presented and discussed herein. The Caltech grant is divided into several Tasks: A and B, representing theoretical particle physics and astrophysics, respectively, C representing the individual HEP experiments, S, for general administrative and technical support, including computing, and N representing costs for those global networking infrastructure and development activities administered by Caltech for the HEP community. The proposed base budget for Tasks A, C and S for FY2004 is increased by 4%, representing the cost of technical inflation. We propose an increase in Task B, Theoretical Particle Astrophyics, by $155K, as development of our capabilities in this exciting and rapidly expanding field is well motivated and, indeed, required, to maintain the synergy between our theoretical efforts and current and proposed experiments. The budget request for Task N is increased in accord with the DOE’s computing infrastructure plan. As the computing requirements of BABAR, MINOS and CMS are continuing to increase, we request $100K for ongoing upgrades to our computing installation. The indirect rate, fixed by agreement with the Institute at 52.61% at the start of this grant cycle, remains unchanged, although the Institute staff benefit rate has been raised to 27%. Last year we began combining the CLEO and CMS groups. The combination of the groups of Weinstein and Newman will measurably strengthen the Caltech CMS effort. The consolidation of these groups will be complete early in FY2004. A $22K supplement beyond the base budget for CMS travel is also requested.

iii iv Budget Request

CALIFORNIA INSTITUTE OF TECHNOLOGY HIGH ENERGY PHYSICS GRANT PROPOSAL FY2004 (GY05 of 5) BUDGET BREAKDOWN (Thousands of Dollars)

Actual Proposed FY03 FY04

Operations: (Base Program)

I. Th. Part. Physics (Task A) 750 780 II. Th. Part. Astrophysics (Task B) 95 250 III. Experimental (Task C) 2,190 2,277 IV. Technical (Task S) 565 588 V. LHCNET (Task N) 2,000 2,480 Total Operations 5,600 6,375

Equipment, R&D, other:

I. Computing Equipment 100 100 TotalEquipment,R&D,other 100 100

Total Grant 5,700 6,475

Additional DOE Funding:

I. CMS Travel (supplement) 22 22

Total Additional DOE Funding: 22 22

Total Funding 5,722 6,497 v

Budget Discussion

We present here discussions and breakdowns of our DOE budget request for the Caltech HEP program for FY04 organized by task and sub-tasks.

I. Theory The Theory Group at Caltech (Tasks A and B) are leaders in broad areas of theoretical physics, from string theory and M theory to particle physics phenomenology, quantum computing and theoretical particle astrophysics. Faculty members Kamionkowski, Kapustin, Ooguri, Preskill, Schwarz, and Wise work closely with a productive group of senior research fellows, postdoctoral scholars and graduate students. The Caltech theory group has also benefited from significant private gifts, which have greatly enhanced the main DOE-supported effort, and have enabled recent extended visits by Callan, Hawking, Vasa and Witten. For FY2004, we are requesting an inflation increase for Task A, discussed in Chapter 1 and increase of $155K in support for Marc Kamionkowski’s exciting and timely work in theoretical particle astrophysics, discussed in Chapter 2.

TABLE Ia Theoretical Particle Physics TASK A

Actual Proposed FY03 FY04

Theoretical Particle Physics 750 780

TABLE Ib Theoretical Particle Astrophysics TASK B

Actual Proposed FY03 FY04

Theoretical Particle Astrophysics 95 250 vi Budget Request

II. Experimental Operating Funds Table II shows a proposed breakdown of the experimental operating budget into shared support areas and direct support for the various experimental groups (e.g., sub-tasks). The detailed budgets for each sub-task is given in Tables II-VII. Proposed funding for FY2004 is shown, along with comparisons with the actual support in FY2003. Our technical infrastructure is managed under a separate task (Task S), which is delineated in Table III. Note also that costs for LHCNet, which have been managed by Caltech for many years,were broken out at the start of this grant cycle as a separate task, Task N (see Table IV).

TABLE II Experimental Operating Funds - Breakdown by Groups TASK C

Actual Proposed FY03 FY04

HEP Common Support 222 232

Experimental Groups: a) CMS 759 735 b) BABAR 701 810 c) MINOS 508 500

Total Task C 2,190 2,277

Shared Common Support Experiment-independent common support includes secretarial support, graphic arts, expenses for seminar speakers, travel to conferences in which the traveler does not give specific experimental results, certain small equipment purchases - (e.g., oscilloscopes, electronics), etc.. Finally, we occasionally provide a small amount of support to seed new experimental activities that do not fit naturally into the direct support of the experimental groups. The proposed budget for FY2004 (see Table II) contains only an inflation adjustment.

CMS Group The Caltech CMS/L3 group is closing out its work on L3 at LEP this year. Since the experiment was first proposed in 1983, the group has had leading roles in computing, networking, the development of BGO crystals and the RFQ system for BGO calibration, and especially in data analysis. The group led two of the experiment’s three new particle search analysis groups, and has been a major contributor to the measurement of the W mass, width and search for anomalous electroweak couplings at LEP2. In the final stages of L3 analysis, the group has led the searches for supersymmetric leptons, the direct measurement of the number of neutrino species, the searches for extra dimensions, and other physics involving final states with one or two photons and missing energy. The group also leads the US CMS Collaboration, which is now in the latter part of the construction phase, and is scheduled to run in 2007. It has principal roles in CMS and US CMS software, computing and networking for CMS and HENP as a whole, as well as the development of the CMS ECAL crystals vii and the construction of a CMS monitoring system. It has made substantial contributions to the analysis, optimizing the detector performance for electrons and photons, and for forward muons, and leading some of the Higgs search developments, involving the CMS ECAL and tracker. The analysis efforts focusing on electron and photon signatures are synergistic with the interests of the CLEO group members who are now joining CMS, and who also are contributing to the ECAL test beam, monitoring and calibration work at CERN. Funds requested for CMS Travel refer to a supplemental request to the DOE used to support US physicist’s participation in LHC experiments. These funds were distributed among the universities in the US CMS collaboration. Funds requested for CMS Equipment (FNAL MPO) refer to CMS ECAL monitoring construction. This project was baselined in 1998 with funds provided through US CMS construction project office via a Fermilab MPO. Since then the scope of the project was changed by adding two additional laser systems following recommendations of the Lehman committee. The total cost of the project is approximately $1,800K. Starting in 2002 we began to shift a small amount of effort to MINOS (limited by funding). As the last L3 grad student will finish this Fall, we intend to shift this support to MINOS. Grad students joining our group within the next two years will do their theses on MINOS physics during the period up to LHC startup in 2007. The CMS/L3 effort was under financial stress and shrank significantly over the last few years. The group needs to cover salaries and travel for the group in FY2003 at a level that will prevent the group from shrinking further, in the face of growing CMS responsibilities. The minimum level is $ 790K, including support for the former CLEO group members who are are joined with the CMS group, and partial support for graduate students on MINOS.

BABAR The Caltech group was the prime mover in the proposal to build a high luminosity asymmetric e+e− storage ring facility at SLAC and played an important role in the construction of the BABAR detector (where David Hitlin was founding Spokesman and Frank Porter is the Computing System Manager and Council Vice-chair. Porter will become chair of the Collaboration Council in the Fall of 2003). We have had major responsibilities in on-line computing (On-line Computing Farm, On-line Event Process- ing, Calibration System, Level 3 Trigger algorithms, Distributed histogramming and Graphical User Interfaces), oﬀ-line computing (Reconstruction Code, Physics Event Generator, Particle Identiﬁcation), as well as in the construction of the calibration source for the CsI calorimeter. Emphasis is now on operating the detector, processing data, generating large Monte Carlo simulation datasets, dealing with statistical issues, and data analysis for physics results. Caltech has been an important contributor to a variety of BABAR’s physics publications. We have recently, in response to a call from BABAR management, taken on several responsibilities on PEP-II, including detailed beam-beam simulations in the strong-strong regime and the development of an instrument to measure vertical beam size. In addition to these activities on BABAR, Hitlin is leading a study of the physics and technical challenges of a higher-luminosity B factory. The Caltech group has embarked on R&D towards the possibility of using scintallating liquid xenon as a replacement for the CsI(Tl) calorimeter in the higher-luminosity environment.

MINOS Caltech work on MACRO is now essentially complete and the focus of activity is on MINOS construction, proton intensity upgrades, start of analysis of atmospheric neutrinos from the far detector and start of analysis work for beam neutrinos. Barry Barish is the spokesman of the U.S. MACRO collaboration. viii Budget Request

Work on MINOS has been centered on development and construction of the scintillator system but is now shifting towards analysis and proton intensity upgrades. Doug Michael is MINOS Co-Spokesperson and has acted as the manager for the scintillator system. Caltech has recently completed our very significant construction responsibility for one half of the scintillator modules for the far detector. We request support to continue our activities for the existing group. Essentially the entire budget is for salary and travel. The MINOS budget here provides partial travel support and summer salary for Barish and Peck, travel and salary for Michael, travel and salary for two postdocs, travel for MINOS students and some support for undergraduate work on MINOS. In the last year, we hired two new postdocs, Hai Zheng and Chris Smith. We have had two graduate students in the last year who started to work on MINOS but neither is currently working on MINOS. One left Caltech for health reasons and the other has decided to pursue a different field of physics. We are looking to add 1-2 students in the fall. Student salaries will be supported from the “CMS” budget, with Newman as thesis advisor. Technician and hardware expenses for MINOS have been covered by funds from Fermilab. This has included significant support for Hanson and Mossbarger over the last few years but this support for MINOS is now finished. ix

Technical Support (Task S) The Technical Support Task (Task S) provides the infrastructure and technical base for the entire experimental HEP research program. The technical infrastructure request represents of the order of 10% of the total DOE support of HEP at Caltech, which represents a steady decline over a number of years, a direct result of continuing budget pressures. Our infrastructure support is now at a level that allows technical contributions for our experimental efforts in only a limited way. We have recently completed a major role in the construction of MINOS scintillation counters. Technical support continues to evolve away from traditional mechanical engineering-dominated efforts towards the new paradigm in object-oriented computing and database storage, integration of computing with wide area networking, and electronic design using modern electronic design tools. The investment in computer equipment for the high energy group has been developed as a partnership between Caltech and DOE. We have used this system to provide support for L3, CLEO, MACRO, BES and now BABAR , MINOS and CMS Monte Carlo production efforts and data analysis. Experience tells us that an on-going computing equipment budget is crucial to operating a system that won’t fall rapidly into obsolescence. Because of the rapid evolution in the computing industry, and in our computing requirements, at a minimum, the level of such funding should be 20% of the initial investment. Thus, we propose a budget for computing equipment of $100k in FY2004. We firmly believe that continuing computing equipment support at a minimum of 20% of the total investment per year in upgrades is the absolute minimum required to maintain our computing system at an adequate level. The FY97-FY99 level was $100K, which was demonstrably inadequate to keep our systems current. We requested a total of $344K in FY1999-2000, plus matching funds, to upgrade the system. The DOE response of $300K over three years, while substantial, was insufficient by itself to keep the system capable of dealing with the torrent of data coming from the new B factories. We have included in this proposal a request for continued investment in an attempt to stave off obsolescence.

TABLE III General and Technical Support TASK S

Actual Proposed FY03 FY04

Computer Operations 203 211 General and Technical Support 362 376

Total Task S 565 588 x Budget Request

VIII. LHCNET Networking (Task N) The details of the request are included in the LHCNet Chapter for Task N. The long term plan for LHCNet is aimed at meeting the needs of the LHC experiments for transatlantic networking, as well as the needs of DOE/DHEP’s other major programs, based on projections of the ICFA Network Task Force, ICFA-SCIC, the Hoffmann Review Panel on LHC Computing, as well as CMS and ATLAS. The projected needs for HEP as whole, and the detailed needs and costs for US- CERN networking in support of the LHC program, have been reviewed by the Transatlantic Network Working Group co-chaired by H. Newman and L. Price (ANL) in 2001, and have been modified slightly in 2002-3 to reflect the continued improvement in bandwidth costs, and the actual costs of network routing and switching equipment. The basic requirements and cost parameters used to determine the entries in the table are summarized below:

1. By doubling the bandwidth each year of the cycle, reach 10 Gbps in development by FY04, and in production by FY05. This meets the needs of CMS and Atlas and covers the majority of other transatlantic network use by HEP, such as the developing international Data Grids used by DOE’s major HENP experiments.

2. Cost per unit bandwidth is estimated to decrease by 37% per year, or a factor of 2 eery 1.5 years starting in 2004, following cost evolution in other ﬁelds of information technology.

3. Reach a constant DOE expenditure level during LHC operation, from 2007 onwards. This assumes that non-US sources contribute a signiﬁcant amount to the overall transatlantic link cost, as is the case now.

TABLE IX Networking - Task N (Operating)

Actual Proposed FY03 FY04

Link Charges 1,000 1,200 OH + Infrastructure 1,000 1,280

LHCNET Network Total 2,000 2,480

Link Bandwidth (Mbps) 2,500 5,000

The amounts in the table are the projected US DOE commitments. Additional contributions will come from non-US sources. Note that in FY03 we requested $ 2.24 M according to the plan reviewed by DOE in 2001, and we only received $ 2.00 M. By delaying link upgrades and equipment purchased, we have been able to keep our costs at $ 2.00 M for FY03. The request for FY04 for Task N is thus $ 2.48 M. HEP Personnel — FY 2003

Caltech High Energy Physics Program November 1, 2002 - October 31, 2003 (Percentage of yearly salary charged to the Grant is indicated in parentheses) A. Professorial Faculty 1. Experimental 2. Theoretical

Barish, B. C. (9%) Kamionkowski, M. (18%) Hitlin, D. G. (18%) Kapustin, A. (18%) Newman, H. B. (18%) Ooguri, H. (18%) Peck, C. W. (18%) Politzer, H. D. ( 0%) Porter, F. C. (18%) Preskill, J. P. ( 9%) Weinstein, A. J. (12%) Schwarz, J. H. (18%) Wise, M. B. (18%)

B. Research Faculty 1. Experimental

Michael, D. (100%) Shevchenko, S. (100%)

C. Postdoctoral Scholars 1. Experimental 2. Theoretical

Albert, J. ( 50%) Brandhuber, A. (0%) Bornheim, A. (100%) Calmet, X. (0%) Smith, C. (100%) Chepelev, I. (0%) Narsky, I. (100%) Cheung, E. (100%) Pappas, S. (100%) Gomis, J. (0%) Zheng, H. (100%) Graesser, M. (100%) Two(4 mos.) (100%) Kaminsky, K. (50%) Moriyama, S. (0%) Okawa, Y. (0%) 3. Visitors Schulz, M. (100%) Su, S. (0%) Mao, R. (0%) Zhu, K.(10 mos.) (0%) Theory visitors (7) (0%)

xi xii HEP Personnel — FY 2003

HEP Personnel — FY 2003 Caltech High Energy Physics Program

D. Experimental Research Staﬀ 1. Physicists 2. Computer Scientists & Engineers

Bunn, J. ( 0%) Adamczyk, D. (100%) Dubois-Felsman, G. (100%) Aslakson, E. ( 0%) Legrand, I ( 0%) Collados, D. ( 0%) Litvin, V. ( 0%) Denis, G. (100%) Ryd, A. (9 mos.) (100%) Fernandes, J.(10 mos.) ( 0%) Steenberg, C. ( 0%) Galvez, P. (100%) Sun, W.(5 mos.) (100%) Iqbal, S. ( 0%) Wilkinson, R. (100%) Nae, D.(8 mos.) ( 0%) Yang, S.(4 mos.) (100%) Ravot, S.(9 mos.) (100%) Zhang, L. ( 0%) Singh, S. ( 0%) Zhu, R. (100%) Thomas, M. (7 mos.) ( 0%) van Lingen, F.(8 mos.) ( 0%) Voicu, R.(11 mos.) (0%) Wei, K. (0%) Xia, Y. (8 mos.) ( 0%) xiii

HEP Personnel — FY 2003 Caltech High Energy Physics Program

E. Graduate Students 1. Experimental 2. Theoretical

Chen, E. (100%) Abeyesinghe, A. (0%) Dvoretskii, A. (100%) Ahn, C. (0%) Erwin, J. (100%) Bonderson, P. (0%) Gataullin, M. (3 mos.) (100%) Borokhov, V. (17%) Lipeles, E. (100%) Ciocarlie, C. (0%) Piatenko, T. (100%) Cortese, J. (0%) Randall, P. (6 mos.) (100%) Daftaur, S. (0%) Samuel, A. (100%) Dortsen, M. (0%) Shapiro-Bridger, A. (100%) Evnin, O. (0%) Sun, W. (7 mos.) (100%) Harrington, J. (0%) Jenkins, A. (0%) Kile, J. (0%) Lee, C. (0%) Lee, P. (25%) Lee, W. (25%) Li, Y. (25%) McLoughlin, T. (25%) Mochon, C. (25%) Okuda, T. (25%) O’Connell, D. (0%) Ozakin, A. (17%) Park, J. (25%) Spedalieri, F. (0%) Swanson, I . (0%) Toner, B. (25%) Wang, P. (0%) Wessling, M. (25%) Wu, X. (25%) xiv HEP Personnel — FY 2003

HEP Personnel — FY 2003 Caltech High Energy Physics Program F. Staﬀ Associate Engineer - J. Hanson (100%) Electronics Technician - J. Trevor (0%) Laboratory Technician (100%) 8 Mechanical assemblers (5 mos.) (0%) 2 Secretaries (150%) 1 Secretary (38%) 1 Secretary (P/T) (0%) 1 Secretary (85%) 1 Secretary (35%) 1 Secretary (94%) 1 Oﬃce Aide (P/T) (100%)

G. Computing Sr. Computing Analyst - J. Barayoga (100%) Part I

Theory

1. Research in Theoretical Physics

A. Brandhuber, X. Calmet, I. Chepelev, E. Cheung, J. Gomis, M. Graesser, K. Kaminsky, A. Kapustin, S. Moriyama, Y. Okawa, H. Ooguri, J. Preskill, M. Schulz, J. Schwarz, S. Su, M. Wise

1.1 String Theory

The string theory group consists of professors Kapustin, Ooguri, and Schwarz, senior research fellow Gomis, and postdoctoral scholars Brandhuber, Chepelev, Cheung, Kaminsky, Moriyama, Okawa, and Schulz. There are also about a dozen graduate students in the string group. The group has beneﬁted from sabbatical visits by S. James Gates from U. of Maryland (Sept. 2001 to June 2002), Curtis Callan from Princeton U. (Jan. to June 2002), and Cumrun Vafa from Harvard U. (Jan. to July 2003). There have been various additional distinguished visitors including Edward Witten (three weeks each in Jan. 2002 and Jan. 2003) and Stephen Hawking (six weeks in March and April 2002).

In [1] Brandhuber studied M-theory compactification on manifolds of G2 holonomy which develop particular codimension seven singularities that localize chiral fermions charged under SU(N)and SO(2N) gauge groups. The geometry of these spaces is that of a cone over a six-dimensional Ein- stein space, which can be constructed by (multiple) unfolding of hyper-Kahler quotient spaces. In a dual type IIA description the corresponding background is given by stacks of intersecting D6-branes, and chiral matter arises from open strings stretching between them. Usually one obtains (bi)fundamental representations but by including orientifold six-planes in the type IIA picture he was able to find more exotic representations like the antisymmetric, which is important for the study of SU(5) grand unification, and trifundamental representations. He also exhibited many cases where the G2 metrics can be described explicitly, although in general the metrics on the spaces constructed via unfolding are not known. In [2] Brandhuber studied families of pp-wave solutions of type-IIB supergravity that have (light-cone) time dependent metrics and RR five-form fluxes. They arise as Penrose limits of supergravity solutions that correspond to rotating or continuous distributions of D3-branes. In general, the solutions preserve sixteen supersymmetries. On the dual field theory side these backgrounds describe the BMN limit of N = 4 SYM when some scalars in the field theory have non-vanishing expectation values. He studied the perturbative string spectrum and in several cases he was able to determine it exactly for the bosons as well as for the fermions. He found that there are special states for particular values of the light-cone constant P+. In a collaborate effort,[3] Brandhuber showed that the Konishi anomaly equations are an effective tool to construct the exact effective superpotential of the glueball superfields in N = 1 supersymmetric gauge theories. He used the superpotentials to study in detail the structure of the spaces of vacua of these theories and considered chiral and non-chiral SU(N) models, the exceptional gauge group G2, and models that break supersymmetry dynamically. Quantization of a constrained classical system can be regarded as a deformation of the algebra of functions on the classical phase space (Poisson manifold) of the system as an associative algebra. Recently there has been many spectacular developments in deformation theory of associative algebras following the first general solution of deformation quantization by Kontsevich [4]. The solution presented by Kontsevich uses in an essential way ideas of perturbative string theory. Kontsevich’s construction

3 4 1 Research in Theoretical Physics

was further clarified by Cattaneo and Felder in [5], where an explicit path integral formula for the star- product of functions on Poisson manifolds was given. In the work of Iouri Chepelev [6] and graduate student Calin Ciocarlie, the Kontsevich–Cattaneo–Felder construction was extended to the case when the classical phase space is an arbitrary supermanifold. The star-product of functions of bosonic and fermionic coordinates is represented as a path integral of a certain two dimensional sigma-model. The latter is a topological truncation of the superstring. The superembedding method used in [6] for the quantization of this topological sigma model may have relevance to the problem of covariant quantization of the superstring. Edna Cheung has been interested in studying strings in the background of nonconstant Neveu– Schwarz B-field. This field, being the partner of the graviton and coupling only to strings, should be a unique window to stringy behaviours. Already at a constant B-field level it renders spacetime noncommutative. A toy model to begin the study is provided by N = 2 strings whose gauged N =2 supersymmetry on the worldsheet provides a lot of simplification and retains all the stringy characteristics. In [8] she classified all the possible strings by analyzing the different ways one can embed the N = 2 superconformal algebra into the often bigger symmetry algebra allowed by the four dimensional target spaces. She put the known types of N = 2 strings in a unified framework and proposed new ones. Another four dimensional target space with constant field strength of NS B-field is provided by the Nappi-Witten space, in the context of N = 1 strings, where this NS field distorts the spacetime and cannot be treated as small perturbation to the flat space. Nappi-Witten space has the topology of Minkowski R3,1. This model is exactly solvable thanks to the underlying WZW model structure. In [7] she proposed a new field realization of the Nappi-Witten algebra, constructed the tachyon vertex operators and computed the N-point correlation functions in this model. The new free field realization makes the spacetime interpretation very transparent, which has not been the case with other solvable WZW models. Similar attempts to give a spacetime interpretation to the conformal field theoretical results are given in the SL(2,R) WZW model with relation to AdS3 space[9]. Jaume Gomis has been working in the last year on various aspects of the duality between gauge theory and string theory in gravitational plane-waves. In [10], he and Hirosi Ooguri have studied this gauge-gravity duality for gauge theories closer to QCD, exhibited enhancement of symmetries, and identified the dual worldsheet CFT description of these gauge theories. In collaboration with Lubos Motl and Andrew Strominger, Gomis has identified [11] the dual description of a certain plane wave geometry in terms of a dual CFT. A precise identification of states and expansion parameters in the dual theories was found. In a collaboration with Sanefumi Moriyama and Jongwon Park [12, 13], a proposal was made for how to compute string interaction in the plane wave background using gauge theory. The proposal was tested in [12, 13] and found to reproduce all string amplitudes. Moreover, a direct diagrammatic correspondence was found between gauge theory and string theory computations and all the Neumann matrices, and the prefactor of string theory was precisely transcribed in terms of gauge theory data. In [13] the authors have extended and tested the proposal to the full open + closed string theory. Anton Kapustin, in collaboration with K. Hori, studied field theory on wrapped NS five-branes using world-sheet methods [14]. The correspondence between field theory on NS five-branes and string theory in the corresponding linear dilaton background is an example of holographic correspondence, but unlike in the case of the AdS/CFT correspondence there is no Ramond-Ramond field to complicate the worldsheet description. Kapustin and Hori constructed 2d gauged linear sigma models that describe strings propagating in the vicinity of NS five-branes wrapped on cycles in Calabi–Yau manifolds and studied the low-energy physics using the mirror map. They showed that in the case when the low-energy theory lives in four dimensions, the low-energy Lagrangian is described by the Seiberg–Witten prepotential of an N = 2 gauge theory, an expected result. They also showed that the world-sheet CFT becomes 1.1 String Theory 5 singular precisely when the low-energy field theory flows to a nontrivial infrared fixed point. Together with two students, Vadim Borokhov and Xinkai Wu, Kapustin studied operators in 3d abelian gauge theories which carry vortex charge [15, 16]. These operators are analogous to topological disorder operators in 2d CFTs, and can be studied using similar methods. One complication is that gauge theories in three dimensions are strongly coupled in the infrared, and to make computations possible it is necessary to use a large N expansion. Kapustin et al. computed the dimensions and other quantum numbers of vortex operators in both supersymmetric and non-supersymmetric abelian gauge theories and showed that they agree with the predictions of 3d mirror symmetry. Using supersymmetric non-renormalization theorems, they gave a proof of 3d mirror symmetry for all abelian gauge theories. Anton Kapustin and graduate student Yi Li studied D-branes in topological Landau–Ginzburg models [17, 18]. They showed that the problem of classification of D-branes reduces to a purely algebraic one (matrix factorization of a polynomial), and that all topological correlators can be computed in a closed form, once a solution of the factorization problem is chosen. This work provides simple examples of completely soluble open-closed topological string theories. Along the lines of the pp-wave/SYM correspondence, Peter Lee, Sanefumi Moriyama and Jongwon Park were trying to figure out how the correspondence extends to the interacting string theory. Although one proposal for the correspondence is quite successful for the supergravity modes [19], they found this proposal needs modification for the stringy modes [20]. Especially, the prefactor in the pp-wave string field theory is not cast into the proposed form of an energy difference due to an extra relative minus sign between different modes. One of the most crucial open problems in vacuum string field theory is that the absolute value of the D-brane tension has not been reproduced. Yuji Okawa proposed a description of open string fields on a D-brane in vacuum string field theory, and showed that the open string mass spectrum and the D25-brane tension are correctly reproduced based on the proposal [21]. This resolved the controversy in the literature, and provided strong evidence to support vacuum string field theory. It is known that noncommutative Chern–Simons theory can be classically mapped to commutative Chern–Simons theory by the Seiberg–Witten map. Kirk Kaminsky, Yuji Okawa and Hirosi Ooguri provided evidence that the equivalence persists at the quantum level by computing two and three- point functions of field strengths on the commutative side and their Seiberg–Witten transforms on the noncommutative side to the first nontrivial order in perturbation theory [22]. The large N duality between gauge theories and string theories continued to be the main line of research by Ooguri. One of such dualities, proven by Ooguri and Vafa last year [23], has attracted much attention. In particular it played an essential role in the work by Dijkgraaf and Vafa where they showed that effective superpotentials for a large class of supersymmetric gauge theories in four dimensions can be computed using eigenvalue distributions of associated random matrix models in the planar limit. Ooguri and Vafa explored this duality further and showed that nonplanar diagrams of the matrix models compute gravitational corrections to the superpotentials [24] [25]. They also found that these gravitational corrections are realized in the gauge theory in a novel fashion, where the standard Grassmannian property of gluino fields are deformed to make them obey the Clifford-type algebra. This leads to an interesting Lorentz violating coupling between photon and graviton, and its astrophysical consequences are currently under investigation. Understanding time dependent singularities in string theory is another topic that Ooguri studied this year. In the past few years, Maldacena and Ooguri wrote a series of papers in which they have solved the worldsheet theory for string in the three-dimensional anti-de Sitter space. It turns out that one can construct an interesting class of time-dependent background geometries by taking quotients of the anti-de Sitter space, called BTZ black holes. By the AdS/CFT correspondence, these backgrounds correspond to entangled states of two copies of conformal field theories in two dimensions, and as such these should be well-defined. It turns out that a particular limit of a BTZ black hole gives rise to 6 1 Research in Theoretical Physics the time-dependent orbifold studied by Liu, Moore, and Seiberg, whose work generated various puzzles about string theory in such a geometry. Ooguri, together with Kraus and Shenker, used the AdS/CFT correspondence to address some of these puzzles [26]. They found that a string theory amplitude in a BTZ geometry has two equivalent descriptions. In the first, only regions outside the horizon appear explicitly, and so amplitudes are manifestly finite. In the second, regions behind the horizon and on both sides of the singularity appear, thus yielding finite amplitudes for virtual particles propagating through the black hole singularity. They found that this equivalence between descriptions only outside and both inside and outside the horizon is reminiscent of the ideas of black hole complementarity. Since arriving at Caltech, Michael Schulz has focused his research on studying more exotic compactifications of string theory, in which the compact manifolds are non-Calabi–Yau, non-Kähler, and potentially even non-complex. In work performed in collaboration with Shamit Kachru of Stanford University and also Prasanta Tripathy and Sandip Trivedi of the Tata Institute of Fundamental Re- search, Schulz has described a novel class of supersymmetric string orientifolds in which the compact manifolds are twisted tori [27]. Twisted tori are mathematically simple objects that differ from ordinary tori only in that some of the circles are nontrivially fibred over the rest. They are topologically distinct from Calabi–Yau manifolds, and in the examples of interest are also non-Kähler. Without simplifying assumptions, such as a no-flux ansatz or the assumption of Becker-type spinor conditions, it has proven extremely difficult to systematically solve even the low energy equations of motion of string theory. So, rather than attack the problem directly, Schulz and collaborators sought to first obtain at least a restricted class of non-Calabi–Yau compactifications via duality from a class of more well-understood torus orientifolds which three of the authors had previously studied [28]. The previous work focused on moduli stabilization from internal Neveu–Schwarz and Ramond-Ramond fluxes in the T 6 orientifold. However, via T-duality, the quantized Neveu–Schwarz fluxes can be transformed into discrete twists in the compact geometry. Together with his collaborators, Schulz studied the twisted torus orientifolds resulting from the original theory via one, two and three T-dualities. In addition to providing examples of non-Calabi–Yau, non-Kähler compactifications, some unexpected results were also obtained. The theory arrived at after three T-dualities has a purely geometrical lift to M-theory. In the case of N = 2 supersymmetry, it can be interpreted as M-theory on a Calabi– Yau threefold times a circle, with a nonstandard circle (in the torus fiber of the Strominger–Yau–Zaslow fibration of the Calabi–Yau) chosen for the M-theory reduction. Reducing on the standard circle gives an N = 2 Calabi–Yau compactification of type IIA string theory. A consequence of this result is that all of the T-dual N = 2 (twisted) torus orientifold vacua are related by an M-theory circle swap to the conjectured web of N = 2 Calabi–Yau string vacua. While this is a noteworthy result, it would be more interesting still, if one could provide examples of exotic compactifications analogous to the twisted torus orientifolds, but which differ by being non-duality-related to more standard string vacua like Calabi–Yau vacua, or ordinary torus orientifolds. Schulz is currently attempting to construct such examples, as well as to provide independent checks of some of the dualities mentioned above. He is also working toward providing a more intrinsic description of the twisted torus orientifolds within each dual theory, without using T-duality as a crutch. A fundamental obstacle to testing the conjectured duality between N = 4 super Yang–Mills theory 5 and type IIB superstring theory in AdS5 × S has been the inability to compute string effects in this background. However, it was recently discovered that in a certain (Penrose) limit one obtains a pp-wave geometry in which the GS formulation can be quantized in light-cone gauge making string computations tractable. This has led to a deeper understanding of the duality for a corresponding limit of the gauge theory (proposed by Berenstein, Maldacena, and Nastase). John Schwarz has been exploring light-cone gauge superstring field theory in this background. This generalizes the formalism developed by Green and Schwarz for flat space 20 years ago. The generalization includes an additional mass parameter µ that vanishes in the flat space limit. However, to make contact with the dual perturbative gauge theory, one is interested in the large µ limit. The formulas for the Neumann coefficients (worked out by 1.2 Particle Phenomenology 7

Spradlin and Volovich) need to be explored in this limit, which is mathematically challenging. The first step was carried out in [29] and the project was carried to completion in work in collaboration with He, Spradlin, and Volovich [30]. The limit turns out to give very simple expressions up to corrections that are exponentially small. This made it possible to carry out nontrivial tests of the BMN duality and to make predictions for the behavior of the dual gauge theory to all orders of perturbation theory. Schwarz, together with visiting Moore Distinguished Scholar Curtis Callan (on sabbatical leave from Princeton) and four Caltech graduate students (Lee, McLoughlin, Swanson, Wu), has been exploring whether these results can be extended beyond the pp-wave limit. Specifically, the leading 1/R2 corrections to the Penrose limit are retained and treated as a perturbation. This project has been underway for more than a year and is finally almost complete, largely thanks to the perserverance of the students. Numerous nontrivial tests of the duality that probe string contributions have been successfully carried. Cumrun Vafa stayed at Caltech for January to July 2003 as a Gordon Moore Distinguished Scholar. He wrote five papers during this stay, about the matrix model descriptions of supersymmetric gauge theories [24] [25][32][33] and about the new method to compute all genus topological string amplitudes [34].

1.2 Particle Phenomenology

Presently the phenomenology group at Caltech consists of Professors Preskill and Wise, postdoctoral scholars Calmet Graesser and Su and and a number of graduate students. In the fall Shufang Su will be leaving for a facutly position at University of Arizona and Christian Bauer will be arriving. Most of Mark Wise’s recent research has focussed on phenomelogical issues with direct relevance to experiment. For example the Brookhaven measurement of g − 2 for the muon gathered a lot of inerest from the high energy community over the last few years. In collaboration with M. Ramsey-Musolf Wise computed, in chiral perturbation theory, the logarithmically enhanced part of the light-by-light 2 scattering contribution to g − 2. The part enhanced by log [mπ/ΛQCD] was previously computed and Ramsey-Musolf considered the part enhanced by log[mπ/ΛQCD] which is also model independent [35]. Caltech has an active effort in B phsics both on the experimental and theoretical side. Recently Wise studied [36] (in collaboration with Leibovich and Ligeti) the contributions of suppressed terms in the operator product expansion, to the endpoint region of the electron spectrum in semileptonic B decay. They found that their contribution was suprisingly large and that they are important for the extraction of an accurate value of Vub from these decays. A recent development in standard model physics is the development of the Soft Collinear Effective Field Theory (SCET). This effective field theory for QCD is useful in kinematic situations when there are strongly interacting particles with large energy E and low invariant mass m, and the expansion parameter is λ = m/E. It has been applied to B decays like B → Dπ where is was used to provide a simple all orders in perturbation theory proof of factorization. Recently the work of Wise has focussed on applications of this effective field theory. With Leibovich and Ligeti he clarified the role of light quark masses in this effective field theory [37] and with Bauer and Manohar he used it to study enhanced nonperturbative effects in hadronic jet distributions for e+e− → hadrons [38]. Shufang Su has worked mostly on the phenomenology of low energy supersymmetry. She studied the ˜ → 0 0 pair production of the light top-squark with its consequent decay of t1 tχ1 (with χ1 being the lightest neutralino), followed by t → bW [39]. The stop mass, left-right mixing and the lightest neutralino mass can be obtained from the production cross section with polarized electron beam and stop decay kinematics. In addition, information on the neutralino mixing can be extracted from measurement of 8 1 Research in Theoretical Physics

the angular distribution of the b-jet in the top decay, which provides some knowledge on the Higgs mixing parameter µ. More recently Su considered the results of deep inelastic ν-(¯ν-) nucleus scattering, which can be 2 interpreted as a determination of the scale-dependence of sin θW , and imply a +3σ deviation from the predicted Standard Model value. Two new measurements of parity violating electron scattering at SLAC and elastic parity violating ep scattering at the Jefferson Lab could be used to determine 2 sin θW at a low-energy scale. Working with A. Kurylov and M.J. Ramsey-Musolf, Su analyzed the supersymmetric contributions to ee and ep parity violation processes. Supersymmetric effects on ν- (¯ν-)N scattering at NuTeV were also investigated[40], [41]. Working together with A. Dedes, S. Heinemeyer and G. Weiglein, Su studied the physics of the Higgs sector of the Minimal Supersymmetric Standard Model (MSSM) in the framework of the three most prominent soft SUSY-breaking scenarios [42]. They analyzed the observability of the light CP-even MSSM Higgs boson at the Tevatron, the LHC, a linear e+e− collider, a γγ collider and a µ+µ− collider. Parameter regions with suppressed production cross sections (compared to a SM Higgs boson with the same mass) were identified. For lepton and photon colliders, the impact of precision measurements of the Higgs branching ratios were explored. While the discovery of the lightest CP-even MSSM Higgs is almost certain at the LHC, detection of the heavy Higgs bosons H0, A0 and H± of MSSM poses a special challenge at future colliders. Even at the LHC, a wedge-shaped region of the MA −tan β parameter space at moderate tan β remains open, in which the heavy MSSM Higgs bosons will not be discovered. The associated production of a heavy Higgs with a light SM particle offers the possibility of producing the heavy Higgs at the e+e− collider with mass more than half the center-of-mass energy, when the dominant pair production process is kinematically forbidden. Su (with H. Logan) calculated the cross section for associated production of the charged Higgs boson and W ± gauge boson in high energy e+e− collisions in the MSSM framework [43]. With J. Gunion and T. Farris, Su studied the single heavy CP-odd A0 production process e+e− → ννA¯ 0 [44]. Xavier Calmet has been working on three different projects since he arrived at Caltech. The first project was on the time variation of the fundamental parameters of grand unified theories [45]. This was motivated by the astrophysical measurements indicating that the electromagnetic coupling parameter might be time dependent. He considered the time variation of fermion masses induced by a time variation of the grand unified scale. The second project was on electroweak symmetry breaking [46]. Motivated by the so-called little Higgs models that allow to shift the naturalness problem from 1 TeV to 10 TeV, Calmet reconsidered Veltman’s relation arguing that an accidental cancellation of the quadratically divergent corrections to the Higgs boson mass appearing at one loop, could also imply that the true scale for the naturalness problem is not 1 TeV but rather 10 TeV. The last project was on non-commutative field theories [47]. He developed Seiberg-Witten maps for a non-commutative theory described by a deformation parameter theta with is space-time dependent and calculated the leading order operators that take into account this particular nature of space-time. During the last year Michael Graesser has continued to be interested primarily in physics beyond the Standard Model (SM), and in particular, supersymmetric extensions. I have concerned myself with the related issues of supersymmetry breaking and the supersymmetric flavor problem within the context of brane world models, and also axions, within the context of supersymmetry and early cosmology. In [48], Graesser studied (with T. Banks and M. Dine) whether the conventional upper bound on the axion decay constant (from overclosure) could be relaxed in supersymmetric theories. In many supersymmetric extensions to the SM there are often scalars that decay after nucleosynthesis. This is a very severe cosmological problem, and must be addressed first before discussing the early cosmology of the QCD axion. Through surveying some models, they have found that solving this problem, by the late decay (but before nucleosynthesis) of a scalar, generically also allows for a much larger QCD decay constant. They point out though, that because in supersymmetry these scalars will appear with 1.2 Particle Phenomenology 9 pseduoscalar partners, the cosmology of these pseduoscalars may be problematic. In [49], Graesser and collaborators argue that the Randall-Sundrum model for obtaining an anomaly– mediated supersymmetry spectrum does not appear to be generic in string theory compactifications. John Preskill’s current research focuses on the interface of quantum field theory with quantum computation and statistical physics. Continuing work begun in [50], Preskill with graduate student J. Harrington and undergraduate student C. Wang investigated the properties of the Higgs-confinement phase transition in lattice gauge theories with quenched disorder [51]. Using a combination of analytic methods and Monte Carlo simulations, they mapped out the phase diagram of the Z2 model in three dimensions, establishing that confinement can be driven by magnetic disorder even at zero temperature (that is, without any quantum fluctuations of the magnetic field). In passing, they also studied the random-bond Ising model in two dimensions (which has very similar features), disproving the widely accepted conjecture that the disorder strength at the boundary between the ferromagnetic and para- magnetic phases is a temperature-independent constant at low temperature. The confinement-Higgs critical point in a gauge theory with quenched randomness can be mapped exactly to the accuracy threshold in a noisy quantum computer — the transition between a low noise phase in which robust quantum computation is possible, and a high noise phase in which decoherence inevitably causes a large-scale quantum computation to fail. In [51], this insight is exploited to obtain improved numerical estimates of the accuracy threshold. In another intriguing connection between quantum field theory and quantum computing, graduate student C. Mochon has studied how nonabelian anyons (particles in two dimensions obeying exotic quantum statistics) can be employed to carry out efficient quantum computations that are intrinsi- cally resistant to decoherence. He analyzed anyons that realize the quantum double of an arbitrary finite group G, showing that nonsolvability of the group is a sufficient criterion for universal quantum computation [54]. Another important topic in quantum information science is quantum cryptography, which unlike quantum computation is already realizable with existing technology. Preskill has studied quantum key distribution, proving its information theoretic security against arbitrary attacks by an eavesdropper. In [52], Preskill and M. Koashi show that security is uncompromised even if the source of quantum states used in the protocol is completely unreliable. In [53] Preskill with D. Gottesman, H.-K. Lo, and N. Lütkenhaus investigate the impact on security of sufficiently small flaws in both the source and the detector used, showing that secure key can still be extracted, and estimating the impact of the flaws on the rate of key generation. The crucial feature that distinguishes quantum information and classical information is quantum entanglement, the nonlocal correlation between the parts of a quantum system that has no classical analog. The task of distinguishing whether a bipartite mixed state is entangled or not is a hard problem in general, and until recently there were no efficient computational methods known that could detect the entanglement of an arbitrary state. Such a method was developed by graduate student F. Spedalieri [55]. His method establishes a complete hierarchy of tests for entanglement, each a semidefinite program that, if successful, constructs an “entanglement witness,” an observable that could in principle be measured to prove that the state exhibits quantum nonlocality. 10 1 Research in Theoretical Physics

Bibliography

[1] P. Berglund and A. Brandhuber, “Matter from G2 Manifolds”, Nucl. Phys. B 641, 351 (2002), [arXiv:hep-th/0205184].

[2] A. Brandhuber and K. Sfetsos, “PP-Waves from Rotating and Continuously Distributed D3- Branes”, JHEP 0212, 050 (2002), [arXiv:hep-th/0212056].

[3] A. Brandhuber, H. Ita, H. Nieder, Y. Oz and C. Romelsberger, “Chiral Rings, Superpotentials and the Vacuum Structure of N = 1 Supersymmetric Gauge Theories”, [arXiv:hep-th/0303001].

[4] M. Kontsevich, “Deformation quantization of Poisson manifolds, I”, [arXiv:q-alg/9709040].

[5] A.S. Cattaneo and G. Felder, “A path integral approach to the Kontsevich quantization formula,” Commun.Math.Phys. 212 (2000) 591-611. [arXiv:math.QA/9902090].

[6] I. Chepelev and C. Ciocarlie, “A Path Integral Approach To Noncommutative Superspace”, [arXiv:hep-th/0304118].

[7] Y. -K. E. Cheung, L. Friedel, and K. G. Savvidy “Closed Strings in Gravimagnetic Fields,” in preparation.

[8] Y. K. Cheung, Y. Oz and Z. Yin, “Families of N = 2 strings,” to appear in JHEP [arXiv:hep- th/0211147].

[9] J. M. Maldacena and H. Ooguri, “Strings in AdS(3) and the SL(2,R) WZW model. III: Correlation functions,” Phys. Rev. D 65, 106006 (2002) [arXiv:hep-th/0111180]; J. M. Maldacena, H. Ooguri and J. Son, “Strings in AdS(3) and the SL(2,R) WZW model. II: Euclidean black hole,” J. Math. Phys. 42, 2961 (2001) [arXiv:hep-th/0005183]; J. M. Maldacena and H. Ooguri, “Strings in AdS(3) and SL(2,R) WZW model. I,” J. Math. Phys. 42, 2929 (2001) [arXiv:hep-th/0001053].

[10] J. Gomis and H. Ooguri, “Penrose limit of N = 1 gauge theories,” Nucl. Phys. B 635, 106 (2002) [arXiv:hep-th/0202157].

[11] J. Gomis, L. Motl and A. Strominger, “pp-wave / CFT(2) duality,” JHEP 0211, 016 (2002) [arXiv:hep-th/0206166].

[12] J. Gomis, S. Moriyama and J. w. Park, “SYM description of SFT Hamiltonian in a pp-wave background,” [arXiv:hep-th/0210153].

[13] J. Gomis, S. Moriyama and J. w. Park, “Open+Closed String Field Theory From Gauge Fields,” To appear.

[14] K. Hori and A. Kapustin, “Worldsheet descriptions of wrapped NS ﬁve-branes,” JHEP 0211, 038 (2002) [arXiv:hep-th/0203147].

[15] V. Borokhov, A. Kapustin and X. k. Wu, “Topological disorder operators in three-dimensional conformal ﬁeld theory,” JHEP 0211, 049 (2002) [arXiv:hep-th/0206054].

[16] V. Borokhov, A. Kapustin and X. k. Wu, “Monopole operators and mirror symmetry in three dimensions,” JHEP 0212, 044 (2002) [arXiv:hep-th/0207074].

[17] A. Kapustin and Y. Li, “D-branes in Landau–Ginzburg models and algebraic geometry,” arXiv:hep- th/0210296. BIBLIOGRAPHY 11

[18] A. Kapustin and Y. Li, “Topological Correlators in Landau-Ginzburg Models with Boundaries,” [arXiv:hep-th/03105136]. [19] P. Lee, S. Moriyama and J. w. Park, “Cubic interactions in pp-wave light cone string field theory,” Phys.Rev.D66, 085021 (2002) [arXiv:hep-th/0206065]. [20] P. Lee, S. Moriyama and J. w. Park, “A note on cubic interactions in pp-wave light cone string field theory,” Phys. Rev. D 67, 086001 (2003) [arXiv:hep-th/0209011]. [21] Y. Okawa, “Open string states and D-brane tension from vacuum string field theory,” JHEP 0207, 003 (2002) [arXiv:hep-th/0204012]. [22] K. Kaminsky, Y. Okawa and H. Ooguri, “Quantum aspects of the Seiberg–Witten map in noncommutative Chern–Simons theory,” [arXiv:hep-th/0301133]. [23] H. Ooguri and C. Vafa, “Worldsheet derivation of a large N duality,” Nucl. Phys. B 641, 3 (2002) [arXiv:hep-th/0205297]. [24] H. Ooguri and C. Vafa, “The C-deformation of gluino and non-planar diagrams,” [arXiv:hep- th/0302109]. [25] H. Ooguri and C. Vafa, “Gravity induced C-deformation,” [arXiv:hep-th/0303063]. [26] P. Kraus, H. Ooguri and S. Shenker, “Inside the horizon with AdS/CFT,” [arXiv:hep-th/0212277]. [27] S. Kachru, M. B. Schulz, P. K. Tripathy and S. P. Trivedi, “New supersymmetric string compactifications,” JHEP 0303, 061 (2003) [arXiv:hep-th/0211182]. [28] S. Kachru, M. B. Schulz and S. Trivedi, “Moduli stabilization from fluxes in a simple IIB orientifold,” [arXiv:hep-th/0201028]. [29] J. H. Schwarz, “Comments on superstring interactions in a plane-wave background,” JHEP 0209, 058 (2002) [arXiv:hep-th/0208179]. [30] Y. H. He, J. H. Schwarz, M. Spradlin and A. Volovich, “Explicit formulas for Neumann coefficients in the plane-wave geometry,” Phys. Rev. D 67, 086005 (2003) [arXiv:hep-th/0211198]. [31] C. G. Callan, H. K. Lee, T. McLoughlin, J. H. Schwarz, I. Swanson, and X. Wu, “Curvature 5 Corrections to String Theory in AdS5 × S : Beyond the pp-Wave,” [32] M. Aganagic, K. Intriligator, C. Vafa and N. P. Warner, “The glueball superpotential,” [arXiv:hep- th/0304271]. [33] R. Dijkgraaf and C. Vafa, “N = 1 supersymmetry, deconstruction, and bosonic gauge theories,” [arXiv:hep-th/0302011]. [34] M. Aganagic, A. Klemm, M. Marino and C. Vafa, “The Topological Vertex,” [arXiv:hep- th/0305132]. [35] M. Ramsey-Musolf and M. Wise, “Hadronic Light by Light Contribution to Muon g-2 in chiral perturbation theory,” Phys. Rev. Lett 89, 041601 (2002) [arXiv:hep-ph/021297]. [36] A. Leibovich, Z. Ligeti and M. Wise, “Enhanced Subleading Structure Functions in Semileptonic B. Decay,”Phys. Lett. B539, 242 (2002) [arXiv:hep-ph/0205148]. [37] A. Leibovich, Z. Ligeti and M. Wise, “Comment on Quark Masses in SCET,” [arXiv:hep- ph/0303099]. 12 1 Research in Theoretical Physics

[38] C. Bauer and M. Wise, “Enhanced Nonperturbative Effects in Jet Physics,” [arXiv:hep- ph/0212255]. [39] T. Moroi, R. Kitano and S. Su, “Top-Squark Study at Future e+e− Linear Collider,”JHEP 0212, 011 (2002) [arXiv:hep-ph/0208149]. [40] A. Kurylov, M.J. Ramsey-Musolf, S. Su, “Supersymmetric Effects in Deep Inelastic Neutrino Nu- cleus Scattering,” [arXiv:hep-ph/0301208]. [41] A. Kurylov, M.J. Ramsey-Musolf, S. Su, “Probing Supersymmetry with Parity Violating Electron Scattering,” [arXiv:hep-ph/0303026]. [42] A. Dedes, S. Heinemeyer, S. Su and G. Weiglein, “ The Lightest Higgs Boson of MSUGRA MGMSB and MAMSB at Present and Future Colliders: Observability and Precision Analysis” [arXiv:hep- ph/0302174]. [43] H. Logan and S. Su, “Variation of the Cross-Section for e+e− → W +h− in the Minimal Supersym- metric Standard Model,” Phys. Rev. D67017703 (2003) [arXiv:hep-ph/0206135]. [44] T. Farris, J. Gunion, H. Logan and S. Su, “e+e− → ννA¯ 0 in the Two-Higgs Doublet Model,” [arXiv:hep-ph/0302266]. [45] X. Calmet and H. Fritzsch, “Grand Unification and Time Variation of Gauge Couplings ,” Pro- ceedings of the 10th International Conference on Supersymmetry and Unification of Fundamental Interactions (SUSY02), [arXiv:hep-ph/021142]. [46] X. Calmet, “Softening the Naturalness Problem,” [arXiv:hep-ph/0302056]. [47] X. Calmet and M. Wohlgenannt, “Effective Field Theories on Non-Commutative Space-Time,” [arXiv:hep-ph/0305027]. [48] T. Banks, M. Dine, and M. Graesser “Supersymmetry, Axions and Cosmology, ” [arXiv:hep- ph/0210256]. [49] A. Anisimov, M. Dine, M. Graesser, S. Thomas “Brane World Susy Breaking from String/M– theory, ” JHEP 0203:036,2002. [arXiv:hep-th/0201256]. [50] D. Beckman, D. Gottesman, A. Kitaev, and J. Preskill, “ Topological quantum memory,” J. Math. Phys., 43, 4452-4505 (2002) 4452-4505. [arXiv: quant-ph/0110143]. [51] C. Wang, J. Harrington, and J. Preskill, “Confinement-Higgs transition in a disordered gauge theory and the accuracy threshold for quantum memory,” Annals Phys. 303 065022 (2003) [arXiv:quant- ph/0207088]. [52] M. Koashi and J. Preskill, “Secure quantum key distribution with an uncharacterized source, ” Phys.Rev.Lett.90, 057902 (2003) [arXiv:quant-ph/0208155]. [53] D. Gottesman, H.-K. Lo, N. Lütkenhaus, and J. Preskill, “Security of quantum key distribution with imperfect devices, ” [arXiv:quant-ph/0212066]. [54] C. Mochon “Anyons from non-solvable discrete groups are sufficient for universal quantum computation,” Phys. Rev. A 67, 022315 (2003) [arXiv:quant-ph/0206128]. [55] F. Spedalieri, “Characterizing entanglement in quantum information, ” Caltech Ph.D. thesis, June 2003, 108 pages. 2. Theoretical Particle Astrophysics

M. Kamionkowski

This Task was started when Marc Kamionkowski arrived in 1999 as Professor of Theoretical Physics and Astrophysics, bringing strength in theoretical particle astrophysics and early-Universe cosmology. This Task forges ties between Caltech’s particle-physics research and our substantial activities in experimental and observational particle astrophysics and cosmology. Cosmology is now in the midst of its most exciting decade ever, and Caltech experimentalists are at the forefront. Caltech’s BOOMERanG (PI: A. Lange) was the first to see multiple acoustic peaks in the CMB power spectrum, demonstrating the flatness of the Universe (using a technique first proposed by Prof. Kamionkowski and collaborators) and verifying a spectrum of primordial density perturbations remarkably like those predicted by inflation. A new Caltech experiment (BICEP) will soon begin looking for the inflationary signature in the CMB polarization first predicted by Prof. Kamionkowski and collaborators. Complementary CMB research is also being pursued by Prof. Readhead’s group (the CBI experiment), and Caltech/JPL will lead the US effort in the Planck satellite. Prof. R. Ellis is leading the weak-lensing aspects of the SNAP satellite, and Prof. Sunil Golwala, a new addition to our experimental junior faculty, plans to pursue direct searches for both particle dark matter and novel probes of dark energy. Prof. McKeown is beginning a promising initiative to study ultra-high-energy cosmic rays. There is then a plethora of related observational cosmology being done in optical/IR astronomy by Profs. Djorgovski, Sargent, Cohen, Steidel, Soiffer, Scoville, and Blain at Caltech’s Keck and Palomar telescopes and with space missions (e.g., SIRTF and GALEX) in which Caltech plays a leadership role. The interpretation of this flood of data, as well as the groundwork for future steps, will be a joint enterprise involving theorists, experimentalists, and data analysts. With Kamionkowski, Caltech is positioned to match its experimental firepower with strong theoretical leadership. This synergy will allow Caltech, and the DoE, to reap huge scientific payoffs. Much of the activity associated with this Task has been nucleated with a generous startup package from Caltech. These startup funds are now winding down, so we are requesting this year an increase in the budget for this Task to sustain the group’s prior level of activity. A budget justification is discussed in a separate section below.

2.1 Progress Report for May 2002–April 2003

The primary budget items charged to this Task during this time were partial support (May-August) for Dr. Kenneth Nollett, a postdoc in particle and nuclear astrophysics; partial support (shared with particle and nuclear theory) for a new postdoc, Dr. Andriy Kurylov, in particle/nuclear theory and particle astrophysics; research expenses for Dr. Asantha Cooray, a senior research fellow (the equivalent of a research assistant professor) in theoretical cosmology; and summer salary for Prof. Kamionkowski. Research supported by this Task has produced 33 refereed papers that have appeared and/or been submitted for publication in refereed journals during the past year, and 10 contributions to conference proceedings. Research was carried out on novel dark-energy and dark-matter models [1, 12]; gravitational microlensing [3, 6]; the ﬁrst stars [4, 14]; weak gravitational lensing and dark energy [10, 23, 35]; the CMB and inﬂation [7, 8, 9, 10, 11, 13, 24, 27, 28, 29, 30, 31, 32, 36, 37, 38]; large-scale structure and

13 14 2 Theoretical Particle Astrophysics

galaxy formation [25, 26, 33, 34]; big-bang and stellar nucleosynthesis [21, 22]; and low-energy neutrino scattering, deep-inelastic neutrino scattering, and supersymmetry [39, 40, 41, 42, 43]. A popular article by Kamionkowski on detection of inflationary gravitational waves was reprinted in a special edition of Scientific American [20]. Cooray wrote a review article for Physics Reports [26] on the halo approach to galaxy clustering. Rather than summarize all this work here, we review some highlights: A CMB excess and the first stars. Oh, Cooray, and Kamionkowski [14] showed that an excess of small-scale CMB power detected by Caltech’s CBI experiment may be due to the first stars in the Universe. Theorists had before surmised that the excess was due to unresolved galaxy clusters, but the required number of clusters was significantly higher than the inflationary prediction. Our work showed that the early star formation suggested by the WMAP large-angle CMB polarization requires a large number of supernovae that would inject a huge amount of energy into the CMB. Clustering of these supernovae would yield all or part of the CBI small-scale excess. Our explanation predicts that there should remain some unresolved small-scale CMB fluctuations with higher-resolution higher-sensitivity maps. Weak gravitational lensing and dark energy. Several weak-gravitational-lensing surveys in blank regions of the sky have turned up lensing patterns that suggest the presence of a galaxy-cluster–mass object, but with none of the x-ray emission usually seen in such clusters. Kamionkowski and Wein- berg [5] showed that these lenses could be understood as cluster-mass overdensities that have not yet fully collapsed and virialized. Their subsequent work [10] showed that the abundance of these x-ray- underluminous objects, as well as their redshift distribution, depends on the expansion history, and thus on the dark-energy equation of state. They estimated that future lensing surveys, such as those carried out by SNAP or LSST, would find a sufficient number of these objects to begin to place important constraints to the dark-energy equation-of-state parameter w. More detailed predictions for the abundances as well as algorithms to find these objects still need to be developed before the test can be applied. CMB polarization and dark energy. Based upon an earlier suggestion by Kamionkowski and Loeb (1998), Cooray [30, 35] investigated the possibility to use the CMB polarization observed toward galaxy clusters to contrain the expansion history, and thus w. The idea is that when the cluster electrons scatter CMB radiation, the CMB quadrupole moment incident on the cluster gets converted to polarization. Thus, by measuring the polarization toward numerous clusters at a range of redshifts, the temporal variation of the CMB quadrupole can be mapped. Since this temporal variation depends on the expansion history through the integrated-Sachs-Wolfe effect, there is a cluster-polarization dependence on w. Cooray’s work shows that this novel probe of w may be possible with CMBPOL, an ultra-sensitive CMB polarization experiment that appears in NASA’s roadmap. Large-scale structure. Cooray (with Sheth) also completed during the past year a 129-page review article in Physics Reports [26] on the halo approach to galaxy clustering. One of the aims of cosmology today is to determine the primordial power spectrum P (k) of the mass distribution, as the details of this power spectrum reflect features of the potential of the inflaton, the scalar field that drives inflation. Another aim is to look for evidence for non-Gaussian perturbations, which should be small, but nonzero, in many models of inflation. A variety of techniques aim to determine the power spectrum by measuring the distribution of mass (usually inferred from the galaxy distribution) in the Universe today, and then applying cosmological dynamics to reconstruct the primordial distribution. All of these techniques require a detailed understanding of how gravitational infall affects galaxy clustering. On large scales these dynamics can be understood through linear perturbation theory, while on small scales they are most often understood through numerical simulations. In recent years, a powerful analytic model, calibrated to numerical simulations, has been developed. This model allows an easily implemented algorithm for determining clustering properties of mass and of galaxies over a wide variety of distance scales for a variety of cosmological parameters and initial conditions. In their article, Cooray 2.1 Progress Report for May 2002–April 2003 15 and Sheth provided a detailed review of this halo-clustering formalism, and applied it to describe the galaxy distribution, large-scale velocity and pressure fields, weak gravitational lensing, and low-redshift contributions to CMB temperature fluctuations. Physics of dark energy. The past few years have seen a rapid rise of interest among a number of particle theorists in the possibility that the dark energy may be phantom energy, in which the dark- energy equation-of-state parameter w<−1, giving rise to a super-accelerated expansion. Kamionkowski (with Robert Caldwell and a student, Nevin Weinberg) wrote a short paper about the cosmological consequences of such an equation of state, showing that one possibility is that the Universe ends in a “Big Rip”, in which the Universe goes to infinite expansion in finite time, ripping everything in the Universe apart as it does so. Quite amusingly, this paper received considerable attention in the popular press (e.g., CNN, MSNBC, Science News, New Scientist...), including a Los Angeles Times editorial that concluded with a strong endorsement of SNAP! Other activities. During the past year, Kamionkowski rotated off NASA’s SEU Subcommittee and began serving on the HEPAP P5 Subcommittee and the External Advisory Board for the NSF Center for Cosmological Physics (Chicago), and will soon begin on an advisory board for VERITAS. He stepped down as a receiving editor for JHEP and began as a receiving editor for the new online Journal of Cosmology and Astroparticle Physics; he continues to serve as Astrophysics Editor for Physics Reports. During the past year, Kamionkowski gave plenary talks at the ICHEP conference (Amsterdam), the 2002 SLAC summer school, an NLC workshop, Cosmo ’02, the Cozumel galaxy-formation workshop, and a Carnegie centennial symposium, and he chaired a session on dark matter at an NAS Frontiers of Science symposium and at the annual NAS meeting; these talks were on the CMB, cosmological parameters, particle dark matter, galaxy formation, and dark energy. Next fall, Kenneth Nollett will begin a permanent research position in the Argonne nuclear theory group. Eric Agol, another postdoc advised by the PI, will move to a junior-faculty position at the University of Washington; another (Peng Oh) will begin a junior-faculty position at UC Santa Barbara; and another (Andrew Benson) will begin a Royal Society Fellowship in Cambridge. Michael Santos, a student completing a PhD under Kamionkowski’s supervision, declined postdoc offers at Fermilab, CITA, and the IAS and accepted an NSF international postdoctoral fellowship to be taken to Cambridge.

2.1.1 Earlier work by Kamionkowski and this group

Before joining the faculty at Caltech, Kamionkowski carried out research on supersymmetric dark matter, the CMB and inflation, neutrino and nuclear physics, gravitational microlensing, and phase transitions in the early Universe. His work on dark matter has been important for direct searches (e.g., by CDMS, ZEPLIN...), energetic-neutrino searches (e.g., in AMANDA, Kamiokande, MACRO, IceCube), cosmic-ray antimatter searches (e.g., by AMS), and high-energy gamma-ray observatories (e.g., STACEE, GLAST, and VERITAS). He and collaborators were the first to propose that CMB maps could determine the universal geometry and cosmological parameters, as a suite of recent experiments have now done. Kamionkowski and collaborators also proposed a unique signature of inflationary gravitational waves in the CMB polarization, a new “holy grail” of cosmology and the motivation for one of the Einstein vision missions in NASA’s new roadmap. He also presented an influential argument that quantum gravity should violate global symmetries, with specific application to the Peccei-Quinn solution to the strong-CP problem. Kamionkowski carried out the current state-of-the-art calculation of the proton-proton reaction that initiates nuclear fusion in stars, and he also did a detailed calculation of the neutrino-electron elastic-scattering reaction important for solar-neutrino detection. At Caltech, Kamionkowski and his group have been working more on these topics, as well as on galaxy and structure formation, dark energy, and a few other topics. Here are some highlights of this research (prior to the past year): (a) Kamionkowski and Liddle pointed out that the dearth of power 16 2 Theoretical Particle Astrophysics on subgalactic scales may be indicating a nontrivial feature in the inflaton potential. (b) Another paper pointed out that the sensitivity of a CMB-polarization experiment to inflationary gravitational waves may be increased significantly if a long integration is performed on a small region of sky; this was the motivation for the Caltech/JPL BICEP experiment, a recently funded CMB polarimeter that will begin to probe the inflationary parameter space from the South Pole. (c) Kamionkowski and collaborators pointed out a possible intrinsic alignment of galaxies that might serve as a contaminating background for weak-lensing surveys; a few months after the proposal, an Edinburgh group announced observational evidence for the proposed intrinsic correlations. (d) Kamionkowski and collaborators showed that if the WIMP had spin-dependent interactions, it could account for the DAMA annual modulation and evade detection in germanium and xenon detectors; they also showed, however, that this explanation would then be ruled out by null searches for energetic neutrinos from the Sun. (e) Kamionkowski and collaborators proposed a spinning-field model for dark energy and/or dark matter that introduces novel physics and new possibilities for cosmological perturbations; although it now seems difficult to work into a realistic dark-energy model, it still provides important corrections to ultra-low-mass–boson models for dark matter. (f) Finally, Kesden, Cooray, and Kamionkowski studied how the CMB curl component due to inflationary gravitational waves can be disentangled from that due to gravitational lensing. This work is playing a central role in the design of mission concepts for NASA’s CMBPOL experiment. In addition to the postdocs and students mentioned above, Piero Ullio, a postdoc supported by this DoE Task during the first year of funding, moved to a tenure-track position in high-energy theory at SISSA. One of Kamionkowski’s students from Columbia University (A. Refregier) entered a permanent CNRS research position in France, another (C. Cress) entered a tenure-track position in South Africa, and another (X. Chen) is now a postdoc at the KITP.

2.2 Plans for Future Research

We anticipate that in the future, this group will continue to carry out research at the interface of particle physics and cosmology and astrophysics. This will include research on particle dark matter, dark energy, the CMB, neutrino physics and astrophysics, large-scale-structure, and the early Universe, and other tests of new physics beyond the standard model. Given the nature of this theoretical research, it is difficult to anticipate in detail what we will study on a 3–5 year timescale; this will be determined ultimately by experimental and theoretical developments here and elsewhere. Instead, we briefly describe below some of the problems we plan to concentrate on during the forthcoming year. These will likely then serve as springboards for subsequent research. The CMB and inflation. Kamionkowski and Cooray are beginning to work closely with Caltech/JPL experimentalists to formulate the experimental requirements of a post-Planck CMB polarization experiment aimed at detection of inflationary gravitational waves. We will study the optimal experimental strategies for pursuing this goal as well as other goals, such as gravitational lensing and neutrino-mass and dark-energy probes, and cosmological-parameter determination. We will investigate the optimal angular resolution, sensitivity, and sky coverage and whether these new experiments will require new data-analysis techniques. We will study the frequency coverage that will be required to subtract Galactic foregrounds. Variable α and the CMB. Kurylov, Kamionkowski, and a student (Sigurdson) are currently investigating consequences of spatial variation of the fine-structure constant α for cosmology. Observational hints for time variation of the fine-structure constant have led to renewed theoretical investigations of models with variable α, including several that point out a possible connection with scalar-field models for dark energy. If α can vary with time, it might also vary in space. We are thus investigating the 2.2 Plans for Future Research 17 constraints to such spatial variations that can be provided by the CMB. Spatial variations to α will change the CMB power spectrum, induce a curl component in the CMB polarization, and also induce higher-order temperature/polarization correlations. We are constructing a relativistic field-theory model for variable α that may induce such effects. Looking more broadly, spatial variations in α are described by a new type of cosmological perturbation (in addition to the adiabatic and isocurvature perturbations usually considered) in which the initial densities and entropies are constant, but reaction rates may vary with time. Generalized WIMP interactions. Kurylov and Kamionkowski plan to carry out a generalized analysis of WIMP interactions. The most commonly studied WIMP is the lightest supersymmetric particle (LSP), a Majorana fermion. In such models, the Lagrangian for WIMP-nucleon interactions is specified, and the direct- and indirect-detection rates then calculated from this Lagrangian. Although this approach covers a broad range of well-motivated WIMP candidates, it excludes other possibilities that may arise in other (e.g., extra-dimensional) models for new physics at the electroweak scale or in other theories that have not yet been anticipated. We will carry out an analysis of WIMP-nucleon interactions that will be general for any pointlike WIMP of arbitrary spin, for Dirac as well as Majorana fermions, and for WIMPs for which there may be a particle-antiparticle asymmetry. We plan to search the resulting parameter space for models that may be consistent with both the DAMA annual modulation as well as the null results from other direct and indirect searches. We plan to present our results in a way that may be easily adapted by WIMP model-builders, and we plan to develop front-end software for neutdriver and DARKSUSY that will allow these two publicly-available SUSY dark-matter codes to handle a broader variety of WIMP models. The largest-scale structure. We also plan to explore the possibility to study empirically the myste- rious and nagging deficiency of large-angle correlations in the CMB. Rather than speculate about what super-horizon physics might produce such a deficiency, we plan to investigate whether the CMB results might be confirmed or disproved empirically by gravitational lensing of the CMB. We believe that this may be the only alternative observational probe of the matter power spectrum on such large scales. Decaying charged particles and the CMB. We also plan to investigate cosmological constraints to some gravitino dark-matter scenarios. If the gravitino is the LSP, it may be produced by decays of the NLSP, the next-to-lightest SUSY particle. The NLSP would be produced as a thermal relic in the early Universe, and would thus have a cosmological density in the early Universe close to the critical density. After some time (but well before the present epoch), however, it would decay to the gravitino which would then be the dark matter today. Since the NLSP is not the dark matter today, there is no reason why it could not be charged; e.g., a chargino or slepton. There has been renewed interest in this scenario recently and attention paid to cosmological constraints to the parameter space consisting of the gravitino and NLSP masses and the decay lifetime. We plan during the next year to work out a new CMB constraint that has not yet been studied: If the NLSP is charged, it will oscillate with the baryon-photon fluid in the early Universe and thus affect the structure of the acoustic peaks in the CMB. With the precision of current and forthcoming measurements of the CMB power spectrum, an NLSP with a lifetime comparable to ∼ 104−6 years should produce detectable effects in the CMB power spectrum. We plan to quantify the constraints much more precisely. The results should be more generally applicable to other scenarios that involve decaying particles. Dark-matter spike at the Galactic center. Gondolo and Silk recently proposed that the supermassive black hole at the Galactic center may be surrounded by an extraordinarily dense distribution of dark matter. If so, and if the dark matter is composed of neutralinos, then there should have been a huge flux of radiation from neutralino annihilation observed toward the Galactic center. We plan during the next year to study the feasibility of using forthcoming measurements of stellar orbits very close to the black hole to test or constrain this hypothesis. Although the stellar orbits measured so far are not yet constraining, our estimates show that the Gondolo-Silk spike may have observable consequences for 18 2 Theoretical Particle Astrophysics

future observations. We plan to calculate the orbits that should arise in the presence of a dark-matter spike, and to find techniques for disentangling these from other more conventional effects that may similarly affect the orbits. Galactic-halo merger formalism. In inflation-inspired structure-formation models, small structures form first and then subsequently merge to form higher-mass objects. Quite remarkably, there are still fundamental flaws in our understanding of this seemingly simple problem, and there is still no self- consistent analytic formula for the merger rate. Moreover, the merger rate is extremely difficult to evaluate in numerical simulations, especially for the merger of two objects of very unequal masses. An improved understanding of the merger process is imperative to understand galaxy formation and thus infer the primordial distribution of mass, and to predict, for example, the mergers of supermassive black holes that the LISA satellite hopes to see through gravitational waves. The mathematically consistent description involves the Smoluchowski coagulation equation, which describes such merger processes. The problem, however, is that we do not know the merger kernel (this is usually given in coagulation problems in statistical physics, chemistry, biology, and elsewhere). We have recently come across an algorithm that may provide the merger kernel, given the Press-Schechter distribution for halo masses that the solution to the coagulation equation must give. We will study this algorithm to see if it can provide a numerical solution for the merger rate, or shed some light on an analytic solution.

2.3 Budget justiﬁcation

New scientific opportunities in non-accelerator physics, particle astrophysics, and cosmology are rapidly shooting out in a number of directions. As the DoE moves toward developing a broader experimental portfolio in these areas, it will become increasingly important to support the research of theoretical groups with broad activity in a variety of these emerging areas. So far, Kamionkowski’s research program at Caltech has been enabled largely by a generous startup package from Caltech that is now running down. We are thus requesting from the DoE funding to sustain into the future the level of activity this group has maintained the past few years. The requested level of funding will be required to carry out research in the areas described above. The funding will support one graduate student to work primarily on particle dark matter and/or dark energy and another to work primarily on the CMB tests of inflation. Caltech theory students can usually find fellowship support their first few years; thus DoE funding will be used to fund only advanced students, when they are already experienced and most productive. The postdoc will choose topics in particle astrophysics and cosmology of interest to him/her—most likely dark energy, large-scale structure, the CMB, and/or inflation. The education and professional development of a talented group of graduate students and postdocs will be another “deliverable” of the proposed research. The total budget requested is as follows:

Postdoc @ $45,000 (salary and travel, fringe, and ICR) $100,000 Graduate Student 1 (salary, tuition, ICR, travel) $50,000 Graduate Student 2 (salary, tuition, ICR, travel) $50,000 Kamionkowski (summer salary, travel, fringe, and ICR) $50,000 TOTAL $250,000

In summary, the proposed funding will foster research in theoretical particle astrophysics and cosmology under Kamionkowski’s guidance at Caltech. This research will provide a theoretical basis that 2.4 Additional support for related projects 19 will support the DoE’s growing investment in major initiatives at the interface of particle physics and cosmology/astrophysics; e.g., SNAP, CDMS, IceCube, GLAST, VERITAS, AMS, etc., as well as parallel projects by the NSF and NASA with similar scientiﬁc goals. Finally, past experience has shown that theoretical research along these lines may well lead to novel ideas, which we cannot anticipate now, for subsequent generations of experimental investigations.

2.4 Additional support for related projects

The following awards are current:

• NASA: Astrophysics Theory Program, “Cosmology and Fundamental Physics From Space,” 4/15/02– 4/14/05, $234,000. No salary for PI.

Kamionkowski has no pending awards. 20 2 Theoretical Particle Astrophysics

Bibliography

[1] “Spintessence! New Models for Dark Matter and Dark Energy,” L. A. Boyle, R. R. Caldwell, and M. Kamionkowski, Phys., Lett. B 545, 17 (2002). [2] “Theoretical Estimates of Intrinsic Galaxy Alignment,” J. Mackey, M. White, and M. Kamionkowski, MNRAS 332, 788 (2002). [3] “X-rays from Isolated Black Holes in the Milky Way,” E. Agol and M. Kamionkowski, MNRAS 334, 553 (2002). [4] “The Contribution of the First Stars to the Cosmic Infrared Background,” M. R. Santos, V. Bromm, and M. Kamionkowski, MNRAS 336, 1082 (2002); [5] “Weak Gravitational Lensing by Dark Clusters,” N. N. Weinberg and M. Kamionkowski, MNRAS 337, 1269 (2002). [6] “Finding Black Holes with Microlensing,” E. Agol, M. Kamionkowski, L. Koopmans, and R. D. Blandford, ApJ Lett. 576, L131 (2002). [7] “Weak Lensing of the CMB: Cumulants of the Probability Distribution Function,” M. Kesden, A. Cooray, and M. Kamionkowski, Phys.Rev.D66, 083007 (2002). [8] “Small-Scale Cosmic Microwave Background Polarization from Reionization,” D. Baumann, A. Cooray, and M. Kamionkowski, astro-ph/0208511. To appear in New Astronomy. [9] “Aspects of the Cosmic Microwave Background Dipole,” M. Kamionkowski and L. Knox, Phys. Rev. D 67, 063001 (2003). [10] “Constraining Dark Energy with the Abundance of Weak Gravitational Lenses,” N. N. Weinberg and M. Kamionkowski, MNRAS 341, 251–262 (2003). [11] “A New Window to the Early Universe,” E. Hivon and M. Kamionkowski, Science 298, 1349–1350 (2002). [12] “Phantom Energy and Cosmic Doomsday,” R. R. Caldwell, M. Kamionkowski, and N. N. Weinberg, astro-ph/0302506. [13] “Lensing Reconstruction with CMB Temperature and Polarization,” M. Kesden, A. Cooray, and M. Kamionkowski, astro-ph/0302536. To appear in Phys.Rev.D. [14] “Sunyaev-Zeldovich Fluctuations from the First Stars?” S.-P. Oh, A. Cooray, and M. Kamionkowski, astro-ph/0303007. To appear in MNRAS Lett. [15] “Particle Astrophysics and Cosmology: Cosmic Laboratories for New Physics (Summary of the Snowmass 2001 P4 Working Group),” Daniel S. Akerib, Sean M. Carroll, Marc Kamionkowski, and Steven Ritz, in “Snowmass 2001: The Future of Particle Physics,” edited by N. Graf, SLAC eConf C010630, P4001 (2002) [hep-ph/0201178]. [16] “Inﬂation at the Edge,” Marc Kamionkowski, astro-ph/0209273. To appear in “Galaxy Evolution: Theory and Observations,” proceedings of the conference, Cozumel, Mexico, April 8–12, 2002, edited by V. Avila-Reese, C. Firmani, C. Frenk, and C. Allen, RevMexAA SC (2002). [17] “New Views of Cosmology and the Microworld,” Marc Kamionkowski, in “Secrets of the B meson,” proceedings of the XXXth SLAC Summer Institute, August 5–16, 2002 (SSI02), edited by J. Hewett, J. Jaros, T. Kamae, and C. Prescott, eConf C020805, TF04 (2002) [hep-ph/0210370]. BIBLIOGRAPHY 21

[18] “Cosmology and Dark Matter,” Marc Kamionkowski, to appear in proceedings of ICHEP02, 31st International Conference on High Energy Physics, Amsterdam, July 24–31, 2002. [19] “Cosmology and Dark Matter,” Marc Kamionkowski, in proceedings of ICHEP02, 31st Interna- tional Conference on High Energy Physics, Amsterdam, July 24–31, 2002, [Nucl. Phys. B 117, 335–352 (2003)]. [20] “Gravitational Echoes from the Big Bang,” Robert R. Caldwell and Marc Kamionkowski, Scientific American January 2001, 38–43 (2001). Updated and reprinted in a “The Once and Future Cosmos,” a special edition of Scientific American, October 2002. [21] “Primordial Nucleosynthesis with a Varying Fine Structure Constant: An Improved Estimate,” Kenneth M. Nollett and Robert E. Lopez, Phys.Rev.D66, 063507 (2002). [22] “Cool bottom processes on the thermally-pulsing AGB and the isotopic composition of circumstellar dust grains,” K. M. Nollett, M. Busso, and G. J. Wasserburg, Astrophys. J. 582, 1036 (2003). [23] “Second-Order Corrections to Weak Lensing by Large-Scale Structure,” A. Cooray and W. Hu, Astrophys. J. 574, 19 (2002). [24] “Kinetic Sunyaev-Zeldovich Effect from Halo Rotation,” A. Cooray and X. Chen, Astrophys. J. 573, 43 (2002). [25] “Second Moment of Halo Occupation Number,” A. Cooray, Astrophys. J. 576, L105 (2002). [26] “Halo Models of Large Scale Structure,” A. Cooray and R. Sheth, Phys. Rept. 372, 1 (2002). [27] “Cosmic Microwave Background Temperature at Galaxy Clusters,” E. S. Battistelli, M. DePetris, L. Lamagna, F. Melchiorri, E. Palladino, G. Savini, A. Cooray, A. Melchiorri, Y. Rephaeli, and M. Shimon, Astrophys. J. 508, L101 (2002). [28] “Small angular scale CMB anisotropies from CBI and BIMA experiments: Early universe or local structures?,” A. Cooray, A. Melchiorri, Phys.Rev.D66, 083001 (2002). [29] “Lensing reconstruction of primordial cosmic microwave background polarization,” A. Cooray, Phys.Rev.D.66, 103509 (2002). [30] “CMB Polarization towards Clusters as a Probe of the Integrated Sachs-Wolfe Effect,” A. Cooray and D. Baumann, Phys. Rev. D 67, 063505 (2003). [31] “Weak lensing of the CMB: extraction of lensing information from the trispectrum,” A. Cooray and M. Kesden, New Astron. 8, 231 (2003). [32] “Is the Cosmic Microwave Background Circularly Polarized?,” A. Cooray, A. Melchiorri, and J. Silk, Phys. Lett. B 554, 1 (2003). [33] “Cosmic Microwave Background Temperature Evolution by Sunyaev-Zel’dovich effect observations,” E. S. Battistelli, M. DePetris, L. Lamagna, F. Melchiorri, E. Palladino, G. Savini, A. Cooray, A. Melchiorri, Y. Rephaeli, M. Shimon, Memorri della Societa Astronomica Italiana 74, 316 (2002). [34] “The Far-Infrared Background Correlation with CMB Lensing,” Y.-S. Song, A. Cooray, L. Knox, and M. Zaldarriaga, Astrophys. J., in press, astro-ph/0209011. [35] “Growth Rate of Large Scale Structure as a Powerful Probe of Dark Energy,” A. Cooray, D. Huterer, and D. Baumann, Phys. Rev. Lett., submitted, astro-ph/0304268. 22 2 Theoretical Particle Astrophysics

[36] “After MAP: Next Generation CMB,” A. Cooray, in Cosmology & Galaxy Formation, 5th Beijing Workshop in Cosmology, eds. Y. P. Jing, Beijing.

[37] “After Acoustic Peaks: The Next CMB Frontier,” A. Cooray, American Astronomical Society Meeting, BAAS, 201 140.

[38] “CMB-induced Cluster Polarization as a Cosmological Probe,” D. Baumann and A. Cooray, in proceedings of the CMBnet meeting, Oxford, February 2003, New Astron. in press, astro-ph/0304416.

[39] “Charged Current Universality and the MSSM,” A. Kurylov and M. J. Ramsey-Musolf, to appear in the Proceedings of 2002 annual APS meeting, division of nuclear physics, Albuquerque. [40] “Radiative Corrections to Low-Energy Neutrino Reactions,” A. Kurylov, M. J. Ramsey-Musolf, and P. Vogel, Phys. Rev. C67, 035502 (2003).

[41] “Supersymmetric Eﬀects in Deep Inelastic Neutrino-Nucleus Scattering,” A. Kurylov, M. J. Ramsey-Musolf, and S. Su, hep-ph/0301208.

[42] “The Weak Charge of the Proton and New Physics,” Jens Erler, Andriy Kurylov, and Michael J. Ramsey-Musolf, hep-ph/0302149.

[43] “Probing Supersymmetry with Parity Violating Electron Scattering,” A. Kurylov, M. J. Ramsey- Musolf, and S. Su, hep-ph/0303026. Part II

Experimental Program

3. The Experimental Program

3.1 Introduction

The experimental High Energy Physics group at Caltech has compiled a strong record of leadership on the international scene, playing leadership roles in BABAR, CMS and MINOS. This record of accomplishment has been made possible by an excellent faculty and strong support from DOE and Caltech’s Division of Physics, Mathematics and Astronomy. Activities in astrophysics and fundamental particle physics have traditionally had high priority at the Institute. We have received assistance from the Division in the form of faculty appointments, support for outstanding postdoctoral fellows, grants for development or the purchase of equipment, start-up funds for new faculty, funds for visitors, flexible teaching and leave arrangements, substantial support for computing, renovation of facilities and forward financing for equipment projects. Our HEP experimental program is made up of a group of six professorial faculty, a strong postdoctoral and senior research staff and a group of excellent graduate students. We are well advanced in searching for one or more junior faculty members to join the experimental effort. The MACRO, L3 and CLEO experiments have come to an end after long and very productive lives. Our experimental activities now center on the physics program of BABAR (Hitlin and Porter), finishing analyses on L3 (Newman) and CLEO (Weinstein) and preparing for CMS (Newman, Weinstein) and MINOS (Barish, Newman, Peck). In this chapter, we describe the mode of operation of the Caltech HEP group, as well as a brief history of our experimental program. We then present an overview of our physics program, comment on the staffing situation, budget, technical infrastructure and other issues.

3.2 History of the Caltech HEP Group

The development of experimental high energy physics at Caltech closely parallels that of the whole field of experimental particle physics. The experimental high energy physics program at Caltech began more than thirty years ago with the construction of the Caltech Electron Synchrotron, built and used by R. Bacher, M. Sands, R. Walker and A. Tollestrup. As the field developed, the quest for higher energies mandated larger accelerators at national laboratories shared by many different university and laboratory groups. The experimental effort at Caltech, like that at other Universities, evolved into “user group activities” at the national facilities at Brookhaven, Fermilab, and SLAC, as well as at CERN and Gran Sasso. While the Caltech synchrotron was closed in the mid-60’s, the infrastructure, building, etc., have continued to serve as a technical center for staging experiments conducted elsewhere. This technical infrastructure has, however, over the years been substantially reduced due to ongoing budget pressures. This deterioration of university infrastructure was highlighted by the Gilman HEPAP Subpanel. Our response to these pressures has been to shift a portion of our technical focus to innovative developments in computing (online and offline) and networking, although we still retain substantial technical capabilities. We are, for example, building one half of the scintillation detectors for MINOS. The Caltech effort has been extraordinarily successful over the last three decades, with the Caltech experimental group playing major roles in many important experiments. At Fermilab, the Caltech group (Fox, Gomez and Pine) led experiments on multiparticle production using the Fermilab MPS, which

25 26 3 The Experimental Program

made the first observation of quark-quark scattering in hadronic processes. An experiment by Tollestrup and Walker on charge exchange processes at high energy clearly showed how the quark substructure of the proton becomes visible as one probes shorter and shorter distances. The neutrino experiments of Barish and Sciulli made a particularly important impact on the development of the unified theory of weak and electromagnetic interactions introduced by Weinberg and Salam. Following these experiments, our emphasis shifted to e+e− collisions, where results from the Crystal Ball (Peck and Porter) were extremely important in understanding radiative transitions among states involving charmed quarks. The Mark III (Hitlin) provided impressive quantitative results on both J/ψ and D physics in the rich SPEAR energy region. DELCO (Barish) at PEP enlarged our understanding of the τ lepton. The Mark J (Newman) at DESY also made important contributions to our understanding of gluons, electroweak interference and heavy quarks. Our program then developed into involvement in both the upgraded Mark II (Barish, Peck, Porter and Weinstein) and the SLD (Hitlin) for the SLC. The Mark II made important early contributions to Z0 physics, while SLD was able to exploit the polarized electron beam of SLC to make the single most precise measurement of ALR as well as important results in B physics. The L3 experiment at LEP was extraordinarily successful in its precision measurements of the electroweak sector, including the Z0 and W masses and electroweak couplings. This data also has shown that if the standard model Higgs exists, it is below 212 GeV. In its final year of running, reaching 208.6 GeV, the LEP combined data showed hints of a Higgs boson at 115 GeV. Throughout LEP’s second phase in 1996-2000, at energies above the W -pair threshold, L3 used its special capabilities for identifying and measuring leptons and photons to search for a host of new particles, using precise BGO calibrations from Caltech’s RFQ accelerator system. This included searches for the light supersymmetric neutralino, heavy neutral leptons, and excited lepton’s where L3 had the highest sensitivity at LEP. Newman’s Caltech group is completing its work on L3 data analysis, where it had lead roles in the SUSY and exotic particle search efforts, was a major contributor to the W -boson and electroweak coupling measurements. This group has taken a lead role in the final stages of the analysis, in the search for new physics processes using photon + missing energy signatures, capitalizing on its work on BGO calibration. This experience also has been carried forward to CMS, where we have established a leading role in the development of precision measurements of final states containing two photons at the LHC, which are the key to Higgs searches in the mass range up to 150 GeV. The CLEO experiment at Cornell’s Laboratory of Nuclear Science was the premier laboratory for the study of the decays of the heavy b and c quarks, and the τ lepton, in the nineties. The Caltech group (Weinstein and Barish) played major roles in the success of the experiment, especially in the study of τ decays, semileptonic B decays, and rare B decays. Caltech’s involvement with CLEO ended with the termination of the CESR Υ(4s) program. Caltech was deeply involved in plans for the GEM detector at the SSC, with Barish as Spokesman. The termination of the SSC necessitated substantial readjustments and redirection of our program. The program of the Mark III was carried forward by Hitlin and Porter at the BES experiment in Beijing. This work resulted in a dramatically improved measurement of the τ mass, which resolved a seeming inconsistency between the τ mass, lifetime and leptonic branching fraction, as well as new high statistics measurements of J/ψ decays. BABAR has been a major success. Hitlin (the founding Spokesman) and Porter (the computing system manager and Council Vice-chair) have been heavily involved in BABAR since its inception. BABAR was the first experiment to a measure statistically significant CP violating asymmetry in B0 decays to CP eigenstates, a long sought experimental prize. The experiment has recently discovered a new resonant + 0 state decaying to Ds π at a mass that was quite unexpected. Many additional results on CP violation 3.3 Operating Mode of the Caltech HEP Group 27 and rare decays are flowing steadily from BABAR, which plans a program extending through the coming decade. In addition, plans are being developed for an upgraded experiment to run at a version of PEP-II upgraded to a luminosity of 1036 cm−2s−1.

3.3 Operating Mode of the Caltech HEP Group

The Caltech Experimental Group presently consists of six professorial faculty members, eleven research faculty and nine Ph.D. students, plus technical and administrative staff. We have scoped our research activities and commitments around a group of this size and support level. We have traditionally been able to play significant roles in three or four different experimental efforts, while doing R&D towards future projects. This requires careful planning and coordination of our limited technical and computing resources. Having several projects in different stages creates the activity and consistency necessary to maintain high level technical and computer facilities. This diverse program has made Caltech a very active center for high energy physics and has been an essential element in attracting postdocs, graduate students, and visitors. Students and postdocs have historically gone on to excellent opportunities at other US and international institutions. Retention in the field of high energy physics of people who either received their Ph.D.’s from Caltech or did postdoctoral work here has been extraordinarily high. This attests both to the high quality of individuals attracted to Caltech and to the impact these people are able to make on their experiments while they are here. HEP experimental projects can be broken into four phases: design, construction, running, and analysis. All four require coordination with the national laboratories and collaborators; but with the exception of running the experiment, the other phases can have significant aspects of the activity on campus. We believe that it should be a goal to keep as much of our activities as possible on campus. This requires, for example, maintaining state of the art on-campus computing facilities, developing good telecommunications and video conferencing facilities, and having adequate design and technical skills on campus. It is our belief that our program is better as a result of this effort, as we have a strong intellectual base, including both grad students and postdocs on the campus. A key to achieving this with limited resources is to share our infrastructure at Caltech among all our projects. We have no permanent groups at Caltech and instead, form groups around individual experimental projects and support each directly (e.g., salaries, travel, equipment, computing and technical resources). Groups with a different mix of people often form or reform around new efforts. Technical staff does not belong to a specific group, but is assigned to projects as required. Combining the needs of several efforts, which are not at the same stage of development, we are able to level the load on shop and computer facilities. An example was the integration of the SLD and MACRO technical work. We developed and constructed a major part of the electromagnetic calorimeter for the SLD detector. Mechanical prototypes were built to develop a design for construction of the actual calorimeter. An instrumented prototype was then built and tested in a beam at SLAC to determine the optimum uranium compensation for a liquid argon calorimeter. These tests showed that previous results indicating that a uranium-liquid argon calorimeter was compensating were erroneous; the calorimeter was therefore implemented using lead radiator with liquid argon. The construction of modules for the barrel electromagnetic calorimeter took place in three universities, including Caltech. Following that construction, we were able to shift much of our technical work to the MACRO detector for the Gran Sasso. The first full scale production prototype of the scintillator detector modules for use in this facility was built and tested at Caltech during the phase when the primary technical activity was the SLD calorimeter. After optimization in the design, Caltech took on the major responsibility for construction of these liquid scintillator detectors, which consisted of more than 500 scintillator detectors, 28 3 The Experimental Program each 12m long x 0.75m wide and filled with 2 tons of liquid scintillator. This construction, undertaken following the completion of SLD calorimeter, used a significant fraction of our technical resources for several years. In addition, the special electronic circuits for monopole triggering (a challenging task) were developed and built at Caltech.

When MACRO construction ended, we embarked on new technical R&D projects: (1) R&D on BaF2 for the GEM/SSC electromagnetic calorimeter, including studies of radiation damage; (2) development of muon trigger electronics for GEM; and (3) detector development for BABAR, including work on aerogel particle identification systems, readout and radiation damage in CsI(Tl) detectors and development and construction of a radioactive liquid source for calibration of CsI crystals. Following the termination of the SSC, we phased out the GEM development work and the main technical activity shifted to a role in construction of the BABAR detector, the DAQ system for CLEO III and R&D activities for CMS and MINOS. The R&D activities have been successful; Caltech has recently completed construction of the majority of the MINOS farsite scintillator modules, as well as the CMS crystal calibration and monitoring system. We are also a major development center for CMS computing and Grid software, and the recognized leaders in current developments of Grid-based data analysis. As our technical infrastructure declined in earlier years, we undertook a central role in computing, networking and more recently network-distributed data analysis developments for our field. No less than our detector developments, this has shaped the way that the largest international collaborations do physics. Examples include the MARK J experiment at DESY which used the first transatlantic networks, the design and implementation of computing and data analysis systems for L3, and more recently the design and development of the computing, networking and remote collaboration systems for CMS, for the LHC experiments, and for DOE’s major international collaborations in general. In 2000-2001, Caltech has forged the CERN-Internet2 relationship where CERN is the only non-US full member, and has been instrumental in making HENP a focal discipline in Internet2 research and development, bringing HEP as a field to the forefront of the international networking scene. In 2002-3 we established a world-leading role in the development of new network protocols, both in the HEP group itself and in collboration with computer scientists at Caltech, and led the teams that currently hold all of the Internet2 Land Speed Records for high speed data transfers over wide area networks. In all of these cases, the developments of these new-generation systems were led by the Caltech HEP group. The Caltech BABAR group has also pioneered the development of modern on-line event-processing computing technology, as well as the development of sophisticated software on-line trigger software. The computing system at Caltech has, for the decade of the 90s, primarily been an IBM/RISC system, assembled with shared Caltech and DOE funding. The system was heavily used for analysis of data from and generation of Monte Carlo events for the different experiments. However, by the end of the decade it was clear that we needed a more substantial upgrade than possible with the sporadic equipment funding, in order to maintain a facility that would meet evolving and increasing demands. In fiscal years 1999-2000 we thus implemented a significant upgrade with costs shared between Caltech and DOE. The new system follows the recent trend back to separated functions: We have a central linux compute farm, and central IDE-RAID fileservers with DLT backup. The desktops have either Windows or Linux computers, according to user preference. The building networking has been upgraded to switched 100BaseT, and we have faster fiber connections to the fileservers. It is important that we systematically evolve this facility to meet ever increasing demands and to maintain a modern state-of-the-art system. This requires an ongoing computing equipment investment, which is reflected in the current proposal, including the out years. The fact that all the research groups share common computer and technical facilities has allowed 3.4 The Physics Program 29 us to make a strong impact on all facets of high energy experiments from Caltech. We have been able to do this even with very limited resources, and while contending with the same infrastructure decay seen in most universities in recent years. We must keep our technical resources and computer facilities at a level where we are able to continue to take significant responsibility on experimental projects as they become more and more complex and ambitious. We have become, for example, one of the largest US university Monte Carlo production centers for BABAR. We place the highest priority on maintaining these capabilities; the separation of support for infrastructure from the individual experimental tasks helps structurally to protect this crucial element of our program. We have been quite successful in leveraging local computing resources to obtain access to other facilities, such as the Caltech CACR and CERN. By engaging in frontline R&D on network-distributed systems we also have gained access to other resources such as the Condor-driven systems at Wisconsin, which have been offered to us by Wisconsin (Computer Science) and at other sites managed by the National Partnership for Advanced Computing Infrastructure (NPACI) for large scale Monte Carlo event generation. We worked with Caltech CACR to carry out a successful “seamless Data Grid prototype” that was a key factor helping with the approval of NSF’s Distributed Terascale Facility (DTF), now known as the TeraGrid. Our work on Higgs searches with full simulations of backgrounds at the LHC has been recognized as a “flagship application” of the TeraGrid, one node of which is at CACR, and we will start major use of the TeraGrid for simulations, reconstruction and large-scale Grid development this year. We have also obtained research funds in past years from IBM and Hewlett- Packard for computing activities directly related to high energy physics.

3.4 The Physics Program

The main elements of our experimental program are presently three ongoing experiments. We are actively searching for a new young experimental faculty member, who will likely become involved in a new eﬀort. Complete discussions on each of the current experimental eﬀorts are given in the body of the report.

3.4.1 CMS and L3

The CMS/L3 group led by Harvey Newman has completed its search for the Standard Model or Super- symmetric Higgs, and is completing its search for other new particles with L3 at LEP. Caltech led two of L3’s three particle search analysis efforts, and has been a major contributor to the measurements of the W mass, width and electroweak couplings. We recently completed leading roles in the the searches for supersymmetric leptons in 2002, and are currently leading searches for extra dimensions, supersymmetry and other new physics processes using signatures with one or two precisely measured photons and missing energy. In its final year of running, LEP reached 208.6 GeV in the center of mass and and saw hints of a Higgs with a mass of approximately 115 GeV (at the upper end of its mass reach), but the statistics accumulated in a few months’ running were insufficient to definitively establish the signal. We recently This group has now turned to the development of Higgs and other new particle searches with CMS at the LHC, using lepton and photon signatures, and exploiting the capabilities of CMS’ precision Lead Tungstate calorimeter and all-Silicon tracker. We founded and led the development and implementation of the offline computing and networking systems for the L3 experiment, including the network-distributed simulation production. We designed, built and operate the RFQ system used for BGO calibration, which achieved sub-percent precision since 1997, and half-percent calibration resolution in 1999-2000 when LEP reached its highest energies. 30 3 The Experimental Program

The CMS/L3 group also has leading roles in the scientific direction, software, computing and networking, and the development of the crystal electromagnetic calorimeter for the CMS experiment. H. Newman was elected US CMS Collaboration Board Chair in 1998, re-elected in 2000 and again in 2002. He is the US representative on the CMS Management Board. He was the chair of the experiment’s Software and Computing Board until 2001, and had the primary role in initiating the US CMS Software and Computing effort and the launching of the US CMS Software and Computing Project during 1996-2000. R. Zhu was re-elected as Chair of the US CMS ECAL Institute Board this year, and shares responsibility for crystal development with the ECAL Project Manager and Technical Coordina- tor in CMS. We have construction responsibility for the precision laser-based monitoring system for the ECAL. Through Hewlett Packard-, NSF- and DOE-funded projects for the study of next generation database technology and “Data Grids”, and special computing facilities provided by Caltech’s Center for Advanced Computing Research under an NPACI grant, we have established a unique role in the study and development of the worldwide-distributed data analysis and database systems for the LHC program. Based on this role, we began to develop the first “Grid-enabled Analysis Environment” for CMS in 2002, and have since assumed a leading role in this area, in the US HENP Grid projects. This role is supported by our management, operation and ongoing development of US-CERN networking for the US HENP community since 1989. In 2002 we took on a lead role for networking on behalf the imternational HENP community, by chairing the ICFA Standing Committee on Inter- Regional Connectivity (ICFA SCIC). This group’s preparatory work on Higgs and other new particle searches using electron and photon signatures in the CMS crystal ECAL and tracker will be strengthened this year, as it is being joined by the CLEO group (Weinstein and two postdocs).

3.4.2 BABAR

The group led by David Hitlin and Frank Porter has a major role in the study of CP violation and rare processes in B decays being carried out at PEP-II, the SLAC asymmetric B Factory, with the BABAR detector. PEP-II has already exceeded design luminosity, a remarkable achievement. The BABAR detector was moved onto the beam line in March 1999, and has been taking colliding beam data since May − 1999. Over 120 fb 1 has already been accumulated, and the first statistically significant measurements of the CP -violation parameter sin 2β has been published. More data is being accumulated, and many other physics topics are being addressed, several already published or submitted for publication. For example, an exciting recent discovery is the Ds(2317), a charm-strange state which does not fit well with prior theoretical expectations. The effort of this group is currently devoted nearly completely to BABAR, with a leading role in the online event processing, reconstruction and several other software areas, and a leadership role in the collaboration (e.g., Hitlin completed his second term as BABAR Spokesman in 2000, while Porter leads the BABAR statistics working group and was elected to the Collaboration Council Vice-Chair position). Hitlin and Porter have also worked on studies of e+e− annihilation near tau and charm threshold, the focus of the BES experiment. The precise measurement of the mass of the τ lepton made by the group has had far-reaching implications, affecting precision universality tests using τ lifetime and electronic branching ratios. More recently, Caltech has worked on the Ds physics and ψ physics. We terminated our involvement in BES with the departure of our last graduate student, though there is a small lingering interaction on results still being prepared for publication. Finally, looking toward the future, Caltech has been involved in the activities toward a linear collider detector. Porter served until recently as an interim coordinator (with Ray Frey from Oregon and Andre Turcot from Brookhaven) of the Calorimetry Working Group for the North American effort. On another future possibility, Hitlin is leading a study of the physics and technical challenges for a higher luminosity 3.4 The Physics Program 31

− − (1036 cm 2s 1) B factory, accompanied by an upgraded BABAR detector called SuperBABAR.

3.4.3 MACRO and MINOS

The MACRO experiment (Barish, Peck, Michael) was one of the largest underground detectors in the world, designed to search for rare particles in cosmic rays (monopoles, nuclearites, etc.), study downgoing muons of very high initial energy and therefore the interactions and composition of the primary cosmic rays that produced them, study upgoing muons induce by neutrino interactions and search for bursts of anti-neutrinos from gravitational collapse within the galaxy. The operation of MACRO ended in December of 2000 and final analyses and papers are now in preparation. No magnetic monopoles were observed. The measurements from MACRO on upgoing muons and contained vertex events produced from interactions of atmospheric neutrinos provide one of the best measurements of oscillation effects on atmospheric neutrinos. The results from MACRO are consistent with those from Super-Kamiokande and analysis of the angular distribution has shown that oscillations between νµ and ντ are favored over νmu to νe or νsterile. The MINOS (Main Injector Neutrino Oscillation Search) experiment (Barish, Peck, Newman, Michael) is designed to pursue the oscillations associated with the atmospheric anomaly using a neutrino beam from the Fermilab Main Injector aimed towards Soudan, Minnesota. The detector will employ a large sampling calorimeter with magnetized iron plates and with layers of solid scintillator. The MINOS experiment will provide unambiguous evidence for the oscillations, measure the flavor participation in the oscillations (even in complex, multi-flavor mixing scenarios and those including sterile neutrinos) and provide precision measurements of the oscillation parameters. Caltech has played a leading role in the proposal and development of this experiment. Doug Michael was elected as MINOS Co-Spokesperson in July 2002 and continues as the manager for the scintillator system. The Caltech group has recently completed production of one-half of the scintillator modules for the far detector. The Caltech group is strengthening a major new effort directed towards improving the proton intensity delivered to the MINOS target, starting to work on analysis of atmospheric neutrino events in the far detector which is now almost complete. We are also starting to work on analyses for beam neutrinos, with a focus on the νµ disappearance measurements and on νe appearance.

3.4.4 CLEO II and CLEO III

The CLEO-III run on the Υ(4S) and the bottomonium resonances (Υ(3S), Υ(1S), Υ(2S)) is now over, and the Collaboration has made the transition to CLEO-c. The Caltech group is now finishing up our last physics analyses using CLEO II, II.V, and III data. Our final results on the rare B → Kππ and K∗π decay rates and CP asymmetries are now published. Final results on the full differential distribution in inclusive semileptonic B decay (both B → Xc ν and B → Xu ν), including moments, tests of the Heavy Quark Expansion, and extraction of |Vub| and |Vcb|, will be presented at the summer 2003 conferences. Measurements of the rates for the rare radiative decays B → γKππ are in preparation for the summer conferences. Measurements of the rates and differential distributions for the decays Υ(3S) → ππΥ(2S), Υ(3S) → ππΥ(1S), and Υ(2S) → ππΥ(2S) (in 12 different final states) are nearing completion. The anomalous distributions in Υ(3S) → ππΥ(1S) are now quantitatively understood in terms of the underlying matrix elements. Our study of the differential distribution in τντ Kππ, and the measurements of SU(3)f violation and axial-vector mixing in these decays, is also nearing completion. Analysis of CLEO data should be essentially complete by the summer of 2003. We have already begun to work on CMS physics: postdoctoral scholar Bornheim is now resident at CERN and is very active in CMS ECAL development, and postdoctoral scholar Pappas is beginning to come up to speed on CMS tracking and physics analysis. By the end of calendar 2003, we will have completely refocused 32 3 The Experimental Program our efforts onto the CMS physics program.

3.5 LHCNet: Networking, Data Grid and Collaborative Sys- tems

The Caltech group (Newman) first proposed the use of international networks for HENP research, and has had a pivotal role in transatlantic networks for our field since 1982. Our group was funded by DOE to provide transatlantic networking for L3 (“LEP3NET”; since renamed LHCNet) starting in 1986, based on earlier experience and incremental funding for packet networks between the US and DESY (1982-1986). From 1989 onward, the group has been charged with providing CERN-US networking for the HEP community, and mission-oriented transatlantic bandwidth for HEP tasks. Apart from our direct responsibility for transatlantic networks for HEP over the last 20 years, the Caltech group has had key roles in the development of international networks, distributed data analysis systems and systems for remote collaboration. Examples of network developments include the debugging of the first US-Europe public (X.25) network service, TELENET in 1982; work on the Technical Advisory Group of the NSFNET in 1986-7; setting up and hosting the visit of IBM to CERN on behalf of US physicists, leading to the funding by IBM of the first T1 transatlantic link for research (at LEP) in 1988-91; network studies for the LEP and former-SSC eras; and ongoing tests of high throughput data transfers and high performance bandwidth management needed by DOE’s major experiments. Recent examples of our remote collaborative and “Data Grid” system developments are: Caltech’s VRVS packet videoconferencing system (now on more than 10,000 hosts in 61 countries, in HENP and other fields) since 1995; and the GIOD, MONARC, ALDAP, Particle Physics Data Grid (PPDG), Grid Physics Network (GriPhyN) and iVDGL projects discussed in the CMS/L3 and LHCNet chapters of this report. The VRVS system software was almost entirely rewritten by our team in 2002 (following a long- established development plan, on schedule). In February 2003 we successfully released VRVS Version 3.0, which represents a major upgrade in both features and scalability, making a smooth transition to the new system while VRVS operations continued on a daily basis. Since 1996 our group has been charged to provide and support US-CERN network services for the US CMS and US ATLAS programs for the LHC. In 1997-8 we took a leading role in assessing and planning for the networking of future experiments worldwide, through the ICFA Network Task Force. From 1999 Caltech has continued this role on a more permanent basis, through the ICFA Standing Committee on International Connectivity (SCIC), established by ICFA in 1998. H. Newman took over as ICFA SCIC Chair, as a member of ICFA in February 2002. Since December 1995 Caltech has been the major partner in the “USLIC” consortium with CERN, Canadian HEP, IN2P31, and the United Nations (WHO) in Geneva funding a dedicated CERN-US line. Operations, management, and the development and application of systems for traffic monitoring and control, are shared between the CERN/IT External Networking Group and our Caltech group. In 1999-2001 our group has been instrumental in bringing about CERN’s membership in UCAID, the managing organization of Internet2; the only non-US organization allowed equal member status with the Internet2 member US universities. Since 2002 we have established a primary role in making HENP one of the central disciplines for Internet2 R&D efforts, through Internet2’s Application Strategy Council, and its End-to-End Intiative, where Caltech is the HENP contact. We started and lead an Internet2 HENP Working Group in

1Institut National de Physique Nucl´eaire et de Physiques des Particules, France. See http://www.in2p3.fr 3.6 Staﬃng, Budget, and Plans 33

October 2001. In 2002 we broadened these eﬀorts in support of CMS data analysis, through contacts with the AM- PATH (South American) network and CMS collaborators in Brazil who are planning a large Regional Center, as well as work on network and regional center plans in India, Korea, Pakistan, and Switzer- land. We have run Grid workshops in Rio de Janeiro and Bucharest (at the request of the Romanian government) in the winter and spring of 2002. In the Fall of 2002 we led a session on the “Role of New Technologies” at the Pan-European Ministerial Conference in Bucharest, which led to work with the US State Department and with CERN on preparation for the World Summit on the Information Society in Geneva this December. The long term goal of LHCNet and our Data Grid projects is to provide the levels of managed transatlantic bandwidth, and high performance data handling over transoceanic networks, required for the LHC as well as DOE’s other major physics programs (including BABAR and FNAL Run2). A long- range plan of progressive bandwidth upgrades and projected costs, reviewed in 2001 by a Transatlantic Network Working Group commissioned by DOE and NSF (and co-chaired by Newman), and recently updated is presented in the LHCNet chapter of this Annual Report. In support of this work, we have developed and taken charge of a leading edge program of network developments, in collaboration with CERN, Caltech Computer Science, SLAC, FNAL, Los Alamos, Internet2, the TeraGrid and the European DataTAG project.

3.6 Staﬃng, Budget, and Plans

The effort at Caltech is balanced between the extraction of physics from the ongoing BABAR program, the Υ phase of CLEO-III, and the final phase of L3, and development work and construction on two longer term experiments, MINOS at Fermilab and CMS at the CERN LHC. We are also expending some effort on two nascent projects, SuperBABARand the NLC. We have significant involvement in all these efforts, with visible leadership roles, technical responsibility, and substantial physics accomplishments. The Caltech HEP program promises a well-focused, productive effort over the coming years. It has an interesting mix of physics objectives, with real potential for major discoveries in supersymmetry, CP violation and neutrino oscillations. These ongoing experiments represent a diverse, full program; the detectors are technically complete and the Caltech groups and their roles are well established. We have been able to use our technical resources to imake important contributions to the construction of the BABAR, CLEO-III, MINOS, CMS detectors, and are beginning R&D on new projects. We have suffered over the past few years from funding cutbacks, which have to some extent limited our ability to carry out our program. It has been necessary to reduce staff, generally by attrition, especially in the technical staff, in order to live within the present level of funding. We have, in response, evolved our technical facilities toward emphasis on electronic design, building upon Caltech- provided design tools (CADENCE design software) and innovation developments in online and offline computing. With our major computing software system development activities for BABAR , CLEO-III and CMS, we are now contributing to the development and construction of new detectors in an important manner that respects the diminution of our mechanical and electronic technical infrastructure. This evolution of our technical infrastructure from a mechanical emphasis to an emphasis on electronic design, modern software engineering, and networking is our response to persistent funding limitations in the infrastructure area; it has thus far proved to be effective. We have been realistic in this proposal about our requests, redirecting resources as older efforts end and new ones require more effort; we are seeking only an inflation adjustment, as the Institute 34 3 The Experimental Program will mandate salary increase in the coming year, and salaries dominate our expenditures. This request represents real needs to effectively carry out our program, while remaining cognizant of the existence of budget pressures that are certain to persist. Our plan is to keep a very strong effort in doing physics with BABAR build up efforts on CMS and MINOS, while concluding analysis of physics from CLEO-III and L3. We believe that we have developed a well-balanced program, in which Caltech plays leadership roles in a wide variety of experiments. We envision a smooth transition from current involvements to new programs over the coming decade as these new projects get underway. We expect to make at least one new faculty appointment in the coming year, with an eye on bring a new experimental effort to Caltech. 4. CLEO-II and CLEO-III A. Bornheim, E. Lipeles, S. Pappas, A. Shapiro, W. Sun, A. J. Weinstein

4.1 Executive Summary

The CLEO-III run on the Υ(4S) and the bottomonium resonances (Υ(3S), Υ(1S), Υ(2S)) is now over, and the Collaboration has made the transition to CLEO-c. The Caltech group is now finishing up our last physics analyses using CLEO II, II.V, and III data. Graduate student Sun has successfully defended his PhD thesis in May, and (tentatively) plans to continue as a postdoc at Cornell. Lipeles will defend by the end of summer 2003 and will begin a postdoc at UCSD and CDF. Shapiro expects to complete her thesis by the end of summer 2003 as well. Postdocs Pappas and Bornheim aim to complete their physics analyses and prepared their publications by the end of summer 2003 as well. All of our remaining institutional commitments to the CLEO program will be complete by then. By fall 2003, we will shift our focus entirely to CMS physics, merging with the Caltech CMS group. The budget requests for last year (FY03) and this year reflect this transition. Our group is particularly interested in developing the electron and photon detection and identification capabilities of the CMS detector, in the search for new physics. Postdoc Bornheim is now resident at CERN (as of 11/02) and has taken on responsibilities in the integration and maintenance of the Caltech-built CMS ECAL Monitoring System at CERN. He participated in the CMS ECAL beam test in 07-09/2002, and is contributing to the analyses of those data and the development of precision calibration of the ECAL system. He is serving as the safety officer for the CMS ECAL laser source at CERN since 01/2003. He pursues various coordination and organisation tasks for the laser source maintenance and barrack construction at CERN. He is currently working on preparations for the 2003 test beam season at CERN. In the context of this work, he has already given a talk on “Crystal Calorimeter Monitoring” at the FINUPHY Workshop on Advanced Electromag- netic Calorimetry and its Applications - FEMC03, March 10 2003, Forschungszentrum Jülich, Jülich, Germany, and has attended the “Workshop Analysis Calorimetry H4” in Paris in 11/2002. Currently, Postdoc Pappas is beginning to come up to speed on CMS tracking and physics analysis tools. In this chapter, we review the CLEO physics activities currently in progress and nearing completion.

• Shapiro’s study of the differential distribution in τντ Kππ, and the measurements of SU(3)f violation and axial-vector mixing in these decays, is nearing completion (section 4.2). • Sun’s final results on the rare B → Kππ and K∗π decay rates [1] and CP asymmetries [2] are now published (section 4.3). • Lipeles’s final results on the full differential distribution in inclusive semileptonic B decay (both B → Xc ν and B → Xu ν), including moments, tests of the Heavy Quark Expansion, and extraction of |Vub| and |Vcb|, are in preparation. Preliminary results were presented at the summer 2002 conferences [3], and final results will be presented at the summer 2003 conferences (section 4.4). • Bornheim’s measurements of the rates for six rare radiative decays B → γKππ are in preparation. Preliminary results were presented at the summer 2002 conferences [4], and final results will be presented at the summer 2003 conferences (section 4.5).

35 36 4 CLEO-II and CLEO-III

• Pappas’s measurements of the rates and differential distributions for the decays Υ(3S) → ππΥ(2S), Υ(3S) → ππΥ(1S), and Υ(2S) → ππΥ(2S) (in 12 different final states) are nearing completion. The anomalous distributions in Υ(3S) → ππΥ(1S) are now quantitatively understood in terms of the underlying matrix elements (section 4.6). • Weinstein has made significant contributions to the measurement of the leptonic energy spectrum and moments in the inclusive semileptonic decay B → Xc ν recently published [5] by C. Boula- houache and the Syracuse CLEO group. Weinstein is also contributing to the study of the leptonic energy spectrum in di-lepton events, with C. Stepanik and the Minnesota group, which is in preparation for the summer conferences. Weinstein presented a summary of results on tau physics from the CLEO II and II.V program, at Tau 2002 in UC Santa Cruz [6]. Bornheim has presented results from CLEO on semileptonic tau decays, and extraction of the CKM matrix elements Vub and Vcb, at Moriond - Electroweak 2003 [7].

∓ 0 ∓ 0 4.2 Study of the decays τ → Ks h π ντ

Cabibbo-suppressed τ decays provide insight into the dynamics of strange vector and axial vector mesons, in analogy with the corresponding Cabibbo-favored processes described earlier. However, the dynamics of the Cabibbo-suppressed modes tend to be more intricate, and thus in many respects more interesting. This is the case because the symmetries which constrain the production of non-strange states in τ decay are either absent (CVC), or weaker (SU(3)f vs. isospin) in the production of modes with kaons.

− 0 − 0 Phenomenology of τ → KSh π ντ

− − The decay τ → (Kππ) ντ is expected to arise primarily via the axial vector weak current. This decay can proceed through two axial vector resonances predicted in the quark model, the K1A (whose quark spins are aligned) and K1B (whose quark spins are anti-aligned). Even though they are S =1 counterparts of the a1(1260) and b1(1235), respectively, there is a fundamental diﬀerence: while the − decay τ → b1(1235)ντ is forbidden by the conservation of G-parity (due to isospin symmetry), K1B production from τ is allowed because the operative symmetry SU(3)f is broken (ms mu,d). We can probe the extent of SU(3)f breaking by measuring the relative amount of the two K1s produced in τ decay [8].

However, the experimentally observed K1 states have the property that one, denoted by K1(1270), ∗ decays preferentially to Kρ while the other, K1(1400), prefers the K π final state. This differs from the ∗ K1A/K1B scenario where each K1 is expected to decay equally to Kρ or K π states. This is explained by mixing between the quark-model K1A/K1B states to give rise to the observed K1’s [8]. Disentangling these two interesting effects (SU(3)f violation and mixing) requires a detailed study of the resonant sub-structure of the Kππ system.

− 0 − 0 CLEO analysis of τ → KSπ (π )ντ

− 0 − 0 To explore this interplay of eﬀects, we search for decays of the type τ → KSπ (π )ντ recoiling against a ‘tagged’ τ + decaying to one charged track and a π0. Full CLEOII and CLEOII.5 data sample corresponding to 13.6 fb−1 has been analysed. From this data sample we select events with 4 charged tracks, with one ‘tag’ track being isolated from the each of the other three ‘signal’ tracks by at least 90◦. ∓ 0 ∓ 0 4.2 Study of the decays τ → Ks h π ντ 37

0 + − Since the KS typically travels several cm before decaying into π π , we require a distinct secondary vertex to be formed by two of the three signal tracks. We reconstruct π0s in each event from energetic (E>50 MeV) photons outside of the ‘hot’ region of the CsI calorimeter (| cos(θ)| < 0.95). The 0 0 remaining combinatoric backgrounds to the KS and π signals are estimated and subtracted from the 0 data using side bands. Since we are not able to identify the track accompanying the KS, the event samples contain contributions from mode with two kaons K−K0π0, which we model in our ﬁtter. Remaining backgrounds, from mis-reconstructed tau decays and from the hadronic continuum, amount to 5% of the total data sample, and are modeled using Monte Carlo. From a 13.64 fb−1 CLEOII/II.5 data sample corresponding to 12.5 million τ-pairs, we obtain for − 0 − 0 0 the branching fraction B(τ → K π π ντ ; Ks → ππ)=(0.104 ± 0.0034 ± 0.0088)%, where systematic error is not complete. This result agrees within errors with the value from the PDG01 world average. 0 − 0 The mass spectra for the KSπ π mode is shown in ﬁg. 4.1. The spectra can be well represented − 0 − 0 by the two K1 + K K π Monte Carlo, with both K1(1270) and K1(1400) contributing.

600

N events data 500 cleo2 MC 0 π0 cleo2 KsK MC 400 cleo2 ττ background MC cleo2 continuum MC

300

200

100

0 0.6 1.0 1.4 1.8 2.2 2.6 3.0 2 MKππ [ GeV/c ]

0 − 0 Figure 4.1: The invariant mass for the KS π π mode for data (points) and Monte Carlo (histogram), normalized to the data.

We employ all the kinematic observables from the Kππ ﬁnal state in an unbinned likelihood ﬁt to a general model (including possible scalar and vector contributions). A precise measurement of the ∗ relative contributions from K1(1270)/K1(1400) decaying to K π and Kρ, as well as the contribution from Wess-Zumino (vector) terms, is nearing completion. From these results we are able to extract the amount of SU(3)f symmetry breaking as well as the K1 mixing parameter. These results should be ready for publication by the end of summer 2003. 38 4 CLEO-II and CLEO-III

4.3 Charmless Hadronic B Decays

Charmless hadronic B decays allow for stringent tests of the Standard Model description of quark- mixing and CP violation. Of particular interest are two classes of decays: CP eigenstates, like π+π− or 0 φ(1020)KS, whose time-dependent CP asymmetries are sensitive to the CKM angles α and β, and decay modes with interfering amplitudes, such as K±π∓, K±π0,andK∗(892)±π∓, which can exhibit direct time-integrated CP asymmetries when the amplitudes carry diﬀerent weak and strong phases. In many non-Standard-Model scenarios, sin 2β measured in CP eigenstates with b → s penguin contributions 0 can diﬀer from measurements using tree-level processes like B → ψKS [9]. Figure 4.2 shows the Feynman diagrams for the physics processes that are expected to dominate charmless hadronic B decays within the Standard Model. Most of these decays receive contributions from more than one diagram, which complicates the extraction of the CKM parameters. Isospin and SU(3) symmetries are often invoked to disentagle these contributions. Such arguments cannot be applied, however, to electroweak penguin transitions (Figure 4.2(d)).

3460997-007 I I u W I b d, s I I I I W d, s I q I t, c, u I b u + 0

B , B I + 0 g B , B q u, d u, d u, d u, d ( a ) ( b )

q I II I b u Z,γ q I I I I I W u t, c, u I + 0 b d, s B , B I + 0 W d, s B , B u, d u, d u, d u, d ( c ) ( d )

Figure 4.2: Dominant lowest order Feynman diagrams for charmless hadronic B decays: a) b → u external W emission (tree, or T ), b) gluonic ﬂavor octet penguin (P ), c) b → u internal W emission also referred to as “color suppressed” diagram (C) d) color allowed electroweak penguin diagram (PEW ).

Using its full dataset CLEO has observed all four B → Kπ transitions, B → π+π−, both B → ηK decays, as well as both B → ηK∗ decays, B → ωπ±, two of the four B → ρπ decays, as well as one of the four B → K∗π decays. All of these observations were made within the last few years and employ a maximum likelihood technique, which Würthwein helped to pioneer. Sun and Würthwein are primary authors of the Kπ, ππ, K∗π, and analyses. As CLEO analysis co-ordinator 98/99, Würthwein contributed significantly to searches for all charmless decays to pseudoscalar-vector final states (ρπ, K∗π,ωh±, etc.) ± ∓ ± 0 0 ± ± ± as well as for CP violation in B decays to K π ,K π ,KSπ ,η K ,ωπ . These studies were discussed in previous editions of this Report and have resulted in four PRL publications [10, 11, 12, 13]. Sun’s work on three-pseudoscalar final states is the subject of one published [1] and two forthcoming publications. 4.3 Charmless Hadronic B Decays 39

4.3.1 B Decays to Charmless Three-Body Final States

Two-body decays of B mesons into channels containing a ρ or K∗ and a π or K are the pseudoscalar- vector (PV) analogs of the two-pseudoscalar (PP) modes B → Kπ/ππ studied by Würthwein. Mo- tivated by the observation of the B → Kπ decay modes, we have searched for b → s transitions in 0 ± ∓ ± ∓ 0 0 ± 0 ± B → PV decays with the final state topologies KSh π , K h π ,andKSh π , where h denotes a charged pion or kaon. − Using 9.1 fb 1 collected on the Υ(4S) resonance, we first search for three body signals in the topologies mentioned above, disregarding any possible resonant substructure. These maximum likeihood 0 + − fit results are shown in Table 4.1. We observe a signal for B → KSπ π with a statistical significance of 8.1σ. Each event is efficiency-corrected according to its position in the Dalitz plot, so the resultant branching fractions are free from model dependence. The efficiencies given in Table 4.1 are averages over the observed events. The resonant substructure of these three-body decays is probed by including the Dalitz plot variables as inputs to the fit and allowing for various intermediate resonances as well as non-resonant phase space decay. We neglect interference among these various processes, and we perform Dalitz plot fits for the three topologies with differing combinations of intermediate resonant and non-resonant states, with up to nine signal components. The only channel where we observe a statistically significant signal is B → ∗ ± ∓ +4.6 ∗ ± → 0 ± +2.2 ∗ ± → ± 0 K (892) π with a yield of 12.6−3.9 for K (892) KSπ and 6.1−1.9 for K (892) K π and a 0 ± ∓ combined significance of 4.6σ. These yields are obtained from a simultaneous fit of the KSh π and K±h∓π0 topologies, with the branching fraction for B → K∗(892)±π∓ constrained to be equal in its two ∗ ± 0 ± ± 0 submodes. With efficiencies of 8.1% and 3.9% for K (892) → KSπ and K π , respectively, we obtain B → ∗ ± ∓ +6 ± × −6 a branching fraction of (B K (892) π ) = (16−5 2) 10 . Most theoretical predictions [14] for this branching fraction lie in the range 2–14×10−6. Our measurement is larger than but consistent with most predictions. Both measurements of B → K0π+π− and B → K∗+(892)π− have been published [1] and form the basis of Sun’s thesis.

Mode Yield Significance (%) B×10−6 0 + − +11.5 +12 ± K π π 60.2−10.6 8.1σ 10 61−11 8 0 − + +7.1 K K π 2.4−2.4 0.4σ 7.6 < 21 + − 0 +14.5 K π π 43.0−13.5 3.7σ 16 < 48 + − 0 +11.5 K K π 0.0−0.0 0.0σ 19 < 14 0 + 0 +10.1 K π π 20.3−8.8 2.7σ 4.1 < 110 0 − 0 +3.7 K K π 0.0−0.0 0.0σ 3.1 < 29 Table 4.1: Maximum likelihood fit results for three-body decays. Reconstruction efficiencies include all daughter branching fractions. Yields are sums and branching fractions are averages over charge- conjugate modes. The branching fraction uncertainties are statistical and systematic respectively. Up- per limits are computed at the 90% confidence level.

4.3.2 CP Asymmetry in B → K∗(892)±π∓

We extend the above analysis to search for direct CP violation in the B → K∗(892)±π∓, characterized 0 ∗ − + 0 by the asymmetry between charge conjugate decay rates: ACP ≡ [B(B¯ → K (892) π ) −B(B → K∗(892)+π−)]/[B(B¯0 → K∗(892)−π+)+B(B0 → K∗(892)+π−)]. The charge symmetry of the CLEO detector, the track reconstruction software, and the dE/dx measurement has been verified in previous CLEO analyses. We find the charge asymmetry of the detection efficiencies to be consistent with the 40 4 CLEO-II and CLEO-III

expected null result. Crossfeed among different charge states is not included in the fit, and its effect is estimated with Monte Carlo simulation.

The free parameters in the fit are yields (N) summed over charge states, Nh+ + Nh− , and charge asymmetries, A+− ≡ (Nh+ − Nh− )/(Nh+ + Nh− ). We do not fit for charge asymmetries in the background components, but we measure them to be consistent with zero. In the fit, yields are corrected for efficiency and crossfeed from other modes, and the CP asymmetry in B → K∗(892)±π∓ is measured A +0.33+0.10 to be CP =0.26−0.34−0.08, where the uncertainties are statistical and systematic, respectively. The dominant contributions to the latter are uncertainties in the PDFs and variations in the fitting method.

We determine the dependence of the likelihood function on ACP by repeating the fit at several fixed values of ACP . By convoluting this function with the systematic uncertainties and integrating the resultant curve, we construct a 90% confidence level interval of −0.31 < ACP < 0.78, where the excluded regions on both sides each contain 5% of the integrated area. Figure 4.3 shows the likelihood function given by the fit and the effect of including systematic uncertainties.

3100403-001 Likelihood I I 1.0 0.5 0 0.5 1.0 ACP

Figure 4.3: Likelihood as a function of ACP before (dashed) and after (solid) including systematic uncertainties. The hatched regions each contain 5% of the integrated area and are excluded at the 90% conﬁdence level.

4.3.3 Constraints on the CKM Angle γ

In the ﬂavor SU(3) decomposition of the amplitudes contributing to charmless B → PV decays [15], the transition B → K∗(892)±π∓ is dominated by two amplitudes, tree and penguin, pictured in Figures 4.2a ∗ and 4.2b, which interfere with a weak phase given by the CKM angle γ =argVub and with an unknown strong phase δ. Based on estimates of the magnitudes of these amplitudes, we extract information on γ both with and without assumptions about δ. We denote tree contributions by t and gluonic penguins by p. The subscripts P and V indicate whether the spectator quark is incorporated into the pseudoscalar or the vector meson, respectively. The amplitudes for ∆S = 1 transitions are primed, while those for ∆S = 0 transitions are un- primed. Considering contributions up to O(λ), the amplitude for B → K∗(892)+π− is represented ∗ + − by A(K (892) π )=−(pP + tP ), and the CP-averaged branching fraction, measured in Section 4.3.1, is proportional to

1 |A(K∗(892)+π−)|2 + |A(K∗(892)−π+)|2 = |p |2 + |t |2 − 2|p ||t | cos δ cos γ. (4.1) 2 P P P P 4.3 Charmless Hadronic B Decays 41

∗ 0 + ∗ 0 + We determine pP from pP = A(K (892) π ) using the CP-averaged B(B → K (892) π ). The tree 0 + − ± ∓ amplitude is obtained from −tP = A(B → ρ(770) π ) using the CP-averaged B(B → ρ(770) π ) and the BABAR time-dependent study of B → π+π−π0 [16], which distinguishes between the four 0 + − 0 − + 0 + − 0 − + decays, B → ρ π , B → ρ π , B¯ → ρ π ,andB¯ → ρ π .WeconverttP to tP with the appropriate decay constants and CKM matrix elements. With B(B → K∗(892)±π∓), we are only sensitive to the product cos γ cos δ. By incorporating the CP asymmetry, we can estimate the amplitudes for the two CP states and thereby separate γ and δ: | |2 | |2 −|¯|2 | |2 | |2 −| |2 γ 1 −1 pP + tP A −1 pP + tP A = cos ± cos , (4.2) δ 2 2|pP ||tP | 2|pP ||tP | where A¯ ≡ A(K∗(892)+π−)andA ≡ A(K∗(892)−π+). Gronau and Rosner [17, 18] have suggested using both B → K∗(892)±π∓ and B → φK decays to constrain the CKM phase, according to the following expression: 2 1 2 pEW pEW cos γ = 1+r − R 1+ − 2cosδ , (4.3) 2r cos δ pP pP

t ∗ + − 2 ∗ − + 2 2 ¯ 2 P where R ≡ [|A(K (892) π )| + |A(K (892) π )| ]/[|A(φ(1020)K)| + |A(φ(1020)K)| ], and r ≡ p . P The electroweak penguin amplitude, pEW , has been calculated under factorization [19, 20] to be half that of pP . With this method, we must make assumptions about δ, which is believed to lie in the range 0◦ <δ<90◦ [21, 22]. The experimental inputs to our analysis are: the ratio of charged to neutral B production at the f+− τ+ fK∗ + − 0 Υ(4S), × ; the ratio of decay constants ; Vus; the asymmetries in the BABAR B → π π π f00 τ0 fρ ∗ ± ∓ analysis; ACP (B → K (892) π ), discussed in Section 4.3.2; and measurements from CLEO, BABAR, and Belle of B(B → ρ(770)±π∓), B(B → K/K¯ ∗(892)0π±), B(B → K∗(892)±π∓), B(B → φK±), and B → 0 ¯ 0 − +0.63 (B φK /K ). Using Equation 4.1, we find cos γ cos δ = 0.58−0.95. Based on the smallness of direct CP asymmetries B → Kπ, one can infer a strong phase between tree and penguin amplitudes in these decays of 8◦ ± 10◦ [23]. If the strong phase in B → PV decays is as small as in B → PP decays, then our analysis favors cos γ<0. In particular, for δ =0◦, the peak value corresponds to γ = 125◦. ◦ − +0.53 Using Equation 4.3, we find cos γ at δ =0 to be 0.38−0.77. A → ∗ ± ∓ − +0.93 Using CP (B K (892) π ), cos(γ + δ) and cos(γ δ) are measured to be 0.13−1.22 and − +1.09 ◦ − 1.18−1.52, respectively. The peak value of γ + δ is 83 ; the peak value of γ δ is unphysical. Consid- − +24 ◦ ering only the 22.4% of trials where both cos(γ + δ) and cos(γ δ) are physical, we find γ = (102−25) − +28 ◦ × −4 and δ =( 11−26) with a correlation coefficient of 4 10 . The precision of the current experimental measurements does not yet permit a meaningful extraction of these phases. To achieve uncertainties of f τ less than 10◦ on γ and δ, the precision of the branching fractions, CP asymmetries, and +− × + must f00 τ0 be improved by an order of magnitude. Vub B B B The standard CKM fit [24] to current values of Vcb ,∆m d ,∆m d /∆m s ,and delineates a 95% confidence level interval of γ ∈ [34◦, 82◦]. On the other hand, a global fit to B → Kπ and ππ decays in the QCD factorization framework [23, 25] favors values of γ near 90◦. An isospin analysis [26] also finds evidence for cos γ<0, which agrees with indications from our analysis. A discrepancy between these constraints on γ might arise from new physics in either B-B¯ mixing or the b →{s, d} penguins. 0 0 The latter scenario would produce different mixing phases in B → J/ψKS and B → φ(1020)KS. 42 4 CLEO-II and CLEO-III

4.4 Inclusive B → Xν using neutrino reconstruction

4.4.1 Motivation

Measuring |Vub| and |Vcb| is central to testing the unitarity of the CKM matrix, a major goal of B physics. These can be measured in inclusive semileptonic B decays to mesons containing u and c quarks respectively. Lipeles is ﬁnalizing an analysis of such decays where both the lepton and the neutrino are reconstructed, but the hadronic system is not reconstructed. This data sample is rich in information both about the hadronic structure of the B → Xc ν decays as well as the inclusive B → Xu ν rate. Heavy quark eﬀective theory (HQET) combined with the operator product expansion (OPE) has provided a new level of prediction in inclusive semileptonic B decay. The theory can be used to relate Γ(B → Xclν )to|Vcb| and Γ(B → Xulν ), in a restricted region of phase space, to |Vub|. This theory has a few free parameters (Λandλ1) which can be extracted from moments of the kinematic spectra. The predictions of this theory can be tested by comparing a set of moments which over constrains the free parameters.

Lipeles’s analysis has two goals. First, to measure moments of the B → Xclν diﬀerential decay rate. This is part of a larger program at CLEO to test the HQET+OPE predictions, and make precision estimates of the nonperturbative parameters. This will lead to an improved extraction of |Vcb|.The other part of this analysis is to measure Γ(B → Xulν ) in a restricted region of phase space and relate it to |Vub|.

Preliminary results for the |Vub| extraction were sent to the ICHEP02 conference in Amsterdam [3]. Two papers, one for each result, will be submitted to refereed journals this summer.

4.4.2 Analysis

The data sample that has been selected consists of events collected at the peak of the Υ(4S) resonance by the CLEO II and CLEO II.V detectors, with an identified lepton and a well reconstructed neutrino. The neutrino is reconstructed using the approximate hermiticity of the detector and the well known initial state to infer the kinematics of the missing neutrino, µ µ − µ pν = pe+e− pvisible. With this sample three relevant independent kinematic variables of the decay can be constructed from the lepton and neutrino four vectors. One complete set of variables is lepton energy E, the neutrino 2 energy Eν , and the invariant mass of the lepton-neutrino system q . In addition there are other kinematic variables that are not independent of these, but are useful for understanding various aspects of the decays. One such variable is the invariant mass of the hadron system integrating over the 2 unknown B direction, MX . Another kinematic variable is helicity angle of the lepton, cosθW,whichis the angle between the lepton in the W rest frame and the W in the lab frame. Figure 4.4 illustrates these variables. The backgrounds in the data sample are secondary leptons (i.e., real leptons from decays of the daughters of a B decay), fake leptons, and leptons from continuum events (i.e. non-BB events). The secondary leptons are an irreducible background which is modeled with Monte Carlo simulation, constrained by measurements of semileptonic D decay. The secondaries also have a significantly different distribution in the kinematic variables than the signal. The fake leptons are modeled with real data where tracks are promoted to being leptons using measured fake rates. Finally, the continuum background is modeled by continuum Monte Carlo. 4.4 Inclusive B → X ν using neutrino reconstruction 43

θ El Wl X W M X 2 q ,Q 0 E B ν ν

2 Figure 4.4: Diagram of the inclusive semileptonic decay kinematics. There are six interesting variables E, Eν , q , q0, Mx,andCosΘW. However in the B rest frame there are only 3 independent variables.

We fit the 3 dimensional decay rate distribution for contributions from the hadronic part of the final ∗ ∗∗ state being D, D , D , non-resonant Xc,andXu. The fit is a binned maximum likelihood where the Monte Carlo statistics have been included in the likelihood [27]. The shapes of the components are modeled with Monte Carlo simulations that have been carefully tuned to reproduce all of our current knowledge about the dynamics each of the final states. The continuum and fake lepton normalizations are well understood, and therefore not allowed to float in the fit. The fraction of secondary leptons which pass the 1.0 GeV lepton energy cut is less well known. The secondaries are therefore allowed to float in the fit.

4.4.3 Fit Results

The full CLEO II and CLEO II.V data set has been fit using the procedures described above. The quality of the fit is good as can be seen in figure 4.5. Branching fractions for the various fit variables are calculated from the resulting fit parameters as shown in table 4.2. The statistical errors are very small, but because of the neutrino reconstruction, systematic errors are signifacantly larger and required a large effort to assess. The branching fractions are all consistent with previous available measurements. ) ) )

2 2 16000 → Θ → → 25000 b u Secondary 12000 b u Secondary b u Secondary Dlν Continuum Dlν Continuum Dlν Continuum D*lν Electron Fakes D*lν Electron Fakes 14000 D*lν Electron Fakes ** ν ** ν ** ν D l High Elep Muon Fakes 10000 D l High Elep Muon Fakes D l High Elep Muon Fakes 20000 Nonresonant-X lν Low E Muon Fakes Nonresonant-X lν Low E Muon Fakes Nonresonant-X lν Low E Muon Fakes Events/(.5 GeV Events/(.5 GeV c lep Events/(.1 Cos c lep 12000 c lep

8000 10000 15000

8000 6000

10000 6000 4000 4000 5000 2000 2000

0 0 0 0 5 10 15 20 25 30 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -4 -2 0 2 4 6 8 10 12 14 2 2 Θ 2 2 q GeV Cos Wl MX GeV

2 2 Figure 4.5: Projections of the fit to CLEO II and CLEO II.V data for q , CosW,andMX . The points with error bars are the data. The different components of the fit (signals and backgrounds) are represented as histograms stacked one on top of the other, in the same order, top to bottom, as in the legend.

To evaluate the systematic error, the Monte Carlo events are reweighted to reﬂect uncertainties in either the physics models or our understanding of the detector. Because of the neutrino reconstruction, 44 4 CLEO-II and CLEO-III

Table 4.2: Preliminary B → Xclν Branching Fraction Results The large model dependence uncertainty is due to the extrapolation to low lepton energy. The eﬀect on the moments is signiﬁcantly less. Branching Fraction ± Statistical Error Mode ± Detector Systematic ± Model Dependence B → Dlν 2.013 ± 0.074 ± 0.400 ± 0.511 ×10−2 B → D∗lν 6.096 ± 0.061 ± 0.611 ± 1.906 ×10−2 B → D∗∗lν 1.081 ± 0.072 ± 0.252 ± 0.421 ×10−2 −2 B → Xclν Nonresonant 0.862 ± 0.102 ± 0.318 ± 0.515 ×10

Table 4.3: |Vub| results Inferred partial branching fraction by region, ∆Bregion, with model dependence in the ﬁrst column. |Vub| calculated from the inferred partial branching fractions with all errors in the second column. The errors on |Vub| are statistical, detector, B → Xclν model dependence, B → Xulν model dependence, and theoretical uncertainty respectively. ∆Bregion± b → u Model Error |Vub|± Stat ± Detector ± b → c ± b → u ± Theory −3 −3 Total B(B → Xulν )(1.546 ± 0.743) × 10 (3.82 ± 0.17 ± 0.55 ± 0.23 ± 0.92 ± 0.12) × 10 2 2 −3 −3 q > 6GeV ,MX 8GeV ,MX < 1.7GeV (0.548 ± 0.069) × 10 (3.98 ± 0.18 ± 0.57 ± 0.24 ± 0.25 ± 0.38) × 10 2 2 −3 −3 q > 11 GeV ,MX < 1.5GeV (0.315 ± 0.032) × 10 (4.05 ± 0.18 ± 0.58 ± 0.25 ± 0.21 ± 0.56) × 10 2 2 −3 −3 q > (MB − MD∗ ) (0.297 ± 0.041) × 10 (4.07 ± 0.18 ± 0.58 ± 0.25 ± 0.28 ± 0.62) × 10 2 2 −3 −3 q > (MB − MD ) (0.359 ± 0.052) × 10 (4.05 ± 0.18 ± 0.58 ± 0.25 ± 0.29 ± 0.54) × 10

we are sensitive to the modeling of generic B decays in addition to the modeling of the detector response, and the modeling of the signal and background. For example we know that the neutrino resolution is strongly eﬀected by the number of undetected particles in the event such as extra neutrinos, KLs, and neutrons. It is necessary to reweight the Monte Carlo to reﬂect the uncertainty in the their number in generic B decays.

4.4.4 |Vub| Extraction

Once the branching fractions have been determined, we can extract from the B → Xulν component a value for |Vub|. We use calculations based on HQET and the OPE to extract |Vub| from the B → Xulν rate in restricted regions of phase space. This is a well controlled expansion and the theoretical uncertainties have been assessed by several authors [28, 29, 30]. However, the ﬁt’s region of sensitivity does not coincide with these regions. It is not possible to make cuts to isolate these regions, because 2 2 of the poor resolution on the neutrino four-vector and hence q and MX . In order to calculate |Vub|, we ﬁrst make a model dependent inference of the partial branching fraction in a region, and then apply the HQET and OPE calculations. This prescription is designed to minimize the reliance on models and instead rely on the controlled expansion in HQET and OPE calculations.

Table 4.3 shows the resulting values of |Vub| extracted for various different regions. Each result corresponds to the same set of fit results with the same models. The model is used to extrapolate/interpolate to the specified region, which is then converted to |Vub| using a theorectical model. These results are described in more detail in reference [3]. 4.5 Parity violation in B → γKππ 45

4.4.5 Moments

The second set of results of this analysis are moments of the kinematic variables of the B → Xclν differential decay rate. The branching fraction results are converted into moments using the models which were used in the fit. One particular set of moments that has become of interest recently is hadronic mass- squared moments as a function of lepton energy cut. For each value for the cut (starting at 1.0 GeV), the fit is repeated and a moment is calculated for that cut. The previous CLEO results was used a 1.5 ∗∗ GeV cut [31]. The D and nonresonant B → Xclν modes contribute contribute signifacantly less at 1.5 GeV than 1.0 GeV. Recently BABAR released a preliminary measurement the hadronic moments with a lower cut of 0.9 GeV [32]. The BABAR result does not agree with the HQET prediction. The CLEO result will cast some light on this discrepancy. Because this is a systematics limited result, the CLEO measurement will have a similar precision as BABAR result. The error depends mostly on how well the data are understood.

4.4.6 Summary

Lipeles’s analysis is in the ﬁnal stages of review by the collaboration. We are aiming for writing to twp papers based the results discussed above this summer. Both the |Vub| measurement and the moments are very relevant to the larger B physics goals In particular, the moments measurements provide precise + − tests of the predictions of HQET+OPE, which are used in the interpretation of B → Xsγ , B → Xsl l , and many other theoretical calculations in B physics.

4.5 Parity violation in B → γKππ

4.5.1 Introduction

Postdoc Adolf Bornheim is pursuing an analysis aiming to measure the decay rates, parity violating asymmetries, and ratios of weak form factors, in the decays B → γKππ (six different charge combinations). This is a measurement first proposed by Weinstein in 1999. The topic has drawn wider attention in recent years with a number of theoretical papers discussing this subject in more detail [33, 34, 35]. CLEO has made several pioneering measurements involving b → sγ [36, 31, 37, 38] and b → sg [39] transitions as well as related analysis which contribute to the interpretation of these measurements [40]. The inclusive branching ratio for b → sγ poses very stringent bounds on physics beyond the standard model [41]. The measurement of the exclusive rates, parity violating asymmetries, and ratios of weak form factors in the decay B → γKππ adds additional information to this very important subject. The parity violating asymmetry in particular is a very sensitive test of the standard model which is independent of the rate measurements of b → sγ decays. According to the standard model, the photon form the b → sγ is predominantly left-handed since the recoil s quark which couples to a W is left-chiral. Thus, the photons from B− and B¯0 decays are left-handed and the photons from B+ and B0 are right handed. Since the polarization of the photon from the b → sγ decay can not be measured directly, the parity violating asymmetries can only be probed by constructing a parity-odd observable using the decay products of the K-resonance recoiling against the photon. This can be done in 3-body decays of the K-resonance where the final state parti- ∗ cles stem from axial-vector decays such as K1(1400) → K π → Kππ. Using the K and one of the two pions of the Kππγ final state one can form a plane. The angle θH between the normal to this plane and the photon, as illustrated in Fig. 4.6, is a parity odd observable. 46 4 CLEO-II and CLEO-III

115.4 / 84 P1 398.5 3.040 n^ 350 P2 125.5 14.02 K π P3 114.9 12.80

2 #events P4 0.1905 0.7032E-02 K 300

π 250 1 π 2 200 Θ 150 HEL 100

γ 50

0 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 Θ cos H

Figure 4.6: The angle θH between the normal on Figure 4.7: The distribution for the angle θH from the plane formed by a K and a π from a K1(1400) a fast CLEO monte carlo simulation. In the sim- decay and the photon from the b → sγ is a parity- ulation parity violation was assumed to be maxi- odd observable. mal.

If the photon is in fact polarized as predicted by the standard model we expect the distribution of this angle to be asymmetric, while new physics might change this to a more symmetric shape, especially if left-right symmetric extensions of the standard model are considered. Weinstein implemented a Monte Carlo model of the decay chain B → K1(1400)γ → Kππγ which simulates the standard model parity violating effect in the b → sγ decay and interference effects in the K1(1400) decay which make the parity violation observable. The K1(1400) is not the only high mass K-resonance which occurs in b → sγ decays. CLEO made → ∗ the first observation of B K2 (1430)γ [42] as confirmed by the Belle collaboration which also reports on a first measurement of a Kππγ final state [43]. Possible contribution from other resonances are only poorly known. The focus of this analysis is to measure the exclusive decay B → γKππ for neutral and charged B and for all possible charge combinations of K and π, regardless of their resonant substructure. There are six modes to be considered which are listed in Table 4.4.

Mode B¯0/B0 B+/B− Table 4.4: B → Kππγ modes considered in this analysis. 1 K±π∓π0 ± ∓ Note that modes two and three are symmetric under parti- 2 Ksπ π 0 0 cle/antiparticle exchange. To use them in a parity violation 3 Ksπ π ± ∓ ± measurement one would have to tag the ﬂavor of the oppo- 4 K π π site B which is not intended for this measurement. However, 5 K±π0π0 ± 0 the measurement of their branching ratios would add valuable 6 Ksπ π information too. The other modes are self tagging.

Modes one, two, five and six are self tagging modes, which means that by reconstructing them we can identify the parent B flavor from either the charge of the kaon for modes one, four and five or from the charge of the pion in mode six. Modes two and three do not allow this since they are symmetric under particle/antiparticle exchange. To use these modes in a parity violation measurement one would have to identify the flavor of the parent B from the opposite B, taking into account mixing effects which would dilute the measured asymmetry. This adds significantly to the complexity of the analysis and is therefore not considered here. However, these modes have to be considered for the rate measurements.

Once a signal for these modes is established one can measure the distribution for the angle θH 4.5 Parity violation in B → γKππ 47

and ultimately extract the hadronic form-factors T1 and T2. These form-factors are related to the parity-violating asymmetry by :

dΓ+ − dΓ− 2hT1T2 AγK 1 = = 2 2 . (4.4) dΓ+ +dΓ− |T1| + |T2|

This asymmetry should be proportional to the expectation value of the angle θH , which in turn is related to helicity form factors H+,H0 and H− by : 2 dΓ 1+cosΘH 1 − cosΘH ∝ H+ + H0sinΘH + H− (4.5) dcosΘH 2 2

4.5.2 Monte carlo studies for the measurement of cos(θH ).

Theoretical calculations [35] and fast MC studies have shown that the proposed measurement is feasible. Detailed studies, based on full CLEO monte carlo, have been carried out to explore the sensitivity achievable in the actual experiment. Studies carried out so far indicate that taking all detector effects into account, the asymmetry should still be observable. Fig. 4.8 shows measured cos(θH ) distributions from a full CLEO monte carlo simulation, based on 20000 generated events. The selection and reconstruction is the same as applied to the CLEO data and will be described in Sec. 4.5.3. As can be seen from this example the asymmetry in the cos(θH ) is still present after including all detector resolutions, inefficiencies and particle misiden- tifications.

B- → γ K-π+π- B+ → γ K+π-π+ LH RH Figure 4.8: Simulated events of 225 Entries 6830 200 Entries 6733 71.17 / 51 40.94 / 51 ± → ± ∓ ± A0 85.27 1.982 A0 86.24 1.983 the decays B K π π . 200 A1 27.99 2.912 175 A1 -19.58 2.885 A2 102.8 5.739 A2 94.83 5.681 In the simulation maximal par- 175 150 ity violation was assumed. The 150 125 events were selected and re- 125 100 constructed as described in 100 Sec. 4.5.3. The distribution 75 75 was ﬁtted to a quadratic func- 50 50 tion. The second parameter is 25 25 proportional to the symmetry. 0 0 -1 -0.5 0 0.5 1 -1 -0.5 0 0.5 1 Θ Θ cos( HEL) cos( HEL)

4.5.3 CLEOII/II.V Data Analysis

− The analysis is based on 13.4 fb 1 of CLEO data taken on and just below the Υ(4S) resonance. The event selection requires a highly energetic photon, strict cuts on track and shower quality criteria, and at least one Kππγ candidate in any of the channels listed in Tab. 4.4 to be reconstructed in the event. On this preselected sample we impose cuts on ∆E and MB on every candidate and select the best candidate per event based on particle identification information dE/dx. To extract a signal from the remaining sample we employ a maximum likelihood fitting method. The fitter uses 6 variables: the beam constrained B-mass, the missing energy, the helicity of the B, the angle of the photon with respect 48 4 CLEO-II and CLEO-III to the beam, the angle of the kaon with respect to the photon, and a neural net variable with combines various event shape variables to discriminate against continuum backgrounds. The raw event yield, the selection efficiency and the significance of the signal as reported by the fitter is shown in Fig. 4.9.

0.16 0 0 0 8.8 0 66.68 7.71 0 50.03 9.11 0.02 21.15 4.81 8.2 18.56 17.13 11.15 0.03 0 0 0 1.06 0 4.6 0 4.04 0.69 0 17.5 100 CLEOII/II.V ON SIGNAL CLEOII/II.V OFF SIGNAL v42_8 15 CLEOII/II.V OFF XFEED CLEOII/II.V ON XFEED 80 12.5

10 60

7.5 40 # EVENTS (FIT YIELD) 5 # EVENTS (FIT YIELD)

2.5 20 111.2 2.85 0 0 -2.5 0 0 0 + + 246 0 π π π π π π + - 0 - 0 MODE - π π π π π π 0 0 0 + + S + S S K K K K K K →γ →γ →γ →γ →γ →γ 0 0 0 + + + B B B B B B Figure 4.9: Raw fit yields, selection efficiency for ON-resonance events and significance as reported by the maximum likelihood fitter. We find a significant yield in two modes, K±π∓π0 and K±π∓π±. To transform the raw yields into branching ratios we have to take into account several efficiencies. Besides the selection efficiency, which is listed in Fig. 4.9 for each mode, we have to consider the fitting efficiency, which is of order one, and the efficiency related to a cut on the mass of the kaonic resonance, which is also of order one but with a rather large uncertainty. The uncertainty on the latter, which is dominated by our poor knowledge of the mass spectrum in b → sg decays, is currently under investigation. It is of crucial importance since it will be a dominating contribution to the overall systematic uncertainties.

4.5.4 Summary

The measurement of the branching ratios is nearing completion. The analysis of the helicity structure is in progress. This measurement of cos(θH ) is one of the most interesting measurements in b → sγ transitions and will provide powerful constraints on the standard model. We hope to publish the branching ratios in summer 2003.

4.6 Analysis of the di-pion transitions in Υ Resonance Decays

The CLEO collaboration has embarked on studies of Υ resonances below the 4S state, using the state of the art CLEO III detector and software (CLEO III R). Data collection at the peaks of the Υ(3S) and Υ(1S) resonances are now complete. Analyses being performed on this data include the study of hadronic transitions between the diﬀerent Υ resonances. The expected bound state spectrum of the b¯b mesons (Fig. 4.10) is well understood in analogy 4.6 Analysis of the di-pion transitions in Υ Resonance Decays 49

3500 3000 2500 2000 1500 1000 500 0 9.3 9.4 9.5 9.6 9.7 9.8 9.9 10 10.1 ϒ →ϒ π+π- Data: (nS) (mS) Mrec 1400 1200 1000 800 600 400 200 0 9.3 9.4 9.5 9.6 9.7 9.8 9.9 10 10.1 ϒ →ϒ π0π0 Data: (nS) (mS) Mrec

Figure 4.10: LEFT: Spectrum of expected bound states of b¯b mesons. The observed states are indicated in bold. Observed transitions are via photons between Υ and χb states and via di-pions between different Υ states. Observation of the hb and ηb states may be within range of the CLEO III R efforts. RIGHT: Spectra of di-pion recoil mass from current CLEO Υ resonance data. The black and red histograms are before and after signal selection cuts, each summed over all 12 decay modes analyzed. The upper histogram is for charged pion modes, the lower for neutral pion modes. Peaks from the Υ(3S) → Υ(1S) ππ and Υ(2S) → Υ(1S) ππ transitions are clearly visible in both. For the Υ(3S) → Υ(2S) ππ transition only the neutral pion modes show a significant peak. with non-relativistic two body systems and QCD potential models for the qq¯ binding potential. Of the 3 3 expected states only the Υ (n S1)andχ (n PJ ) states have been observed. Efforts are under way at 1 CLEO to observe the D-states via their four photon cascades, as well as searches for the hb (n P1)and 1 ηb (n S0) via hindered E-M transitions. Transitions between states of the same angular momentum are allowed via the emission of pions, and efforts are under way to observe hb and ηb via these decays as well. Low energy transitions producing di-pions have been studied in the η, ψ, and Υ systems. The transition η → ηππ is well described by phase space, while the ψ → J/ψ π π (first observed in [44]) and Υ(2S) → Υ(1S) ππ decays require more detailed treatment. The ψ decays have been studied, yielding models in good agreement with the data [45, 46, 47]. The Υ(2S) → Υ(1S) ππ transition shows a similar mass structure. Both are explained by a transition matrix element requiring the di- pion system to be in an S-wave state. In contrast to this, the transition Υ(3S) → Υ(1S) ππ shows a strong enhancement at low di-pion mass that is not accounted for by these simple models. The CLEO collaboration has studied the transitions between the Υ(3S), Υ(2S), and Υ(1S) [48, 49, 50], as well as searching for such transitions starting from the Υ(4S) [51]. The transition energies between the resonances are small, so the decays cannot be treated by perturbative QCD. Due to these low energies, however, the transition amplitudes can be factorized:

M →1S|HT |3S ππ|Hh|0

The transition between the bottomonium states is described by the hamiltonian HT , while the production of the di-pion system is described by the hamiltonian Hh. The ﬁrst approach to calculating the transitions was by Gottfried [52] who proposed a multipole expansion of the gluon ﬁeld. The expansion is in terms of chromoelectric and chromomagnetic moments 50 4 CLEO-II and CLEO-III

of the meson’s quark distribution and the momentum of the outgoing system. However, this treats only the transitions of the heavy quark (QQ¯) system. This does not completely deﬁne the transition matrix element, but it does provide selection rules for the transitions and rough predictions for suppression factors. Transitions from S to S (Υ → Υ) states are allowed, while transitions S to P (Υ → χb) are forbidden. Expanding in terms of gluons, the lowest order allowed transition is a two E1 gluon transition. More recent work [53, 54, 55, 56] has attacked the second part of the matrix element, the production of the di-pion system. The transition matrix element is of the form X|αsE1aE1b|0 , where in our particular case X is the di-pion system. These calculations have generally predicted an S-wave structure for the di-pion system and a dominance of the high di-pion mass region. Since the parent and daughter Υ states are non-relativistic we can write the general Lorentz invariant amplitude in the form: 2 2 M∝ · A qππ − 2mππ + B (E1E2) + C ( · q1 · q2 + · q2 · q1)

2 2 where A, B,andC are slowly varying functions of the Dalitz variables mππ and mΥπ. The Υ polarization is preserved in the first two terms, while the third induces spin flips between the parent and daughter Υ. The third term is suppressed by the multipole transition selection rules. An anomalous enhancement of the Υ(3S) → Υ(1S) ππrate at low mππ can be accommodated by the coefficient B being of comparable magnitude to A with a phase shift to allow destructive interference at intermediate mππ. This is accompanied by a non-trivial angular structure of the di-pion kinematics. The CLEO III R data sets compose five run ranges. The first two were on the Υ(3S) and Υ(1S) resonances. The third was a scan at the Λb pair production threshold, followed by running on the the Υ(2S) resonance. The final run was on the Υ(5S) resonance. Offline reconstruction of the lower three resonance data sets has been completed and analysis is ongoing. Using data taken on the peak of the Υ(3S) resonance, we have access to three types of transitions: Υ(3S) → Υ(1S) ππ,Υ(3S) → Υ(2S) ππ, and Υ(2S) → Υ(1S) ππ (where the Υ(2S) is produced from the decay of the Υ(3S)). The daughter Υ is observed in its decays to both e+e− and µ+µ−.The two pions can be observed as π+π− or π0π0 Thus, we study twelve different final states. The current CLEO data sets yield roughly 2000 Υ(3S) → Υ(1S) π+ π− decays and about half as many decays involving π0 π0. The other decays (Υ(3S) → Υ(2S) ππ and Υ(2S) → Υ(1S) ππ) have yields of about 500 events each. In all cases, These sample sizes allow for detailed two dimensional fits to the di-pion helicity angle and mass (cos θππ and mππ) to extract the matrix elements A and B, and in principle C. By isospin symmetry the charged and neutral pion modes should have the same matrix element, so a simultaneous fit can be performed to the decay modes, extracting a single set of parameters. Hence there are four simultaneous fits that can be performed to each of the three Υ transitions. Fitting with the above amplitude improves the result by relating the kinematics varying rapidly with mππ and mΥπ and isolating the slowly varying behavior in the coefficients A and B. An earlier analysis by CLEO [50] of the Υ(2S) → Υ(1S) ππ transition indicates the presence of a D-wave contribution to the overall transition of a few percent, which in this formalism would be caused by the presence of the non zero B coefficient. The variation of the B coefficient with the Dalitz variables can provide information on intermediate resonances contributing to the decay dynamics. Postdoc Steve Pappas is completing an analysis of exclusively reconstructed hadronic transitions (Υ(3S) → ππΥ(nS), Υ(nS) → + −, where n = 1 or 2), together with Υ(2S) → ππΥ(1S) in this data set. The objective is to measure branching fractions, mass spectra, and more importantly, analyse the structure of the transition matrix elements to clarify the origin of the anomalous decay component. Past analyses have seen no significant contribution of other than S-wave di-pion states, but have not ruled out their presence, either. The large increase in sample size with the current data combined with the fit via the Lorentz invariant matrix element shows clearly the presence of higher angular momentum 4.6 Analysis of the di-pion transitions in Υ Resonance Decays 51

Daughter: Y(2S) Y(1S) Parent: Y(3S) Fit: Re(B/A) -0.377 +/- 0.235 -2.510 +/- 0.027 Im(B/A) 0.000 +/- 1.758 1.179 +/- 0.044

Y(2S) Fit: Re(B/A) -0.619 +/- 0.099 Im(B/A) 0.431 +/- 0.071

Table 4.5: Fitted matrix elements for the three hadronic transitions between Υ states. The fits are performed with A fixed to 1. The anomaly in the transitions appears here as a non zero value of B/A, clearly visible for the Υ(3S) → Υ(1S) ππ mode. The fits were performed simultaneously for both ee and µµ decays of the daughter Υ and both charge states of the di-pions.

0.09 0.35 0.08 0.2 0.07 0.175 0.3 0.06 0.15 0.25 0.125 0.05 0.2 0.04 0.1 0.15 0.03 0.075 0.1 0.02 0.05 0.01 0.025 0.05 0 0 0 0.5 0.75 0.26 0.3667 0.4733 0.58 0.26 0.2867 0.3133 0.34 + - + - 0 0 ϒ(3S)→ϒ(1S)π π : Mππ ϒ(2S)→ϒ(1S)π π : Mππ ϒ(3S)→ϒ(2S)π π : Mππ

Figure 4.11: Projections of efficiency corrected data and fit results; histograms are normalized to unit area. Only the two highest yield modes are shown for each transition. For 3S → 1S and 2S → 1S these are charged pion modes, while for 3S → 2S they are neutral modes. Red and blue points are respectively di-muon and di-electron final states. components (B = 0). Careful analysis of the structures of the coefficients A and B could yield evidence for intermediate states such as the σ resonance or Υ π resonances should they be present. The matrix elements extracted in this analysis are summarized in Table 4.5. Projections of efficiency corrected data and the corresponding fit results are shown in Figure 4.11 for the two highest yield modes for each transition. The fits show very good agreement between the data and the expected shape for phase space modulated by the Lorentz invariant matrix element with coefficients from Table 4.5. These results are being prepared for presentation at the summer 2003 conferences, and publication shortly thereafter. 52 4 CLEO-II and CLEO-III

Bibliography

0 + − ∗± ∓ [1] “Observation of B → KSπ π and Evidence for B → K π ”E.Eckhartet al. (CLEO Collab- oration), Phys.Rev.Lett.89, 251801 (2002).

[2] “Measurement of the Charge Asymmetry in B → K∗(892)±π∓”, B.I. Eisenstein, et al.(CLEO Collaboration), CLNS 103/1823, CLEO 103-07 (2003), accepted by Physical Review D.

[3] “Preliminary results on |V(ub)| from inclusive semileptonic B decays with neutrino reconstruction,” A. Bornheim et al. [CLEO Collaboration], arXiv:hep-ex/0207064, (2002).

[4] “Parity Violation in Exclusive B → γKππ Decays”, A. Bornheim (representing the CLEO Collab- oration), talk presented at Meeting of the Division of Particles and Fields 2002 of the American Physical Society, May 24 2002, Williamsburg, Virginia.

[5] “Measurement of Lepton Momentum Moments in the Decay B¯ → X ν¯ and Determination of Heavy Quark Expansion Parameters and |Vcb|”, A.H. Mahmood et al.(CLEO Collaboration), CLNS 102/1810, CLEO 102-16 (2002), accepted by Physical Review D.

[6] “CLEO contributions to tau physics”, A. Weinstein (representing the CLEO Collaboration), in the Proceedings of the Seventh International Workshop on Tau Lepton Physics (TAU02), Santa Cruz, Ca, USA, Sept 2002, to be published in: Nuclear Physics B.

[7] “Vub and Vcb from CLEO ”, A. Bornheim (representing the CLEO Collaboration), talk presented at XXXVIII Rencontres de Moriond - Electroweak Interactions and Uniﬁed Theories , March 21 2003, Les Arcs, France.

[8] H.J. Lipkin, Phys. Lett. B72, 249 (1977); M. Suzuki, Phys. Rev. D47, 1252 (1993); H.J. Lipkin, Phys. Lett. B303, 119 (1993); E.L. Berger and H.J. Lipkin, Phys. Lett. B189, 226 (1987).

[9] M.P. Worah hep-ph/9711265; D. Guetta Phys.Rev. D58 (1998) 116008; G. Barenboim et al. Phys.Rev.Lett. 80, 4625 (1998).

[10] S. Chen et al. (CLEO Collaboration), “Measurement of Charge Asymmetries in Charmless Hadronic B Meson Decays” Phys. Rev. Lett. 85, 525 (2000).

[11] R.Godang et al., Phys. Rev. Lett. 80, 3456 (1998); D. M. Asner et al. (CLEO Collaboration), Phys. Rev. D 53, 1039 (1996); F. W¨urthwein, Cornell Ph.D. Thesis, 1995; M. Battle, et al.,Phys. Rev. Lett. 71, 3922 (1993).

[12] S.J. Richichi et al. “Two-body B Meson Decays to eta and eta’ : Observation of B → ηK∗”Phys. Rev. Lett. 85, 520 (2000); C. P. Jessop et al. (CLEO Collaboration), “Study of Charmless Hadronic B Meson Decays to Pseudoscalar-Vector Final States”, CLNS 99/1652.

[13] D. Cronin-Hennessy et al. (CLEO Collaboration), “Study of Two-Body B Decays to Kaons and Pions”, Phys. Rev. Lett. 85, 515 (2000).

[14] N. G. Deshpande and J. Trampetic, Phys. Rev. D 41, 895 (1990); L.-L. Chau et al.,Phys.Rev.D 43, 2176 (1991); A. Deandrea et al., Phys. Lett. B 318, 549 (1993); A. Deandrea et al., Phys. Lett. B 320, 170 (1994); G. Kramer, W. F. Palmer, and H. Simma, Z. Phys. C 66, 429 (1995); G. Kramer and W. F. Palmer, Phys. Rev. D 52, 6411 (1995); D. Ebert, R. N. Faustov, and V. O. Galkin, Phys. Rev. D 56, 312 (1997); D. Du and L. Guo, Zeit. Phys. C 75, 9 (1997); N.G. Deshpande, B. Dutta, and S. Oh, Phys. Rev. D 57, 5723 (1998); H. Cheng and B. Tseng, Phys. Rev. D 58, (1998) 094005; A. Ali, G. Kramer, and C. L¨u, Phys. Rev. D 58, (1998) 094009; A. S. Dighe, M. Gronau BIBLIOGRAPHY 53

and J. L. Rosner, Phys. Rev. D 57, 1783 (1998); M. Ciuchini, R. Contino, E. Franco, G. Martinelli and L. Silvestrini, Nucl. Phys. B 512, 3 (1998); N. G. Deshpande, B. Dutta and S. Oh, ar meson,” Phys. Lett. B 473, 141 (2000); M. Gronau and J. L. Rosner, Phys. Rev. D 61, 073008 (2000); J. L. Rosner, Nucl. Instrum. Meth. A 462, 44 (2001); C. W. Chiang and J. L. Rosner, Phys. Rev. D 65, 074035 (2002); Y. -Y. Keum, H. Li, and A. I. Sanda, hep-ph/0201103;D.Duet al., hep-ph/0201253.

[15] C. W. Chiang and J. L. Rosner, Phys. Rev. D 65, 074035 (2002) [arXiv:hep-ph/0112285].

[16] B. Aubert et al. [BABAR Collaboration], arXiv:hep-ex/0207068.

[17] M. Gronau and J. L. Rosner, Phys. Rev. D 61, 073008 (2000) [arXiv:hep-ph/9909478].

[18] M. Gronau, arXiv:hep-ph/0001317.

[19] R. Fleischer, Z. Phys. C 62, 81 (1994).

[20] N. G. Deshpande and X. G. He, Phys. Lett. B 336, 471 (1994) [arXiv:hep-ph/9403266].

[21] M. Beneke, G. Buchalla, M. Neubert and C. T. Sachrajda, Phys. Rev. Lett. 83, 1914 (1999) [arXiv:hep-ph/9905312]; H. n. Li, arXiv:hep-ph/9903323; H. Y. Cheng and K. C. Yang, Phys. Rev. D 62, 054029 (2000) [arXiv:hep-ph/9910291]; M. Bander, D. Silverman and A. Soni, Phys. Rev. Lett. 43, 242 (1979).

[22] M. Suzuki and L. Wolfenstein, Phys. Rev. D 60, 074019 (1999) [arXiv:hep-ph/9903477].

[23] M. Neubert, arXiv:hep-ph/0207327.

[24] A. Hocker, H. Lacker, S. Laplace and F. Le Diberder, Eur. Phys. J. C 21, 225 (2001) [arXiv:hep- ph/0104062]. For most recent ﬁt results, see http://www.slac.stanford.edu/∼laplace/ckmfitter/ckm wellcome.html.

[25] M. Beneke, arXiv:hep-ph/0207228.

[26] R. Fleischer and J. Matias, Phys. Rev. D 66, 054009 (2002) [arXiv:hep-ph/0204101].

[27] R. Barlow and C. Beeston , Computer Physics Communications. 77, 219-228 (1993).

[28] A. F. Falk, Z. Ligeti and M. B. Wise, Phys. Lett. B406, 225 (1997)

[29] M. Neubert, JHEP 0007, 022 (2000) M. Neubert and T. Becher, Phys. Lett. B535, 127 (2002)

[30] C. W. Bauer, Z. Ligeti and M. E. Luke, Phys. Rev. D64, 113004 (01).

[31] “Hadronic Mass Moments in Inclusive Semileptonic B Meson Decays”, CLEO Collaboration (D. Cronin-Hennessy et al.), Phys. Rev. Lett. 87, 251808 (2001).

[32] B. Aubert et al. [BABAR Collaboration], arXiv:hep-ex/0207084, (2002).

[33] M. Gronau et al., “Measuring the Photon Polarization in B → Kππγ”, Phys. Rev. Lett. 88, 051802 (2002).

[34] “Penguins in B decays.”, M. Gronau, TECHNION-PH-2001-37, hep-ex/0110316, (2002).

[35] “Photon Polarization in radiative B decays.”, M. Gronau, D. Pirjol, TECHNION-PH-2002-18, UCSD-PTH-02-10, hep-ex/0205065, (2002). 54 4 CLEO-II and CLEO-III

[36] “Branching Fraction and Photon Energy Spectrum for b → sγ”, CLEO Collaboration (S. Chen et al.), Phys. Rev. Lett. 87, 251807 (2001).

[37] “Study of exclusive radiative B meson decays”, CLEO Collaboration (T. E. Coan et al.), Phys. Rev. Lett. 84, 5283 (2000). “CP asymmetry in b → sγ decays”, CLEO Collaboration (T. E. Coan et al.), Phys. Rev. Lett. 86, 5661 (2001).

[38] “Improved Upper Limits on the FCNC Decays B → K + − and B → K∗(892) + −”, CLEO Collaboration (S. Anderson et al.), CLNS 101/1739, CLEO 101–11 (submitted to Phys. Rev. Lett.).

[39] R. A. Briere et al. (CLEO Collaboration), “Observation of B → φK and B → φK∗”, Phys.Rev.Lett.86, 3721 (2001).

[40] A. Bornheim et al. (CLEO Collaboration), “Improved measurement of |Vub| with inclusive semileptonic B decays”, Phys.Rev.Lett.88, 231803 (2002).

[41] P. Gambino, “Radiative B decays : Electroweak eﬀects and new physics beyond the leading order.”, J. Phys. G 27, 1199 (2001).

[42] T. E. Coan et al. (CLEO Collaboration), “Study of Exclusive Radiative B Meson Decays”, Phys.Rev.Lett.84, 5283 (2000).

[43] S. Nishida et al. (Belle Collaboration), “Radiative B Meson decays into Kπγ and Kππγ Final States.”, BELLE-PREPRINT-2002-10, hep-ex/0205032 (2002).

[44] “Decay of ψ(3684) into ψ(3095)” G. S. Abrams, et al. PRL 34:1181 (1975)

[45] “Chiral Symmetry and ψ → ψππ Decay” L. S. Brown and R. N. Cahn PRL 35:1 (1975) [46] “Angular distributions in the decay ψ → ψππ” R. N. Cahn PRD 12:3559 (1975)

[47] “ψ → ψππ decay as test of partial conservation of axial-vector current” D. Morgan and M. R. Pennington PRD 12:1283 (1975)

[48] “High-statistics study of Υ(2S) → π+ π− Υ(1S)” D. Besson et al. PRD 30:1433

+ − [49] “Study of π π transitions from the Υ(3S) and search for the hb”I.C.Brocket al PRD 43:1448

[50] “Hadronic transitions Υ(2S) → Υ(1S)” J. P. Alexander et al. PRD 58:052004

[51] “Υ di-pion transitions at energies near the Υ(4S)” S. Glenn et al. PRD 59:052003

[52] “Hadronic Transitions between Quark-Antiquark Bound States” K. Gottfried, PRL 40:598 (1978)

[53] “Measuring Quantum-Chromodynamic Anomalies in Hadronic Transitions between Quarkonium States” M. Voloshin and V. Sakharov PRL 45:688

[54] “Hadronic transitions between heavy-quark states in quantum chromodynamics” T.-M. Yan PRD 22:1652

[55] “predictions for hadronic transitions in the b¯b system” Y.-P. Kuang and T.-M. Yan PRD 24:2874

[56] “Coupled-channel eﬀects in hadronic transitions in heavy-quarkonium systems” H.-Y. Zhou and Y.-P. Kuang PRD 44:756 5. CMS at LHC and L3 at LEP2

E. Aslakson, A. Bornheim, J. Bunn, D. Collados, G. Denis, P. Galvez, M. Gataullin, S. Iqbal, I. Legrand, T. Lee, V. Litvin, R. Mao, D. Nae, H. Newman, S. Pappas, S. Ravot, S. Shevchenko, S. Singh, C. Steenberg, M. Thomas, F. Van Lingen, R. Voicu, K. Wei, A. Weinstein, R. Wilkinson, X. Yang, L. Zhang, K. Zhu, R.Y. Zhu

5.1 Executive Summary

The principal physics aims of the CMS group are to search for new particles and interactions, and to study the fundamental electroweak interaction up to the TeV mass scale. LEP, whose last run was completed in November 2000, yielded hints of a Higgs boson with a mass of approximately 115 GeV. With the end of our 22 year committment to L3 this year, we are focusing our efforts on CMS and MINOS. The theses of our graduate students on L3 are now being completed: X. Lei finished his search for supersymmetric leptons in the Fall of 2002, and M. Gataullin’s search for anomalous couplings and other new physics using single and multi-photons will be completed by this Fall. We have correspondingly continued to increase our analysis efforts on Higgs searches with CMS at LHC. As reported in the MACRO/MINOS chapter of this Annual Report, MINOS will make definitive measurements of the oscillations of high energy muon neutrinos. The results from SuperKamiokande, SNO, KAMLAND, and MACRO on atmospheric, solar and reactor-generated neutrinos indicate strongly that MINOS offers an exciting opportunity to explore this new field of physics beyond the Standard Model. In particular MINOS will be the focus of the work of our graduate students between this Fall and 2007. The Caltech CLEO group plans to complete their CLEO commitments by the end of calendar 2003. This group (A. Weinstein and postdocs Bornheim and Pappas) are beginning to shift their focus to CMS physics, and joined the US CMS collaboration by merging with the Caltech CMS group in 2002. Bornheim shifted full time to CMS in 2002, and Weinstein and Pappas will shift their main focus to CMS by this Fall. The CMS budget for FY2003 and the request for FY2004 reflect this transition. In joining the CMS group, Weinstein and the two postdocs working with him are particularly interested in developing the electron and photon detection and identification capabilities of the CMS detector, using the calorimeters together with the tracker, in the search for new physics. The group has taken responsibilities in the integration and maintenance of the Caltech-built CMS ECAL Monitoring System at CERN, and A. Bornheim moved to CERN full time in the Fall of 2002 for this purpose. Over the next five years, we therefore intend to balance our efforts between (1) completing our searches for new physics with L3 data by Fall 2003, (2) completing and operating the laser-based monitoring system for CMS’ precision crystal electromagnetic calorimeter (ECAL), (3) studying a broad range of new physics searches involving photon and/or lepton signatures at the LHC, using the full capabilities of the CMS precision electromagnetic calorimeter and silicon tracker (4) studying neutrino oscillations with MINOS (5) developing and constructing the computing, networking and Grid software systems for CMS at the LHC.

55 56 5 CMS at LHC and L3 at LEP2

5.1.1 L3 at LEP

Completion of L3 at LEP

The LEP2 data at center of mass energies of up to 208 GeV, is being used to obtain final results on the searches for new particles, and reviewed in Section 5.2.3, building on the work of A. Favara and S. Shevchenko who led these efforts throughout the LEP2 program. L3’s clean identification and high resolution for photons, coupled with our group’s analysis and calibrations of the BGO crystal calorimeter with the RFQ calibration system1, have led to L3 obtaining the world’s best limits on the masses of several new particles. These include (1) the lightest neutralino (39.4 GeV; independent of m0), (2) new sequential or excited heavy leptons, (3) heavy singlet neutrinos and (4) the lightest gravitino, in the scenario where the gravitino is the Lightest Supersymmetric Particle (LSP). We were also able to search for large extra dimensions of spacetime, through the direct production of gravitons accompanied by a photon, and exclude effective gravitational scales up to the 1 TeV range. The L3 W-pair sample of 10,500 events, along with our final samples of single-W, WWγ and events with one or two photons accompanied by neutrinos, were used to obtain significantly improved determinations of the W mass, branching ratios, and the limits on triple- and quartic anomalous couplings. The work on quartic anomalous couplings, pioneered in L3, depends on major contributions by student M. Gataullin. L3 also has been the first to use single W production to cleanly test the γWW vertex and disentangle the ZWW and γWW couplings. From the LEP1, LEP2, and SLD electroweak data, the W boson data, and the determination of the top mass, combined with two loop electroweak calculations, the 95% CL upper limit on MH is now 211 GeV. The work of our group also allowed us to improve the limits in searches for SUSY leptons, including a limit of 80 GeV obtained by student X. Lei on the mass of the scalar tau.

Caltech’s Roles and Responsibilities in L3

Over the last 20 years, Caltech shared the major US responsibilities in the experiment with MIT, Princeton, and CMU: on the data analysis, including leadership of 2 of the 3 analysis groups on new particle searches during LEP2, on oﬄine computing and networking (including transatlantic networking for HENP) and on the RFQ system that was the key to L3’s precision measurements of electrons and photons throughout LEP2. The Caltech group’s leading role in the data analysis is reﬂected in the fact that of 271 L3 publications, 39 (14%) are based on principal contributions by members of our group, and an additional 40 (15%) report on analyses done under our leadership or coordination.

BGO Calibration with the RFQ System

The RFQ system was a key factor in enabling L3 to fully exploit the high resolution and clean photon identiﬁcation capabilities of the BGO throughout the LEP2 program. In addition to developing and constructing the RFQ, Caltech’s BGO analysis using the RFQ data (A. Favara, M. Gataullin) reached a calibration precision of 0.5% for the BGO barrel and 0.4% for both endcaps (see Section 5.2.4). This gave L3 enhanced sensitivity to new physics, especially in SUSY and other event signatures containing one or more isolated photons. This is fully exploited in M. Gataullin’s thesis. The experience we have gained in calibrating ans analyzing the data from the L3 BGO at LEP is proving to be very useful in calibrating the CMS Lead Tungstate calorimeter at the LHC, and achieving clean photon and electron identiﬁcation in CMS.

1See Section 5.2.4. 5.1 Executive Summary 57

5.1.2 CMS At LHC

CMS is designed to cover the full range of high pt physics at the LHC, up to the highest luminosities. The CMS design stresses the clean identification and precision measurement of photons, electrons and muons, with a resolution of better than 1% for these particles over a large momentum range. The high precision lead tungstate crystal electromagnetic calorimeter (ECAL) is designed to give CMS superior discovery potential for the Higgs in the critical mass range up to 170 GeV, through its γγ and 4-lepton decay modes. This is complemented by CMS’ hermetic calorimetry, enabling CMS to detect the Higgs ∗ through jets and missing ET signatures resulting from Higgs decays to WW . Using its features for jets and missing ET together with precision lepton measurements, CMS could discover Supersymmetry: in the cosmologically interesting region (where the lightest neutralino contributes to dark matter) in the first weeks of LHC running, or by detecting squarks and gluons up to the 1.5 to 2 TeV mass range within one year at low (one-tenth of design) luminosity. The special capability of CMS to measure H → γγ depends on the resolution of its ECAL, and hence on its accurate calibration during running at the LHC. Caltech has had key roles in the design, development and monitoring of the ECAL, as well as photon identification and reconstruction, and thus in the realization of this important part of the CMS physics program. This year we have extended our studies to Higgs production through vector boson fusion (VBF) that leads to γγ final states accompanied by two high-rapidity jets. This provides a complementary channel to the inclusive γγ search, with a signal/background ratio that could provide information on the production dynamics in the early stages of a Higgs discovery. The CMS Collaboration currently includes 1940 physicists and engineers from 159 universities and laboratories in 36 countries. The US CMS Collaboration, led by Caltech, includes more than 400 members from 38 institutions in US CMS, making the US the largest national contingent. US CMS has full responsibility to build the hadron calorimeter and forward muon systems, and major responsibilities to contribute to the development of the crystals and readout of the precision crystal electromagnetic calorimeter, as well as the trigger and data acquisition systems, and forward pixels. With the adoption of an all-silicon tracker as part of the CMS baseline, UC Santa Barbara, Kansas, and Kansas State joined US CMS, and these groups together with Fermilab are playing and will play a major role in the production of the silicon tracker modules. US groups also are leading the CMS Computing and Core Software efforts, as well as international networking for CMS and HENP at large, and have responsibility for the forward pixel tracking systems. In the past year Florida International University (FIU) and Yale joined US CMS and CMS. UERJ (Rio de Janeiro) also joined CMS, leading a consortium of Brazilian universities, with a special relationship with Caltech and FNAL for the development of inter-regional Grid and network systems, as well as education and outreach. The US CMS Construction Project is approximately 80% complete as of mid-2003. The Common Projects subproject was completed in 2002 and the US ME (forward muon), HCAL and DAQ subpro- jects are on schedule and within budget. The entire HCAL barrel (HB) has been delivered to CERN, and assembled in two halves. All 148 of the cathode strip ME chambers scheduled to be assembled at Fermilab by the Spring of 2003 have been completed, and 45 were sent to CERN after being commissioned at the US Final Assembly and Test (FAST) sites. The DAQ TDR was completed on schedule in the Fall of 2002, and the Level 1 and higher level trigger (HLT) rates have been understood, along with the online CPU requirements which were found to be within the range of the early online requirements estimates. The DAQ system has been redesigned in 8 identical “slices”, enabling US CMS to deliver (in 2005) a one slice system capable of supporting the initial CMS runs at low luminosity. For the US parts of the ECAL, the issue of radiation hardness of the APDs has been resolved and the APDs deliveries are now on schedule: 85k out of the total of 130k APDs were already delivered by the Spring of 2003. 58 5 CMS at LHC and L3 at LEP2

A major development in US CMS in 2002 was the appointment of Dan Green as Manager of the US CMS Research Program Manager, which covers the Construction and Software and Computing Projects, as well as the Maintenance and Operations (M&O), and upgrades of the experiment. The move to a unified program management, strongly supported by the US CMS Collaboration, has already had substantial benefits in the fact that the rate of Construction Project completion has been extended, to provide additional funding to maintain the level of effort in US CMS Software and Computing during 2002-2004. Caltech’s laser-based ECAL monitoring system is on schedule and within cost. The first laser was delivered and installed on schedule in August 2001, and has been successfully used at the CERN test beam since then. The second laser is now under construction and commissioning at Caltech. A third laser (at a different wavelength) was added this year to the scope of the Caltech program, in response to a Lehman review, and is also being constructed at Caltech. The Caltech group has many leading and other key roles in the CMS Collaboration and in US CMS. We have been instrumental, together with Fermilab, in developing and launching the US CMS Software and Computing Project (1998-2001). Our technical roles in CMS are centered around the computing, networking and software, the electromagnetic calorimeter (ECAL) crystal development and monitoring, and we contributed to the optimization of the ECAL design, as well as the tracker design for Higgs and other new particle searches. These roles are founded in earlier work at the former SSC physics program, and at LEP and PETRA since 1978. In addition, the Caltech group has accumulated 20 years of experience and expertise on the development of precision crystal calorimeters, including the L3 BGO calorimeter (since 1983), the BaF2 crystal calorimeter for GEM (1988-1993), and ongoing studies of new high-density crystal scintillators for HENP experiments since 1988. Caltech’s responsibilities and contributions to the development of CMS, and to the LHC program as a whole are summarized below.

• H. Newman was elected to a third two year term as Collaboration Board (CB) Chair of US CMS for 2002-4. As CB Chair of the US CMS Advisory Board and the US representative on the international CMS Management Board. He is a member of the (US CMS and ATLAS and accelerator joint) LHC Collaboration Steering Group that was formed in 2003. He was also Chair of the CMS experiment’s Software and Computing Board (1996-2001).

• R. Zhu has major responsibilities for the development and calibration of the CMS high precision electromagnetic calorimeter (ECAL). This work has had a major impact on the development of rad-hard mass produced crystals, testing and calibration methods, and the development of CMS’ laser-based optical monitoring system (done in collaboration with JPL and Saclay).

• J. Bunn is a Level 3 Manager in charge of the Caltech/UCSD Tier2 in the US CMS Software and Computing Project. Together with Newman he leads the eﬀorts to develop the ﬁrst ”Grid- enabled” working environments for data analysis (GAE) for CMS, for the LHC program as a whole and for other major HENP experiments. We are building a strong development team at Caltech (Steenberg, Aslakson, Van Lingen, Thomas) and CERN (Legrand, Iqbal) for this purpose.

• S. Shevchenko, R. Wilkinson, V. Litvin, T. Lee and H. Newman are carrying out in-depth simulation studies aimed at understanding and optimizing CMS’ capability to detect and study Higgs bosons in the mass range up to 170 GeV. These studies have utilized our prototype Tier2 center and some of the other unique computing facilities installed at Caltech’s Center for Advanced Sci- entiﬁc Computing Research (CACR), along with a large computing cluster at NCSA in Illinois, and other national computing facilities made available through large grants of computing time to our group. This has allowed us to fully simulate the large multi-jet and γ+jet backgrounds to the H → γγ signal (several times 106 events preselected from a generated sample of nearly 5.1 Executive Summary 59

1010 events) for the first time. These studies will expand and will use the TeraGrid starting later this year. To this end, V. Litvin has successfully ported the necessary CMS codes to the 64-bit architecture (running under Linux), and run the first large scale scientific production task on the TeraGrid. The codes have been selected as a TeraGrid ”FlagShip” application.

• With the change to an all-Silicon tracker, we began work on photon reconstruction in the tracker in 2001. Full reconstruction of photons (including photons converted in the tracker) has been a major ongoing Caltech responsibility in the CMS software. This eﬀort will be strengthened, and broadened to include electron as well as photon reconstruction, with the work of Bornheim and Weinstein on CMS starting later this year.

• Staﬀ scientist R. Wilkinson has taken a leading role in developing endcap muon DAQ software for the 2004 slice tests. The Slice Test DAQ project for the CMS endcap muon detector is an important component of the US CMS endcap muon project, and is key to commissioning the CMS detector itself. Working with the US CMS DAQ group and the Endcap muon group, he completed a prototype data acquisition software that will shortly be tested. This involved redesigning existing C code that talks to low-level hardware into C++ object oriented code, collecting the resulting chamber and trigger data into an event builder, and deﬁning the output format. This is an important contribution to the long term viability of the EMU system in CMS since the previous code was not maintainable whereas the new version of this code will serve as the basis of the DAQ software for the EMU system for the next decade.

• Software engineer V. Litvin has taken charge of, and continues to improve the speed and structure of the CMS HCAL code, working together with Princeton faculty member (and former Caltech/L3 student) C. Tully.

• We have established a leading role in the planning, modeling and design and realtime monitoring of the computing, networking, database and Grid software systems for CMS, and the LHC in general. We initiated and now co-lead the DOE-funded Particle Physics Data Grid (PPDG), and the NSF- funded Grid Physics Network (GriPhyN) and International Virtual Data Grid (iVDGL) projects. This work is in collaboration with the leading computer scientists (Foster, Livny, Kesselman) who founded the Grid concept, and have made substantial contributions to the formulation and the work on data management and high performance networking in the European Data Grid Projects. With the launching of the LHC Computing Grid project at CERN in March 2002, we have been asked to assume a leading role in developing the distributed LHC Computing Model, the technology projections for networks, and Grid-enabled analysis for the LHC experiments.

• We installed and commissioned the ﬁrst prototype Tier2 center at Caltech/CACR and SDSC in 2001, working together with the UCSD CMS group. Since then we have been upgrading it progressively according to plan, with US CMS S&C Project and iVDGL funds. The Tier2 center contributes to CMS productions of simulated and reconstructed events, and serves an avid user community of more than 30 physicists at the US CMS universities in California working in the muon, jets and missing ET and electron/gamma PRS groups. It is also used to do R&D on the conﬁguration, operation and management of such a center, in coordination with the Fermilab prototype Tier1 center, the Tier2 prototype in Florida, and a Condor pool in Wisconsin. We have also participated to the development and testing of Grid-enabled production systems for CMS, and to development and deployment of GriPhyN’s “Virtual Data Toolkit” (VDT) in the context of the US CMS Test Grid.

• We continue our work with CMS groups in Brazil, China, India, Korea, Pakistan, and other countries to help with planning and designing similar Tier2 (or Tier1) facilities and with related work on Grids, to be used as part of the worldwide CMS data analysis eﬀort. We organized 60 5 CMS at LHC and L3 at LEP2

a session on “Grids and Information Technology” in Romania in the Spring of 2002, at the request of that government. This led in turn to a session on the “Role of New Technologies in the Formation of the Information Society” at the Pan European Ministerial Conference last November, in preparation for the World Summit on the Information Society (WSIS) in Geneva (December 2003). We are currently working with CERN, and separately with the US State Department’s Telecommunications Advisory Council, on preparations for the WSIS that highlight the role of the scientiﬁc community, and HENP in particular. In the Fall of 2002 we started a bilateral collaboration with computer scientists and masters’ students at the Polytechnica University in Bucharest. This is synergistic with our continuing work on Grid systems and software with the HEP and computer science groups in Pakistan.

• We initiated and led the “MONARC” joint project of the LHC experiments2, whose aim has been to study and simulate the LHC experiments’ network-distributed Computing Models, and the structure and roles of Regional Centers (including Tier1 US Centers at FNAL at BNL) in these Models. This project, led by H. Newman of Caltech as Spokesman3 has provided key input on the requirements for LHC Computing. The main deliverable of this project has been the MONARC simulation system, which is currently being used for continued studies and development of the LHC Computing Model, and for better determining the computing and operational requirements for running data-intensive Grids for the LHC experiments. This system is principally the work of software engineer I. Legrand of our group who now leads a team of developers from Caltech, CERN and Bucharest.

• In 2001-3 Legrand developed a scalable distributed architecture adapted to building vertically integrated Grid systems for physics experiments. Initial applications include a realtime Grid and network monitoring system4 that is now monitoring the Tier1 and Tier2 prototypes in the US, CERN, Bucharest, Taiwan and Islamabad, and is being applied to optimize the interconnections among the VRVS reﬂectors. This system, together with novel “self organizing neural network” optimization techniques we are applying to distributed systems, are expected to drive further development of the LHC Computing Model, including user guidelines and strategies for Grid- enabled analysis (GAE). This system is currently being adopted by Internet2 (by its End-to-end Performance Initiative), and will be further developed by Caltech and Michigan to monitor and help manage the Abilene backbone and its connections to 200+ universities.

• In the Fall of 2001 CMS decided to change its database strategy, moving away from Objectivity in favor of a hybrid solution, which is currently being evaluated. The hybrid consists of Relational Database Management Systems (RDBMS) for metadata, and a combination of approaches for the bulk data, which includes the use of ROOT ﬁles and traditional RDBMS like Oracle 9i, SQLServer, MySQl and PostGreSQL. For example, objects stored in Oracle 9i will be interfaced to CMS software through an object-streaming service and will be accessed by users doing analysis through a well-developed front end such as ROOT. Building on our experience in the HP-funded GIOD project (1996-2001), and using funds from our ALDAP project (Accessing Large Databases in Astronomy and Particle Physics, 1999-2002), we added a database expert to our group in 2001 (Iqbal). This has allowed us to evaluate Oracle 9i and its object interface, as part of the CMS transition to a new database strategy. We have built Web services to Oracle 9i and other databases, in concert with our development of the GAE.

• In 2001 our group (C. Steenberg) began development of a Remote Analysis System server system ‘CLARENS” in the PPDG project, to enable users to eﬃciently carry out CMS analysis with full

2MONARC: Models of Networked Analysis at Regional Centres. See http://www.cern.ch/MONARC. 3Together with L. Perini of ATLAS. 4MonALISA: Monitoring Agents in a Large Integrated Services Architecture 5.1 Executive Summary 61

access to CMS software on the server, while running a lightweight client on any one of a variety of platforms. In 2002, given the progress in the development of Grid tools, and recognizing a growing need for users working with CMS simulated data, we started developing the first prototypes of a “Grid-enabled Analysis Environment” (GAE). This led to an effort on “Analysis Tools” in the PPDG project, under the leadership of J. Bunn of our group, and to NSF/ITR proposals on the GAE and Collaborative systems by Caltech, together with other universities in US CMS and US ATLAS (CAIGEE: CMS Analysis - an Interactive Grid Enabled Environment, funded in 2002: GECSR: A Global Grid-enabled Collaboratory for Scientific Research, submitted in March 2003).

As the principal partner in the US-CERN (and Canada, IN2P3 and the UN) consortium managing and operating the transatlantic link for HENP, we have established longstanding key roles in networking for our ﬁeld5. This role expanded substantially in 2002-3, when H. Newman was appointed as Chair of the ICFA Standing Committee on Inter-regional Connectivity (SCIC), giving Caltech a primary role in planning, developing and co-managing wide area networks for HENP and the broader scientiﬁc community. During 2002-3, in cooperation with the CERN DataTAG project, the Caltech Computer Science department, SLAC, Los Alamos and others, we carried out a number of key developments that have brought us (and HENP) to the leading edge of wide area networking6.

• In 2000-2001 H. Newman co-chaired the Transatlantic Network Working Group (with L. Price of Argonne) commissioned by the DOE/NSF LHC Joint Oversight Group, and wrote the plan for the US-CERN link between now and the start of LHC operation. In 2002-3 he developed a new roadmap for HENP networking over the next ten years, progressing from the Gigabit/sec to the Terabit/sec range over the next decade, that is now widely recognized in the HENP and broader scientific community, as well as by National Lambda Rail and Cisco Systems. Together with S. McKee he co-chairs the Internet2 HENP Networking Working Group. • As a member of ICFA and Chair of the SCIC, we continue to lead the study of future bandwidth requirements for LHC, and our field as a whole, and scientific research in general, while also working on closing the Digital Divide that separates scientists in less-developed world regions7. • We have also been instrumental in developing the HENP-Internet2 relationship, and in making HENP the focal disciplines supported by Internet2’s advanced networking R&D. This relationship also led to CERN being admitted as the only full member of the I2 managing organization (UCAID) outside of the US. H. Newman serves on the Internet2 Applications Strategy Council and is an HENP representative of the Internet2 End-to-End Initiative since 2001. • The “Virtual Room Videoconferencing System” (VRVS) developed by our group since 1995 is now used as a vital tool for remote collaboration among the LHC as well as other experiments, and beyond HENP. Usage continues to grow exponentially by a factor of 2 to 2.5 each year, with the data flow being managed by 57 “reflectors” throughout the world that serve a community of more than 10,000 users in 61 countries. VRVS has been adopted as one of Internet2’s core technologies, and VRVS reflectors are installed permanently at several of Internet2’s points-of-presence (known as GigaPoPs). In 2003, we successfully released Version 3.0 (a major upgrade with 97% of the code rewritten from scratch), and made a rapid transition to the new version, with no major incident while continuing production use of the system. • In 2002-3 we began a collaboration with S. Low of the Caltech Network Laboratory. This led to the first trials of a new “ultrascale” TCP protocol stack (the protocol that carries 90% of the

5Our group’s work on the operation and development of networks and collaborative systems on behalf of the HENP community is summarized in the separate chapter on “LHCNet” in this Annual Report. 6Caltech led the team that won the Internet2 Land Speed Record in November 2002, and again in February 2003 7See http://cern.ch/icfa-scic for a series of reports presented to ICFA, February 2003. 62 5 CMS at LHC and L3 at LEP2

traffic on the Internet) “FAST TCP” that is capable of highly efficient, fair-shared use of wide area networks in the 1-10 Gigabit/sec range. This will have a major impact on the ability of the LHC experiments to do analysis effectively from the US and at other locations remote from the experiment. It also promises to have a profound effect on the future development of the Internet.

• Building on these and other developments, and the existing LHCNet and DataTAG facilities, we led a proposal8 to the NSF in May 2003, to build the ﬁrst intercontinental “hybrid” packet- switched and circuit switched network for scientiﬁc research. Our partners in this proposal include Internet2, CERN, FNAL, SLAC, Michigan, National Lambda Rail, Cisco Systems and Level(3) Communications, along with the leading optical circuit-switched research networks in the US, the UK and the Netherlands.

5.1.3 CMS/L3/CLEO Funding and Personnel

In this Annual Report chapter we document the group’s ongoing leadership and contributions to CMS and the LHC program (as well as the conclusion of 22 years of leadership roles in L3, and strong contributions to CLEO elsewhere in this report). In order to continue our technical roles, and apply them effectively as required to the CMS physics and data analysis program, we are requesting the (minimum) funding required to support the physicists in our combined group in FY2003, as specified below. Our ability to support our core group of physicists has been substantially eroded, in spite of increasing responsibilities in CMS over the last few years. This has been caused by funding reductions in then-year dollars, combined with the effects of inflation as well as necessary promotions of group members over the years, and recently by the weakening of the dollar against the Swiss Franc, by 20% in the last year.

CMS/L3/CLEO Group Personnel and Costs

Our HEP core group currently includes Professor H. Newman, Member of the Professional Staff R. Zhu, Faculty Associate and Member of the Professional Staff J. Bunn9, Senior Research Fellow S. Shevchenko, Staff Scientist R. Wilkinson, and L3 graduate student M. Gataullin10. Senior software engineers I. Legrand and V. Litvin are supported by funding coordinated through the US CMS Software and Computing project. Visiting engineers L. Zhang and K. Zhu, working with R. Zhu on the CMS monitoring project, are being supported by US CMS Construction Project funds11. As mentioned above, our efforts on CMS construction and operations have been significantly strengthened by having A. Born- heim join our group, and we look forward to a further strengthening when Professor Weinstein, and postdoctoral fellow S. Pappas redirect their efforts towards CMS during the course of this year. In order to cover our basic salary and travel costs, the minimum base budget required12 for our CMS activities in FY 2004 is $ 790 K. This level just covers salaries and travel for our existing personnel, and is therefore essential for our continued operations. Funding at this minimum level would allow us to (1) complete our assigned tasks for the US CMS Construction Project and for the Software and Computing Project, especially for the ECAL monitoring task; (2) continue our leading roles in CMS computing, software, and networking; and the design

8UltraLight: An Ultrascale Optical Network Laboratory for Next Generation Science. See http://ultralight.caltech.edu 9partially supported by NSF/KDI and Caltech/CACR funds. 10In 2002 we were joined by graduate Paige Randall working on MINOS. 11Other technical specialists in the group are supported by the LHCNet budget, or by grants from DOE/MICS (PPDG, VRVS R&D), DOE/HENP (PPDG) or NSF (GriPhyN, iVDGL, ALDAP, CAIGEE). 12Not including LHCNET funds, which are reserved for network operations, leasing of the link, routing and switching equipment, maintenance and management costs. 5.2 L3 At LEP 63 and development of the data analysis and Grid systems for CMS and the LHC; and (3) apply these developments as needed to CMS data analysis and preparations for physics.

RFQ: Costs for Return to the US; or Future Use Elsewhere

Caltech’s Radiofrequency Quadrupole (RFQ) calibration system was the basis of precision calibrations of the L3 BGO throughout LEP2. Most of the funding for building and commissioning this system was obtained from a DoE SBIR grant, the World Laboratory in Geneva, and the manufacturer (AccSys Technology) itself, in addition to the Caltech DHEP grant. In 2001 we dismantled the RFQ, removed our lab area and all the equipment in it that had been used to operate, maintain, and repair the RFQ and its RF, ion source and targeting systems over the last ten years. Since we had no funds to ship the RFQ and all of the lab equipment belonging to our group back to the US, we obtained permission from the CERN Directorate to store the $ 2M RFQ system in a temporary holding area at CERN until the end of 2001, as a stopgap measure. Shipping the RFQ back to the US could thus require additional funding, estimated at $ 20K, if we are required to ship the RFQ back to the US in FY200313.

5.2 L3 At LEP

LEP delivered its last beam on November 2nd, 2000, and the analysis of LEP data is entering its ﬁnal stage. The main activities in L3 (and other LEP experiments) include publishing ﬁnal papers and producing LEP combined results, most notably for the W mass and width as well as for the various SUSY searches. During the next six months L3 plans to publish about 30 papers, including at least four papers based on principal contributions by members of the Caltech group. In this section, we summarize the most important L3 and LEP results with an emphasis on those obtained with direct participation of the Caltech group.

5.2.1 Overview of Main L3 and LEP Results

During the whole LEP214 program, running from 1996 to 2000 at center-of-mass energies between − 161 GeV and 208 GeV, L3 collected about 700 pb 1. These data allowed us to search for new physics in a previously inaccessible range, and to test the Standard Model with unprecedented precision. The analysis of the LEP1-LEP2 data has resulted in 271 L3 publications. Of these, 39 (14%) are based on principal contributions by members of the Caltech group, and an additional 40 (15%) report on analyses done under our leadership or coordination. Selected L3 and LEP results obtained using the whole LEP data sample and based on Caltech’s contributions to the analysis, are summarized below.

• Precise Determination of Electroweak Parameters at LEP1 The LEP1 run from 1989 to 1995 was dedicated to precision studies of the Z boson properties. The most impressive result of these studies is the 2 × 10−5 accuracy for one of the most fundamental constants of nature, the Z mass: ± mZ =91.1875 0.0021 GeV. 13We are continuing to seek other uses for the RFQ system, such that we are not obligated to pay to ship it back to the US. 14The LEP1 run was completed successfully with 4 million hadronic and leptonic Z decays collected by L3 through Summer 1995. 64 5 CMS at LHC and L3 at LEP2

6 theory uncertainty ∆α(5) = W-Boson Mass [GeV] had 0.02761±0.00036 ± − 0.02747 0.00012 pp-colliders 80.454 ± 0.059 4 Without NuTeV ±

LEP2 80.412 0.042 2

Average 80.426 ± 0.034 ∆χ χ2/DoF: 0.3 / 1 2 NuTeV 80.136 ± 0.084

LEP1/SLD 80.373 ± 0.033

LEP1/SLD/m 80.380 ± 0.023 Excluded Preliminary t 0 20 100 400 80 80.2 80.4 80.6 [ ] [ ] mW GeV mH GeV

Figure 5.1: Direct and indirect measurements of Figure 5.2: Higgs mass prediction from the overall mW together with their average. electroweak working group ﬁt.

The leptonic partial widths Γee,Γµµ, and Γττ are in good agreement with one another, supporting the hypothesis of lepton universality of the neutral weak current [1]. • Measurements of the Properties of the W Boson Since 1996 LEP operated at energies signiﬁcantly above the WW threshold and studies of W-pair production and W boson properties were a major focus of the LEP2 physics program. The W 15 mass, mW, was measured through direct reconstruction of the invariant mass from the W decay products in qqq¯ q¯ and qq ν¯ events. Using the full LEP2 sample of more than 40,000 W-pair events the four LEP experiments have produced a preliminary combined result [2] for the W mass:

mW =80.412 ± 0.042 GeV,

which is consistent with the indirect determination of mW from a global ﬁt to the electroweak data from LEP1, SLC and the Tevatron, as shown in Figure 5.1. Measuring the non-Abelian triple and quartic gauge boson couplings is the second main goal of the W physics at LEP2. In particular, L3 has pioneered the studies of the quartic couplings with the W+W−γ and ννγγ¯ ﬁnal states, and the Caltech group (M. Gataullin) has had a leading role in these measurements16. The results of our analysis of these properties [3] are consistent with vanishing anomalous couplings. • Search for the Standard Model Higgs Boson The search for the Standard Model Higgs boson has been performed by examining four distinct event topologies: e+e− → H0Z → H0qq¯,H0νν¯, τ +τ −qq¯ and H0 + ¯−. Combining the data from the four LEP experiments, a lower bound on the Higgs mass is set at 114.4 GeV (95% C.L.), and an excess corresponding to about 1.5 σ is observed in the vicinity of 116 GeV [4].

15A. Shvorob of our group completed his thesis in 2000 on the study of 4 jet ﬁnal states from W pairs. 16These channels are exceptionally well measured in L3, due to the clean RFQ calibration and analysis of the BGO performed by our group, that led to elimination of the resolution tails. 5.2 L3 At LEP 65

Indirect experimental constraints [5] are also obtained from the precision measurements of the electroweak parameters, which depend logarithmically on the Higgs boson mass through radiative corrections. Currently these measurements constrain the Standard Model Higgs boson mass to +58 mH =91−37 GeV or to values smaller than 211 GeV, at the 95% C.L. (see Figure 5.2). • Searches for New Particles and Interactions

(1) No indication of a signal has been observed for the Neutral Higgs bosons belonging to a Minimal Supersymmetric Standard Model Higgs multiplet. The mass range up to 84.5 GeV has been excluded [6]. 0 (2) A LEP combined mass limit on the lightest neutralinoχ ˜ has been set: mχ0 > 45 GeV, 1 ˜1 which is independent of SUSY theory parameters [7]. (3) No new exotic heavy leptons have been found in the mass range up to ∼102 GeV [8]. (4) No evidence for new singlet heavy neutrinos has been found and we set [9] 95% C.L. limits −2 2 of (0.2 - 1.0) × 10 on the coupling constant |Ue| for the mass range 80 − 170 GeV. (5) Direct and indirect searches for large extra dimensions have been performed using single photon as well as fermion- and boson-pair production processes. The most stringent lower + − limits to date, Ms > 1.20 TeV and Ms > 1.09 TeV (95% C.L.), on the gravity eﬀective Planck scale have been set by the LEP collaborations [10].

5.2.2 Physics with Single and Multi-Photon Events

The L3 BGO calorimeter stood prominently as the most accurate photon detector at LEP. Its energy resolution was about 1% for photons and electrons with energies above 10 GeV, at least twice as good as the resolution of any other electromagnetic calorimeter at LEP. This made the L3 experiment the ideal place at LEP to search for new physics in photonic final states17. In the Standard Model single and multi-photon events with missing energy are produced via the reaction e+e− → ννγ¯ (γ) which proceeds through s–channel Z exchange and t–channel W exchange. Thus, the single photon topology provides a direct measurement of the invisible width of the Z, while the acoplanar-photons topology can probe + − the WWγγ quartic couplings in the e e → νeν¯eγγ process. Following detailed studies of the detector performance18, we have improved the selection efficiency and reduced systematic errors. To determine the number of light neutrino species, Nν , a fit to the recoil mass spectrum (shown in Figure 5.3) was performed. Combining the result of the fit with our previous LEP1 measurement [12] we obtain: Nν =2.94 ± 0.07, which is more precise than the present world average on the number of light neutrino families determined (by the Particle Data Group in 2002) with the single photon method [13]. Searches for Supersymmetry

In models with Gauge-Mediated SUSY Breaking (GMSB), the gravitino plays a fundamental role in the decay of SUSY particles. In particular, the neutralino is no longer stable and decays through 0 χ˜1 → G˜ γ, if it is the next-to lightest supersymmetric particle. Pair-production of the lightest neutralino leads to a two-photon plus missing energy signature in the detector. Using the expected and measured distributions of photon energies, their polar angles, recoil mass, and the polar angle of the missing

17This section covers some of the work of M. Gataullin of our group. 18Including studies of trigger system performance, photon conversion and cosmic contamination. Working in collaboration with the authors of the KK2f [11] program, we were also able to signiﬁcantly decrease the theoretical uncertainty. 66 5 CMS at LHC and L3 at LEP2

0 momentum vector the exclusion in the χ˜1 −˜eL,R mass plane was derived. Figure 5.4 illustrates that the GMSB interpretation of the eeγγ event observed by CDF [14] is ruled out by the L3 data. In the year 2002 we have also extended the above search to models with non-negligible neutralino lifetime, deriving the upper limits of 0.3 pb (95% C.L.) on the neutralino pair-production cross section for decay lengths up to 25 m.

 189 GeV ≤ √s ≤ 208 GeV

Data 125 300 _ L3 ννγ(γ) Preliminary  ∼0 √s = 189-208 GeV χ1 is pure Bino 100 200

0 ∼ 1

(GeV) χ 0 1 75 ∼ χ m < CDF m

Events / 4 GeV ∼ 100 L,R me 50 Excluded at 95% C.L.

0 50 100 150 200 0 50 100 150 200 ∼ Recoil Mass (GeV) meL,R (GeV)

Figure 5.3: Recoil mass√ spectrum of the single Figure 5.4: Excluded region for a pure bino neu- photon events selected at s = 189 − 208 GeV. tralino model.

Searches for Extra Dimensions

It has been recently proposed that the hierarchy problem of the electroweak field theory can be avoided by simply removing the hierarchy [15]. In such models gravity becomes strong near the electroweak scale and gravitational fields can propagate in the n new large extra dimensions, whereas the Standard Model fields are forced to lie on a 3-dimensional wall in the higher-dimensional space. Direct emission of gravitons leads to a missing energy signature, e+e− → Gγ, in which a photon is produced with no observable particle balancing its transverse momentum. Using all data LEP2 data we have derived upper limits on the new gravitational scale as a function of the number of extra dimensions [16]. Gravitational scales as high as 1.5 TeV are excluded providing the most stringent limits to date.

5.2.3 Searches for Scalar Leptons

Supersymmetry (SUSY) [17] represents the best-motivated known extension of the Standard Model. It offers an elegant solution to the naturalness problem of the Higgs sector [18], is consistent with present experimental data, and predicts new particles to be discovered in this generation of collider experiments. In the Minimum Supersymmetric Standard Model (MSSM) for each of the Standard Model leptons there exists a corresponding scalar SUSY partner (scalar lepton). Because of their relatively light mass and their distinctive final state signatures, scalar leptons are expected to be one of the first SUSY particles to be discovered in collider experiments. 5.2 L3 At LEP 67

˜→ 0 All scalar leptons decay predominantly into their Standard Model partners plus a neutralino, l lχ˜1. At LEP energies, the scalar lepton signal appears as a pair of acoplanar leptons in the detector, where the kinematic distributions depend both on the mass of scalar lepton itself and the mass of the neutralino. Both factors are taken into account by a parameterized selection19. Combining all LEP2 data, a large area of the M˜l − Mχ0 plane has been excluded at 95% C.L. Figure 5.5 shows the excluded region for ˜1 the scalar tau case. L3 preliminary 100 µ = -200 GeV tan β = 2.0 τ~± → τ± ~ R G L3 preliminary 100 τ~± → τ±χ~0 90 R 1 (Br=100%) Observed Average 0.05 pb ∆M = 0 80 (GeV)

R 0.15 pb

(GeV) 70 ~ τ 0 501 ~ χ M M > 60 0.15 pb 0.10 pb Excluded at 95% C.L. 0.10 pb 50 0 60~ 80 100 -3 -2 -1 2 Mτ (GeV) 10 10 10 1 10 10 R Decay Length (L: cm)

Figure 5.5: Excluded region at 95% C.L. in the Figure 5.6: Upper limit on the scalar tau produc- Mχ0 − Mτ˜ plane. tion cross section. ˜1

Scalar Tau in GMSB models

In GMSB models with a Next to Lightest SUSY Particle (NLSP) lighter than ∼ 100 GeV, i.e. accessible at LEP2, the NLSP is either a neutralino or scalar tau throughout the parameter space [19]. In order to search for the scalar tau in L3, L. Xia added new simulation code to the L3 standard simulation program, to handle decays of long lived particles and to calculate the dE/dx signal in the L3 Time Expansion Chamber (TEC). Five different selections have been optimized for different decay lengths of scalar tau, and the upper limit on the production cross section is shown in Figure 5.6. This limit translates into a mass exclusion limit, Mτ˜ > 80 GeV (with an expected limit of 79 GeV), which is independent of theτ ˜ lifetime. This is the first L3 result for this scenario. Combining scalar tau and neutralino searches20,wesetalowerontheNLSPmassat66GeV[20].

5.2.4 The RFQ Project

The L3 BGO calorimeter was designed to provide precise energy measurements of electrons and photons, from less than 100 MeV up to 100 GeV. The experiment’s ability to detect new physics processes, such

19This section covers work by L. Xia who completed his thesis on the search for scalar leptons in 2002. 20See Section 5.2.2 68 5 CMS at LHC and L3 at LEP2

+ − 0 0 21 as e e → χ˜1χ˜1 → G˜ G˜ γγ , depended on the resolution of the electromagnetic calorimeter, and hence on the quality of its calibration.

1000 Barrel σ = 1.05%

500

4 0 3 2 Endcaps 1 10000 σ = 0.85% Events / 0.0025

5000

0 0.9 0.95 1 1.05 1.1 BGO energy / Beam energy

Figure 5.7: Side view of the RFQ accelerator Figure 5.8: The 2000 Bhabha energy spectra in system showing (1) the H− ion source (2) the the barrel and the endcaps obtained with the RFQ 1.85 MeV RFQ accelerator (3) the focusing and calibration. steering magnets and (4) the neutralizer.

The Caltech group pioneered a precise and rapid calibration technique [21] using a pulsed H− beam from a Radiofrequency Quadrupole (RFQ) accelerator, to bombard a lithium target permanently 7 8 installed inside the BGO calorimeter. The radiative capture reaction 3Li(p,γ) 4Be was used to produce an intense ﬂux of 17.6 MeV photons which allowed us to calibrate the thousands of the BGO crystals, on a crystal-by-crystal basis in a short time. The components of the RFQ system are described in more detail in [22]. A general view of the RFQ system installed in L3 is shown in Figure 5.7. From 1997 until the end of the LEP physics program the RFQ calibration was used in physics reconstruction at L3 giving the energy resolution between 0.85% and 1.05% [23]. That was the best resolution obtained since the BGO barrel was calibrated in the CERN test beam in 1987-8, prior to LEP startup. For comparison, other calibrations used through the end of 1996 were giving a resolution of about 2% both in the BGO barrel and endcaps. As an example, the Bhabha spectrum obtained using data collected by L3 in the year 2000 is shown in Figure 5.8. The RFQ calibration system proved to be reliable and robust: during 1997-2000 we have performed six RFQ calibration runs collecting about 100 million triggers and the last RFQ run performed in September 2000 showed no evidence for aging of the RFQ system hardware. Therefore at the end of LEP program it was decided to move the RFQ system from the L3 cavern to a storage room in the Swiss part of CERN. In January 2001 we successfully transferred the RFQ system together with all readout electronics to the designated storage area. Only minimal changes were made to the RFQ setup, so that the system is ready to resume working.

21Where L3 has obtained the most stringent limits to date on the production of a light gravitino. 5.3 CMS At LHC 69

5.3 CMS At LHC

5.3.1 Caltech Role

The Caltech group has been playing a leading role in TeV-scale physics for more than a decade. The group has been involved in pursuing the origin of electroweak symmetry breaking by developing the detector concept and analysis strategies. In both L and GEM at the SSC, the Caltech group was responsible for the Higgs physics performance studies, and proposed a precision crystal calorimeter as part of the detector design. We joined CMS in 1994, and have been taking major responsibilities in the design and construction of the lead tungstate crystal calorimeter, and software and computing development and management, as well as the management of the US CMS Collaboration. H. Newman was elected Collaboration Board Chair of US CMS in 1998, in 2000, and again in 2002, and was Chair of the international CMS Software and Computing Board from 1996-2001. He was a member of the Steering Committee of the Hoﬀmann Review of LHC Computing and has had a leading role in deﬁning and developing the worldwide Computing Model and for the LHC since 1998, as summarized later in this chapter.

Role in the CMS Electromagnetic Calorimeter

The Caltech group made key contributions to the development of the PbWO4 ECAL crystals, including investigations on their performance under irradiation and the factors that determine their radiation hardness. We completed this investigation in 1999. Our leading role in this R&D, which is now being continued to develop new heavy scintillators for future HENP experiments, is a direct consequence of the expertise, methodology and instrumentation developed for the BAF2 calorimeter in GEM at the SSC and the CsI(Tl) calorimeter in BaBar at SLAC. Although US CMS does not have financial responsibility in PbWO4 crystal production, R.Y. Zhu has served as a member of the CMS ECAL Technical Board, sharing responsibility on PbWO4 crystals. Together with the Saclay group, we are responsible for the construction of a precision monitoring system to track any residual effects of damage and recovery in situ at LHC, which will be critical for the ECAL to reach its design goals. In the US CMS project organization, R.Y. Zhu is the Level 3 manager for both monitoring construction (WBS 4.3) and PbWO4 R&D (WBS4.4). In addition, R.Y. Zhu is now serving a second two year term as Chair of the US CMS ECAL Institutional Board, and is a member of the Advisory Board of the US CMS Collaboration. Our group also has made substantial contributions to the CMS ECAL detector design and physics performance simulation efforts. This includes studies to optimize the crystal ECAL configuration, to achieve the high energy resolution for electrons and photons, combined with the uniform acceptance, angular resolution and background suppression (γ − π0 separation) essential for low-mass Higgs detection. We have also used the unique computing facilities at Caltech’s Center for Advanced Scientific Com- puting (CACR) and at other NPACI22 sites across the US, to carry out the first direct full-simulation studies of the multi-jet backgrounds to the Higgs decay into two photons, and to provide multi-Terabyte samples of fully simulated events for use by the collaboration. We are now beginning to expand this collaboration and the scope of our physics studies in the future, since Caltech and San Diego are two of the four TeraGrid sites23.

22National Partnership for Advanced Computing Infrastructure. We applied for, and received, large grants of CPU since 1999-2001 to accomplish these studies. 23See http://www.teragrid.org 70 5 CMS at LHC and L3 at LEP2

Role in CMS Computing, Software and Networking

Caltech has had a leading role in the development of CMS’ oﬄine computing, software and networking systems for the last 7 years. This is founded on our experience as leaders of the design, implementation and operation of these systems for L3 since 1982, and earlier experience at CERN (1974-8) and DESY (1978-82). Our group is the originator, and includes some of the principal operators and developers of international networking (and more recently video conferencing and remote collaborative systems) for high energy physics. This has included the international links used by HEP between DESY and the US, LEP3NET, and the CERN-US consortium managing the transatlantic link, and recent work on international network planning and “Grid” projects (GIOD, MONARC, PPDG, GriPhyN and iVDGL) for the next generation of wide area networks and distributed data analysis systems for experiments (See the Chapter on LHCNET elsewhere in this Annual Report). We have recently taken a leading role in developing the “Grid-enabled Analysis Environment” for the LHC, which is a major step forward in understanding how analysis by worldwide physics collaborations will be done in the LHC era. This work is crucial for the success of the US involvement in the LHC program, since US physicists must be able to participate and take leading roles in the data analysis when they are at their home institutions, as well as when they are at CERN. In 2002-3, we therefore led two NSF/ITR proposals: CAIGEE24 and GECSR25, while participating in a Large ITR proposal with FNAL and others. We originated the US CMS Software and Computing project in 1998 and led it throughout its early stages, working with Fermilab Computing Division and the Fermilab Directorate to get the project baselined in November 2000. We developed the concept of Tiered Regional Centers in 1999 that has since become the basis of the LHC Computing Model for all four LHC experiments, as well as BaBar and some other major experiments. Our group also has developed the VRVS system, now in its seventh year of production use, which is a vital tool for daily collaboration by the LHC and many other collaborations in HENP and other ﬁelds26.

5.3.2 CMS Physics Goals and Detector

The CMS detector, with its emphasis on clean identiﬁcation and precision measurements of electrons, muons and photons, together with good jet and missing energy measurements, has been designed to exploit the full range of TeV-scale physics at the LHC. The main motivation for building the LHC and CMS is to investigate the mechanism responsible for electroweak symmetry breaking. Other important physics motivations of CMS are to search for super-symmetric particles, to study the properties of the top and the bottom quark, to search for new forces in the form of massive bosons similar to the W and Z and for compositeness of quarks and leptons.

The design goal of the CMS detector is to measure electrons, photons and muons with an energy resolution of better than 1% over a large momentum range. The principal detector features include a high resolution and redundant muon system, the best achievable crystal electromagnetic calorimeter (ECAL) and a precision all-silicon tracker. A three dimensional cutaway view of the CMS detector is shown in Figure 5.9. The overall dimensions of the detector are: a length of 21.6 m, a diameter of 15 m and a total weight of 12,500 tons. The hadron calorimeter (HCAL) and ECAL are located in a large,

24CMS Analysis: an Integrated Grid-enabled Environment; with UCSD, UC Riverside and UC Davis 25A Global Grid-Enabled Collaboratory for Scientiﬁc Research; with Michigan, Maryland, FNAL, UT Arlington and others 26This is discussed further in the LHCNet Chapter of this report. 5.3 CMS At LHC 71

CMS A Compact Solenoidal Detector for LHC FORWARD MUON CHAMBERS TRACKER CRYSTAL ECAL HCAL CALORIMETER

Y Z

AIR PADS

Total weight : 12,500 t. Overall diameter : 15.00 m SUPERCONDUCTING COIL Overall length : 21.60 m RETURN YOKE Magnetic field : 4 Tesla CMS-PARA-001-24/11/97 JLB.PP / pg.gm.hr

Figure 5.9: A three-dimensional cutaway view of the CMS detector.

13 m long, 6 m diameter, high-field (4T) superconducting solenoid. US CMS has primary responsibility for the HCAL and the forward muon system. The CMS27 construction is progressing rapidly. For example the hadron calorimeter brass modules have been completed, the muon cathode strip chambers (CSCs) are nearing completion, production of tracker modules is gearing up, ECAL crystal production is proceeding at a high rate and super module construction is under way at CERN. The Level 1 Trigger TDR was submitted in 2000 and the DAQ TDR was finished in late 2002. Higher level trigger studies are being worked on intensively and the major focus is now the completion of the physics TDR, with a series of ”data challenges” leading up to it. The schedule for the LHC machine, adjusted in 2001 to deal with the CERN budget shortfalls, calls for first beam in April 2007. To match with this schedule, the CMS goal is to have a full working detector28 ready by April 1, 2007. The first physics run is expected to reach a luminosity of 2 × 1033 cm−2s−1, corresponding to ∼ 10 fb−1 expected by late 2007 or early 2008, and first physics results in 2008.

27Further information on the CMS design, organization, progress and milestones may be found at http://cmsdoc.cern.ch. 28The complete detector except for the pixels and the fourth forward muon chamber layer ME4. 72 5 CMS at LHC and L3 at LEP2

5.3.3 CMS ECAL: Status, Schedule

The CMS ECAL consists of 77,200 lead tungstate (PbWO4) crystals of 23 cm (25.8 X0) long and 2.2 × 2.2 cm2 front face (1.1 kg) with total√ crystal volume of 10.8 m3 and a weight of√ 90.3 t. The designed energy resolution (∆E/E )is2.5%/ E ⊕0.55%⊕0.2/E for the barrel and 5.7%/ E ⊕0.55%⊕0.25/E for the endcaps, where E is in GeV. Test beams at CERN have shown that this energy resolution can be achieved by using production PbWO4 crystals with Si avalanche photodiodes (APD). Figure 5.10 shows the distributions of the stochastic (left) and constant (middle) terms of the energy resolution, and the 0.45% energy resolution reconstructed in 3 × 3 PbWO4 crystals for 280 GeV electrons (right). Based on this result, CMS decided to use two APDs instead of one APD [24], so that the stochastic term will be reduced to 2.5%.

E = 280 GeV σ/E = 0.45%

100

Events/100 MeV 50

265 275 285 295 E (GeV)

Figure 5.10: The stochastic (Left) and constant (Middle) terms of energy resolution and 280 GeV electron signals (Right) obtained in CERN beam test.

The ECAL is divided into 36 supermodules (SM) in the barrel and 4 Dee’s in the endcaps. The overall construction of the CMS ECAL is in progress. About 18,000 out of 62,000 barrel crystals have been delivered, and used to construct modules (400 or 500 crystals) at CERN and Rome. At the end of 2002, 21 out of 144 modules have been produced. Two bare SMs (1,700 crystals) have been assembled. The APD production rate reached 6,000 APD/month, and About 85,000 out of 130,000 APDs have been delivered. For the endcaps, 4,100 VPTs have been delivered and tested to 1.8T. The baseline ECAL electronics underwent a major revision in 2002 primarily to contain costs. The modified floating-point pre-amplifier (FPPA) was delivered from the foundry at the end of 2002. Preliminary measurements show that original faults (high noise and wrong pulse shape) have been successfully corrected, although the noise is still 30% higher than the specification. Furthermore the yield of FPPAs is 35%, significantly lower than the 50% expected. In January, 2003, successful design reviews were held for 0.25 µm ASICs: FENIX (trigger primitive), MGPA (a backup pre-amplifier based on 0.25 micron technology), ADC and PACE (preshower front end), which was followed by submission. An addendum to the ECAL TDR, outlining the changes in electronics, was submitted in October, 2002. It is expected that the decision on the final choice of the front-end electronics will be made in mid-2003 after system tests. Several hundred channels, equipped with FPPA and MGPA front electronics in the first production supermodule, will be tested in a beam in 2003. The ECAL test-beam has been extended to October, 2003, to accommodate the MGPA delivery schedule. A new ECAL schedule has been produced, taking into account the new electronics project. The ECAL is expected to be completed and commissioned by 5.3 CMS At LHC 73

March 2007. This schedule foresees calibrations of at least 9 SMs in beam in 2004 and 1 Dee in 2006. Caltech has made major contributions to the design and construction of the ECAL. CMS’s choice of a crystal calorimeter is justiﬁed by the search for the Higgs boson in the mass range from 80 to 180 GeV, where H→ γγ is a unique decay channel and its discovery potential is directly related to the ECAL resolution. This has been extensively studied by R.Y. Zhu and H. Yamamoto for the SSC [25], and by C. Seez and J. Virdee for the LHC [26].

There are two crucial issues for maintaining PbWO4 resolution in situ at LHC: the stability of PbWO4 crystals in the LHC radiation environment, and precision calibration in situ. Caltech has made leading intellectual contributions to both aspects. The results of the PbWO4 investigations at Caltech have been reported by R.Y. Zhu at various LHCC reviews, DoE Lehman reviews and major calorimetry conferences, and have been documented in the CMS ECAL TDR [24] as well as in Nucl. Instr. and Meth. and IEEE Trans. Nucl. Sci. NS [27]. Caltech also shares responsibility with the Saclay group for construction of a laser-based monitoring system. As discussed in Section 5.3.5, a key concept of “continuous monitoring in situ” was proposed by R.Y. Zhu et al. [28]. The ﬁrst laser system of the Monitoring Light Source and High Level Distribution System (LSDS) [24] was constructed at Caltech [29], and was installed and commissioned at CERN in August, 200129 [30]. It has been successfully used in the ECAL beam test since then. The 2nd and 3rd laser systems will be installed and commissioned at CERN in 2003.

Table 5.1: DoE Funds (k$) Received through US CMS R&D FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 PbWO4 Crystal R&D 0 0 64.5 16 64 0 0 0 0 Monitoring 30 45 0 134 444 284 537 224 174 Travel for ECAL 3 10 15 12 11.7 10 10 15 15 Travel for Computing 0 5 5 10 14.5 15 10 7 6 Generic Crystal R&D 0000007500 Total 33 60 84.5 172 534.4 309 632 246 195

Table 5.3.3 summarizes the DoE supplementary funds received in previous ﬁscal years for CMS ECAL related activities and generic crystal R&D at Caltech. Starting from FY98, the US CMS project funds have been provided via a purchase order from FNAL.

5.3.4 CMS Crystal R&D and Status

−3 PbWO4 is a heavy crystal scintillator with high density (8.3 g cm ), short radiation length (0.89 cm) and small Moli`ere radius (2.2 cm). Yttrium doped PbWO4 crystals have an emission spectrum with a broad peak at 420 nm, and a FWHM of 120 nm. After extensive R&D, PbWO4 crystals are now in mass production. The 61,200 PbWO4 crystals for the CMS ECAL barrel are contracted to the Bogoroditsk Techno-Chemical Plant (BTCP) in Tula, Russia. As of this writing, 18,000 PbWO4 crystals were received. BTCP has also demonstrated large boules of 61 mm and 85 mm diameter, may be cut into two or four crystals respectively, thus reducing the furnace time. Our study at Caltech shows that the radiation hardness of PbWO4 crystals produced at BTCP is adequate for the CMS ECAL barrel, one type of their crystals is good enough for the endcaps. It is expected that the contract for the 16,000 endcap crystals will be signed in 2003.

29http://www.hep.caltech.edu/∼zhu/mon 010904.pdf 74 5 CMS at LHC and L3 at LEP2

100 BTCP-2455 200oC annealing BTCP-2455 1.4 80 80 BTCP-2455 em: 420 nm ex: 310 nm )

60 -1 1.2

40 60 1 20

From top to bottom 0.8 1000 200oC annealing 40 15 rad/h (82 h) 80 400 rad/h 0.6 em: 420 nm ex: 310 nm 100 rad/h (80 h) 400 rad/h (61 h)

60 Transmittance (%) 9000 rad/h (22.5 h) 0.4 Intensity (arbitrary unit) 20 40

Emission weighted RIAC.(m 0.2 20 dose rate (rad/h): 9 k 35 k 15 100 400 0 0 0 250 300 350 400 450 500 550 300 400 500 600 700 800 0 50 100 150 200 250 Wavelength (nm) Wavelength (nm) Time (hours)

Figure 5.11: The excitation and emission spectra before and after irradiation at 400 rad/h (Left) and the transmittance spectra at equilibrium under irradiations of several dose rates (Middle) are shown as a function of wavelength, while the corresponding history of the emission-weighted radiation induced absorption coeﬃcient (Right) is shown as a function of time, for sample BTCP-2455.

After several years of R&D, radiation damage in PbWO4 crystals is fairly well understood. Fig- ure 5.11 (Left) shows no variation of the excitation and emission spectra indicating no damage in the scintillation mechanism. Radiation damage, however, is observed in the formation of radiation induced color centers, as illustrated in the transmittance spectra shown in Figure 5.11 (Middle). Because of the equilibrium between the color center’s formation process and its annihilation process, the level of radiation damage in PbWO4 crystals is dose rate dependent as shown in Figure 5.11 (Right). It is also understood that radiation damage in PbWO4 crystals originates from structure defects, such as oxygen vacancies. The details of our study can be found in published references [27, 31, 32, 33].

3.5 3.5 1 Type (I) Emission weighted Rad. Ind. absorption After 9000 rad/h in equilibrium versus dose rate of Endcap 3 24 BTCP CMS PWO crystals 3 ) ) Type ( I ) -1 0.9 -1 Type ( II ) Type (III) 2.5 2.5 Barrel

0.8 Type (II) 2 2

Type (II) 1.5 0.7 1.5

1 1 0.6 Normalized T @ 440 nm versus dose rate of Normalized T% @ 440 nm Emission weighted RIAC (m Absorption coefficient (m 0.5 25 BTCP CMS PWO crystals Type (I) 0.5

0.5 2 3 4 0 2 3 4 10 10 10 10 10 10 19 20 21 22 23 24 25 26 27 28 29 Dose rate ( rad/h ) Dose rate ( rad/h ) Longitudinal transmittance @ 360 nm (%)

Figure 5.12: The normalized transmittance at 440 nm (Left) and the EWRIAC (Middle) measured in equilibrium are shown as a function of dose rate, while the EWRIAC under equilibrium at 9,000 rad/h (Right) is shown as a function of initial longitudinal transmittance at 360 nm, for 25 BTCP PbWO4 samples.

Recently, R.H. Mao (a visiting graduate student) and R.Y. Zhu studied 25 PbWO4 crystals produced 5.3 CMS At LHC 75 at BTCP30. Figure 5.12 (Left and Middle) shows that these crystals can be divided into three categories. “Type I” crystals are very radiation hard, as evidenced by their low loss of transmittance at 440 nm (5 to 9%) and their small emission-weighted radiation induced absorption coeﬃcient (EWRIAC; 0.2 to 0.4 m−1) when the crystals are in equilibrium under irradiation at a dose rate of 9,000 rad/h, which is more than 20 times the dose rate expected in the endcaps at LHC. Type II crystals showed 20 to 50% and 0.8 to 3 m−1 respectively under 9,000 rad/h, which were reduced to 2 to 8% and 0.1 to 0.35 m−1 respectively under 15 rad/h. Although not as radiation hard as Type I, the radiation hardness of the type II crystals is adequate for the barrel, where 15 rad/h is expected at LHC. One Type III crystal has a preexisting color center at 420 nm, which may cause confusion for monitoring with 440 nm laser light [34], and so this type should be rejected. It is also interesting to note that there exist no correlations between a PbWO4 crystal’s radiation hardness and its initial longitudinal transmittance. Figure 5.12 (Right) shows no correlation between the EWRIAC and initial transmittance at 360 nm. The observation at 440 nm is the same.

We plan to continue the work on PbWO4 crystals to understand the diﬀerence between Type I and Type II crystals, and to develop PbWO4 crystals of high light yield for future high energy and nuclear physics experiments31 [35].

5.3.5 CMS ECAL Monitoring Construction Project

Once PbWO4 crystals with adequate radiation hardness are produced, calibration is the key to maintaining crystal precision in situ at LHC. Photons from η → γγ decays32 as well as electrons from Z and W decays will be used to determine the ECAL calibration during LHC operation [24]. Other physics processes, such as electrons pairs from resonances, such as J/ψ and Υ decays, may also be used [36]. The photon and electron pair channels have the advantage that the ECAL calibration can be performed by using ECAL information alone with the mass constraint. The short term variation (damage and recovery) of PbWO4 light output, however, have to be taken care of by light monitoring. To continuously measure these variations, monitoring light pulses are to be sent to the crystals in the 3 µsec gap, which is scheduled every 88.924 µsec33 in the LHC beam cycle [37]. Caltech is responsible for constructing the Light Source and the High Level Distribution Subsystem (LSDS) for ECAL monitoring. A brief report on the monitoring construction project is given below. The monitoring wavelength was determined by using a test bench, shown in Figure 5.13 (Left), which measured the PbWO4 crystals’ light output and transmittance as a function of wavelength with precisions of 1% and 0.5% respectively [38]. Figure 5.13 (Middle) shows typical correlations between the relative variations of the transmittance (∆T/T) and the light output (∆LY/LY) for the monitoring light at four different wavelengths: (a) 410, (b) 440, (c) 490 and (d) 520 nm, for a yttrium doped sample ∆T × ∆LY 2 SIC-S762. The correlation was fit to a linear function: T = slope LY The linearity (χ /DoF of the fit) is generally good when light output loss is less than 10%. Figure 5.13 (Right) shows the monitoring sensitivity (the slope) and the linearity as a function of the monitoring wavelength for four samples. Also shown in the figure is the PMT quantum efficiency-weighted radioluminescence. The higher monitoring sensitivity at shorter wavelength is understood as being caused by the poorer initial transmittance as compared to that at the longer wavelength. The best linearity around the peak of the radioluminescence is understood by two radiation-induced color centers peaked at the two sides of the luminescence peak with different damage and recovery speed, as discussed in detail in reference [38]. Based upon this result, we chose 440 nm as the monitoring wavelength [39] with 495 nm as a cross-check wavelength. In 2002, CMS ECAL management decided to add an IR/red laser in the monitoring light source

30http://www.hep.caltech.edu/∼zhu/ryz pwo 030325.pdf. 31http://www.hep.caltech.edu/∼zhu/ryz lc 0208.pdf and ryz lc 0201.pdf. 32A new method developed by S. Shevchenko of our group, see Section 5.3.6 33The gap is designed to be used to reset the kicker magnets. We will use a portion of the available gaps. 76 5 CMS at LHC and L3 at LEP2

Quartz Fiber Normalized Light Output (%) 0.8 SIC-S347 (a) BTCP-2162 (b) 5 Shutter -25 -20 -15 -10 -5 0 5 -20 -15 -10 -5 0 5 Chopper 2 4 0.6 Intergrating Mono- 150 Watt 0 chromator Xe Lamp SIC-S762 3 Sphere -2 (a) (b) 0.4 -4 2

Step Motor -6 0.2 Wrapped 137 1 Cs Interface PbWO -8 4 λ=410 nm λ=440 nm -10 0 0 χ2 χ2 /DOF=3.34 /DOF=0.92 -12 (c) (d) slope=0.470 ± 0.009 slope=0.505 ± 0.009 0.8 SIC-S762 BTCP-5658 5

MERLIN -14 Linearity 2 Sensitivity PMT Radiometry System 4 0 0.6 (c) (d) 3 1 -2 0.4 -4 2 2 -6 0.2 -8 1

QVT Multichannel PC λ=490 nm λ=520 nm -10 χ2 χ2 0 0 Analyzer System /DOF=1.14 /DOF=2.99 -12 400 450 500 550 400 450 500 550 slope=0.477 ± 0.009 slope=0.468 ± 0.009 -14

Normalized Longitudinal Transmittance (%) Wavelength (nm)

Figure 5.13: Left: A schematic showing monitoring test bench. Middle: Correlations between the relative variations of transmittance and light output are shown for Y doped sample SIC-S762. Right: Monitoring sensitivity (solid dots), linearity (open dots) and emission spectrum (solid lines) are shown for four samples.

project34, which will be used to monitor independently the gain variations of the readout electronics chain. Figure 5.14 is a schematic showing the design of the LSDS, which consists of three sets of lasers with corresponding diagnostics, a 2 × 1 switch, a 1 × 80 switch, a monitor and a controller. Each set of lasers consists of an Nd:YLF pump laser and a tunable Ti:Sapphire laser with dual wavelength. Using the 2×1 switch, up to four wavelengths (440, 495, 709 and 796 nm) are available for monitoring. A spare laser system guarantees 100% availability of the monitoring light source at 440 and 495 nm, even during laser maintenance. Each laser set has a main and a diagnostic output. The latter is used to monitor laser pulse wavelength and shape. The parameters of the entire system are set by a PC through a laser DAQ, which also collects overall information on laser performance, e.g. pulse energy, width, timing and wavelength. A 135 hour long term stability test showed that the instability of the pulse energy and FWHM are at the 3.7 and 2.0% level respectively; much better than the CMS specification of 10% [39]. The first laser system and its corresponding diagnostics were installed and commissioned at CERN in August, 200135, and was successfully used in ECAL beam test since then. Figure 5.15 shows consistent linearity between the electron data and the monitoring data obtained by using the 440 nm light from the first laser system at CERN. The monitoring construction project is led and managed by R.Y. Zhu. Three research scientists Dr. Liyuan Zhang, Kejun Zhu and Qing Wei, are involved in the construction, as well as Dr. Duncan Liu from the Optical Instrumentation Division of the Jet Propulsion Laboratory and Caltech HEP technicians J. Hanson and L. Mossbarger. A technician, David Bailleux, has been working at CERN for daily laser maintenance and operation since August 2001. Starting in the Fall of 2002, Dr. Adolf Bornheim joined the ECAL monitoring project at CERN and has taken the responsibility for laser safety. We are currently constructing the second and third lasers, switch and monitor at Caltech, which are scheduled to be delivered to CERN in 2003. As of this writing, the entire LSDS construction project at Caltech is on schedule and on cost.

34http://www.hep.caltech.edu/∼zhu/ryz mon 020416.pdf. 35http://www.hep.caltech.edu/∼zhu/ryz mon 010904.pdf. 5.3 CMS At LHC 77

NET Ext. Trigger Quantronix GPIB Digital Mono− Oscilloscope PC − Ti:S GPIB RS232 Controller Delay chromator Digitizing Apr. 21, 2003 Caltech PIN

Pulse Timing Diagnostic FS CAMAC PIN Quantronix Ti: Sapphire Main ( 440 or 495 nm ) Multi−Channel ADC

PIN o o o o o o Quantronix Nd:YLF (527 nm) Diagnostic

o Chilled Water, 7~18 C, PIN To Level Two Fanout 1~7 kg/cm 2 , 16−24 l/min

1−4: Valves o o NESLAB Quantronix 5: Pressure Regulator Chiller Nd:YLF Power / Cooler 6,7: Pressure Gauge 8: Filter 9: Flow meter 2 x 1 1 x 80 1576 8 2 220VAC, 50Hz, ~10A Trans− Optical Optical Monitoring 3 Phase, 380 VAC former Switch Switch Box 50 Hz, 30A/Phase 349 Interlock

Ext. Trigger Ext. Trigger

GPIB GPIB Quantronix Digital Mono− Oscilloscope GPIB Quantronix Digital Mono− Oscilloscope Ti:S − Ti:S RS232 Controller Delay chromator Digitizing RS232 Controller Delay chromator Digitizing

PC PIN PIN

Pulse Timing Pulse Timing Diagnostic FS Diagnostic FS NET PIN PIN Quantronix Ti: Sapphire Quantronix Ti: Sapphire Main Main ( 440 or 495 nm ) ( 700 or 800 nm ) Off−line laser system PIN PIN Quantronix Nd:YLF (527 nm) Diagnostic Quantronix Nd:YLF (527 nm) Diagnostic

Chilled Water, 7~18 o C, Chilled Water, 7~18 o C, 1~7 kg/cm 2 , 16−24 l/min 1~7 kg/cm 2 , 16−24 l/min 1−4: Valves 1−4: Valves NESLAB Quantronix 5: Pressure Regulator NESLAB Quantronix 5: Pressure Regulator Chiller Nd:YLF Power / Cooler 6,7: Pressure Gauge Chiller Nd:YLF Power / Cooler 6,7: Pressure Gauge 8: Filter 8: Filter 9: Flow meter 9: Flow meter

220VAC, 50Hz, ~10A 1576 8 2 220VAC, 50Hz, ~10A 1576 8 2 Trans− Trans− 3 Phase, 380 VAC former 3 Phase, 380 VAC former 50 Hz, 30A/Phase 349 50Hz, 30A/Phase 349 Interlock Interlock

Figure 5.14: A schematic showing the design of the laser based monitoring light source and high level distribution system.

5.3.6 CMS ECAL Physics Reconstruction and Selection

Higgs discovery potential study

Since 2001, the Caltech group (T. Lee, V. Litvin, H. Newman and S. Shevchenko) has been carrying out in-depth studies of CMS’ discovery potential for Higgs bosons, in the intermediate mass range favored by the precision electroweak data, through the H→ γγ decay. The two photon decay process is one of the most promising channels to search for the Higgs boson in the mass region up to 150 GeV [40, 41, 42, 43, 44]. We have been able to do this study using full detector simulations both for the Higgs signal and all types of the relevant background, for the first time, using the relatively large computing resources at our disposal. A special goal of this study is to simulate for the first time a large enough sample of QCD jet background to directly estimate the di-photon misidentification, and compare it with contributions from the other types of background. The QCD jet background cross section is huge (∼ 109 pb). Therefore, previous studies of QCD jet background [40, 44] have either been done at the generator level or have obtained estimates for the rate rjet at which a jet would be misidentified as a photon. Due to limited computational power, however, these studies estimated the QCD jet background by simply 2 multiplying the rates together (factor rjet). The problem with this method is that the correlations within an event are not taken into account. In addition, the simulation was done with simplified geometry, so non-Gaussian tails in the resolution have not been adequately simulated. About two million events were simulated and reconstructed by V. Litvin using version 5.2.0 of the 78 5 CMS at LHC and L3 at LEP2 ∆ 1 = 1.6 S Electron Data ( LY) 1st Radiation

1 = 1.6 S

2nd Radiation

1 = 1.6 S

Recovery

Blue Laser ( ∆ T)

Figure 5.15: History of electron (120 GeV) and laser (440 nm) data obtained during 2002 beam test are shown as function of time for PbWO4 radiation damage and recovery. Three expansions at right show the same linearity between the electron and monitoring signals.

object-oriented reconstruction program ORCA36 [45]. Both the inclusive production mode of the Higgs boson [40, 41] and more recently the pure Vector Boson Fusion production mode [46, 47] have been studied, for the low luminosity (2 × 1033cm−2sec−1 scenario expected in the ﬁrst year of LHC running. The search for H→ γγ signal at LHC has to treat three types of background:

• Prompt di-photon production from quark annihilation and gluon fusion diagrams, which provides an intrinsic or ‘irreducible’ background. • Prompt di-photon production from signiﬁcant higher-order diagrams - primarily bremsstrahlung from the outgoing quark line. • The background from QCD jets, where an electromagnetic energy deposit results from the decay of neutral hadrons (especially isolated π0s) in a jet and from one jet + one prompt photon.

For the Higgs search in inclusive production mode, photon isolation is a very useful tool to suppress the background, while keeping the Higgs signal eﬃciency reasonably high [48]. The isolation criteria are based on the information from Tracker [49]and ECAL [50].

The bunch length at LHC, σZ (bunch) ≈ 75 mm, results in a longitudinal spread of interaction vertices: σZ (interaction) ≈ 53 mm. If the mean longitudinal position is used when reconstructing the eﬀective mass of a narrow two-photon state, such as the H→ γγ, a large contribution to the reconstructed width is introduced - about 1.5 GeV. However the vertex of the Higgs could be found with the help of additional tracks in the same Higgs event [51, 52, 53]. The Higgs pT is balanced by the rest of the particles in the event and therefore the tracks associated to a Higgs event are harder than the tracks of a minimum-bias event [54]. Therefore the vertex can be identiﬁed by the hardest tracks of the bunch

36http://cmsdoc.cern.ch/orca 5.3 CMS At LHC 79

2000 120 Perfect vertex gluon fusion Events Events γ 1500 With correction 90 + jets

quark annihilation

1000 60 QCD

500 No correction 30 Higgs

0 0 100 105 110 115 120 0 50 100 150 200 Higgs invariant mass, GeV Jet PT, GeV Figure 5.16: Left plot - the reconstructed Higgs mass for three diﬀerent cases: i) exact knowledge of the primary vertex at generator level; ii) vertex selected using the highest pT track; and iii) - no knowledge of the vertex. Right plot - transverse energy distribution of the more energetic tagged jet resulting from VBF production followed by Higgs decay to two photons.

crossing. A simple algorithm is used for primary vertex determination in this study. The primary vertex is selected as the vertex of the track with the highest pT . The efficiency of the H→ γγ event vertex finding (with |Zmeas − Ztrue| < 2 mm) is ∼ 80% . The algorithm gives a vertex resolution of ∼ 60 µm. The left plot in Fig. 5.16 shows the reconstructed Higgs mass for three different cases: i) exact knowledge of the primary vertex from generator level; ii) corrected vertex by present algorithm; and iii) - no knowledge of the vertex. It is clear that the vertex reconstruction significantly improves the mass resolution. The mass resolution after vertex correction is equal to ∼800 MeV. Approximately 39 fb−1 of integrated luminosity is required for a 5 sigma discovery in the inclusive production mode [55]. For the Higgs search in pure VBF production mode the cross section is an order of magnitude lower than for inclusive production mode. However the VBF production mode has an additional distinctive feature, namely, the presence of two energetic forward and backward jets in the final state [46, 47]. These tagging jets provide an additional powerful tool to reduce the background significantly as in the background events the forward jets are not present. The selection is based on jet transverse momenta and directions, and the invariant mass of the tagged jets. The right plot in Fig. 5.16 shows the transverse momentum distribution of the more energetic tagged jet for the Higgs signal and all types of the background. The transverse momentum of tagged jets in the Higgs sample is much harder than in the background samples. About 41 fb−1 of integrated luminosity is required for a 5 sigma discovery using the VBF production mode alone [55]. Using the results for both Higgs production modes, the combined luminosity for 5 sigma discovery is approximately 30 fb−1. We can therefore hope to use these modes at LHC to cleanly detect the Higgs in the first two years of LHC operation, and to give an early indication of a Higgs discovery in combination with other less distinctive decay modes. The VBF mode also can be used to begin to provide information on the Higgs production and decay characteristics, to check that what is being seen 80 5 CMS at LHC and L3 at LEP2 is a Standard Model Higgs, in the early stages of LHC running. At the time of writing this report, we are continuing to simulate more background events in order to further tune the selection and to improve the required luminosity for 5 sigma discovery using both production modes. We will also study other decay modes, such as H → WW∗, in order to optimize the discovery potential in the mass range from 110 to 170 GeV.

Search for Extra Dimensions at LHC

Theories with large extra dimensions are proposed to solve the hierarchy problem between the electroweak scale, defined by the Higgs vacuum expectation value v = 246 GeV, and the Planck scale, which is ∼1018 GeV [56]. These models predict two types of four-dimensional massless excitations: the usual graviton and a graviscalar “Radion”. In order to stabilize the size of the extra dimensions without fine tuning of the model parameters, a mechanism has been proposed [57] by which the radion acquires a mass. The presence of the radion is one of the important phenomenological consequences of these models, and therefore the search for this scalar is a crucial probe of the model. Similar to the heavy Higgs in the MSSM, the radion can decay into Higgs pairs. The specific decay channel Radion → hh → γγb¯b gives an interesting signature with two high-pT photons and two b-jets. In addition to the photon isolation criterion, which strongly reduces the background [48], the b-tagging and the peak in invariant mass of two b-jets are very useful to further reduce the background. The same Monte Carlo background samples [48], which were simulated by the Caltech group to study the Higgs discovery potential in the two-photon decay modes, are now being used at Caltech to study CMS’ ability to discover the Radion, as a signature for extra dimensions.

ECAL Calibration with Inclusive η-particles

A precise calibration is of key importance to maintain the precision offered by the CMS PbWO4 crystal electromagnetic calorimeter. The intercalibration error goes directly into the energy resolution, so it is important to keep it at ≤0.5% level, not to degrade the resolution of Higgs mass, reconstructed from two photon decay. The standard CMS ECAL calibration for individual crystals is proposed to be done using electrons from W-decays [58]. The tracker momentum measurement is used to intercalibrate the ECAL crystals. However, the first estimations show that to accumulate the number of electrons per crystal required to determine the calibration constant with an accuracy of ∼0.5% level, one needs about two months [58]. Therefore additional calibration techniques using a reaction with a higher rate are needed to shorten the ECAL calibration time, and to make a good calibration possible at the low luminosities expected during the first days and weeks of LHC operation. The Caltech group therefore investigated use of inclusive η → γγ decays in QCD jet events, as a possible physics process for ECAL calibration. The idea is to reconstruct the invariant mass of η-particle from two photons and use the mass peak position for calibration, where: σcalib ≈ 2 × (ση/mη)/ Nγ The η → γγ process has several advantages:

• Only ECAL information is needed for this type of calibration37; • The photon reconstruction is less sensitive to the amount of material in the tracker, than the electron reconstruction; • The rate of the η → γγ process is much higher than the rate of W→ eν process, therefore more stringent selection could be used to select non-converted photons from η-particle decay. One can 37 Although use of the primary vertex position, selected using the highest pT track, has been shown to improve the accuracy 5.3 CMS At LHC 81

also lower the trigger thresholds in the early days of running, and get a quick ECAL calibration at a rate compatible with the Level 1 trigger.

50 25

Events ± Events M = 548 20 MeV M = 546 ± 24 MeV 40 20 σ M/M = 3.6%

30 15

20 10

10 5

0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 , Two photon candidate s invariant mass (GeV) Two photon candidate,s invariant mass (GeV)

Figure 5.17: Invariant mass of two photon candidates. Left plot shows the distribution before shower shape cut is applied, right one - after shower shape cut is applied.

The selection of η → γγ events is based on two isolated clusters in the ECAL which are close to each other, and are matched with the same trigger object [59]. To select isolated photons from η-particle decay and to reject non-isolated QCD jet background, a variable based on the shower shape of an electromagnetic cluster is used. Figure 5.17 shows the mass peaks of the η, reconstructed using the two photon candidates’ energies and directions in a QCD jet data sample, before and after shower shape selection is applied. A clean mass peak of the η-particle with little background is seen on the right plot of Fig. 5.17. This shows that a stand-alone ECAL calibration with an accuracy of 0.5% can be achieved in less than one day of dedicated running using the η → γγ process. Calibration in situ during standard CMS data taking runs requires further studies of the CMS trigger, and that is currently underway.

5.3.7 CMS EMU PRS, and DAQ Test Beam Work

Endcap Muon Slice Test DAQ

The slice tests, scheduled for October 2004, are an important part of the US CMS Trigger and DAQ commissioning effort. The goal is to integrate the ME and HCAL chambers, triggers, and DAQ on the surface of the CMS site to check as much as possible before moving the equipment underground. The ME test beam effort is led by Paul Padley of Rice University, and consists of people from Rice, UC Davis, Fermilab, Ohio State University, and the University of Florida. However, they needed software engineering experience, so in January 2003 Richard Wilkinson of Caltech joined, since Wilkinson had developed much of the ME offline software. Wilkinson’s contributions to the ME slice test and local DAQ have included:

• Redesigning and rebuilding detector electronics control software in C++, and integrating it into the XDAQ [60] framework. 82 5 CMS at LHC and L3 at LEP2

• Along with I. Suzuki of Fermilab, creating an interface to the XDAQ Event Builder which can combine data from multiple data and trigger trackﬁnder readouts. • Unpacking the data from the electronics into C++ structures. • Responsibility for writing the event data into a format readable by the CMS reconstruction program ORCA.

Slow Neutron Background in the Muon Chambers

A signiﬁcant source of background hits in the CMS muon chambers is the ”neutron background”; hits which result from the radiative capture of thermal neutrons produced in hadronic interactions by nuclei.

n → γ → e+e−

Because these neutrons may live up to a full second before interacting, they cannot be treated like in-time pileup collisions. CMS had been treating it using a parametrization of the patterns of generated hits derived in 1996 at UC Davis. However, this method is complex and had not been updated, since it needs to be done separately for each detector type in each of the muon subdetectors. Since the neutron background ﬂux is very sensitive to shielding changes, a more ﬂexible approach was needed. In response to these concerns in the Muon PRS group, Richard Wilkinson introduced a simpler approach. Patterns of simulated neutron hits are saved in a ROOT database, and added when needed. Using the actual hits rather than a parametrization will more accurately depict the correlations between the positions, energies, and directions of the hits, and will be easier to update for new geometries. This package will need to be updated when CMS changes its persistency solution to POOL38.

5.4 CMS Software, Computing and Grid Systems

The Caltech group continues to play leading roles in Computing and Software for CMS and US CMS since these activities began in 1995. Since 1998 it has developed a leading role in the development of Grid systems for CMS, LHC and the HENP community.

5.4.1 CMS and LHC Data Analysis Challenges

The distributed data analysis of CMS and the other LHC experiments will present unprecedented challenges in data access, processing and analysis on a worldwide scale. The CMS online systems will filter the 109 interactions per second produced by the LHC using dedicated hardware in the first level trigger (output 75 kHz), and farms of ∼ 4000 online processors will reduce the recorded rate to 100 MB/sec of raw data. This will result in ∼ 10 PB of accumulated raw and processed data stored in the first year of operation, in 2007. The offline processing power required for CMS reconstruction, simulation and analysis (not including workgroup servers at individual institutes and desktops) is estimated at 2 × 106 SI95, or the equivalent of approximately 50,000 of today’s PCs. The annual data volume is expected to increase rapidly in subsequent years, so that the accumulated data volume will reach 1 Exabyte before 2015, and the processing power is expected to increase correspondingly. Apart from the sheer volume of the data, the challenge of LHC computing arises from the global expanse of the collaboration (in 33 countries), the complexity of the detectors and the high luminosity

38See http://lcgapp.cern.ch/project/workbook/pool/current/examples.html 5.4 CMS Software, Computing and Grid Systems 83

LHC environment (an average of 17 pp interactions per crossing at design luminosity). This leads to a new scale, as well as a new scope of coordination and collaboration across world regions, for the software development, production processing, and data analysis eﬀorts.

The CMS Distributed Computing Model

In order to meet these challenges, and in response to the funding realities in the US and many other countries, the Caltech CMS group together with Fermilab, CERN, groups in Italy and elsewhere, developed a distributed Computing Model39 describing how software, networks and hardware would be used to support a timely and competitive data analysis by CMS physicists located throughout the world. This Model was originally conceived as being based on a number of Regional Centers, in addition to the CERN Center, interconnected by a system of high speed network links to provide adequate data access speed, storage capacity and computational power throughout the collaboration. The distributed Computing Model was quantiﬁed and studied, and the scope of the computing and networking requirements for the LHC experiments deﬁned, through the MONARC Project40 based at CERN (H. Newman, Spokesperson).

Organization of the CMS Computing and Software Eﬀort

The software and computing effort in CMS is organized into three projects: Computing and Core- Software (CCS), Physics Reconstruction and Selection (PRS), and TriDAS (Online Computing). These are closely connected and go under the joint acronym CPT. CMS chose from the outset to organize its Computing and Core Software (CCS) activities as a well- defined project, similar to the CMS subdetector projects. Direction and oversight were provided by the Software and Computing Board (SCB) of country representatives, which was led by H. Newman since its inception in 1996-2001. Technical issues were the purview of the Software and Computing Technical Board (SCTB)41 that acted in an advisory role to the Software and Computing Board (SCB)and the Project Manager. This organization successfully oversaw the transition to Object Oriented software and a supporting framework, the building of the ORCA reconstruction program, the interactive user environment IGUANA, and the development of network-distributed database systems and later Grid systems. In 2001 the CCS Task was reorganized to work more closely with the PRS and Trigger and DAQ (TriDAS) tasks, under the umbrella of a “CPT” project. At the start of 2002 D. Stickland of US CMS (Princeton) took over the CCS Project Manager’s job, and H. Newman became the CCS Institute Board (CCS-IB) Chair. With the launch of the LCG project in March 2002, a number of CCS tasks, especially in the AFT area, were recast in the form of Common Projects, with substantial manpower from CMS. In mid-2002, a further refinement of the CCS organization was proposed by CCS and CMS management, with a Technical Board (CCS-TB), Finance Board (CCS-FB), and a CCS Advisory Board (CCS-AB) that will participate in the definition of the high level CCS goals and report to the CMS Collaboration Board. The CCS is responsible for

39The ﬁrst conceptual Computing Model for CMS was presented in its Computing Technical Proposal (CTP), CERN/LHCC 96-45 (December 1996), available on the Web at http://cmsdoc.cern.ch/ftp/CMG/CTP/index.html. 40Models of Networked Analysis at Regional Centers. See http://www.cern.ch/MONARC. 41H. Newman and J. Bunn of our group were members of this board. 84 5 CMS at LHC and L3 at LEP2

• Defining and maintaining the worldwide software infrastructure required for efficient software development and deployment in CMS, including the information and documentation systems. • Establishing and deploying the Core-Software baseline of the experiment and assisting Detector, PRS and TriDAS groups in the case of their particular requirements • Developing the Computing Model of the experiment and validating it by a series of Computing and Physics data challenges • Preparing the CMS Computing and Software TDR • Liaising with the appropriate regional authorities to define, establish and monitor the distributed computing environment available to CMS. • Ensuring the safe and appropriately distributed storage of CMS data and data products • Liaising with all appropriate GRID projects to specify our requirements and test their products. This may involve trial deployments of their products, but typically not the development of the GRID middleware. • Building and deploying the tools that enable efficient use of the facilities available to CMS for data productions of all scales from central CMS, to Analysis groups, to individual physicists • Operating the productions dictated by central CMS requirements.

The PRS is responsible for

• Building and maintaining the Physics and Detector speciﬁc software of CMS for all aspects of Simulation, Reconstruction, Calibration and Physics Analysis. • Preparing and carrying out the work leading to the Physics TDR • Deﬁning the production requirements and time schedules for satisfaction of various CMS milestones, and eventually real data reconstruction and analysis. • Carrying out the Physics Analysis requirements of the collaboration • Verifying the physics performance of the Filter Farm and its algorithms during its operation. • Assuring the quality and integrity of the acquired data.

And the TriDAS is responsible for

• Designing, prototyping and operating the ﬁlter farms of the HLT/DAQ chain • Safe storage of the experiment data at the experiment site • Bringing the data out of the experiment to the oﬄine center

In late 2002, the eﬀort in CMS dedicated to PRS was reorganized, and a new work plan deﬁned. In particular, four new PRS Groups were created:

• Heavy Ions • Higgs • Standard Model • SUSY and other Beyond Standard Model

The new work plan includes (1) selecting analyses for the Physics Technical Design Report, (2) work on OSCAR, deﬁned as being the most important software deliverable for 2003, (3) transition to post-Objectivity persistence mechanisms, (4) selection of physics topics, completing the Fast Monte Carlo (FAMOS) and launching of analyses. 5.4 CMS Software, Computing and Grid Systems 85

5.4.2 CMS Software and Computing Schedule and Milestones: Data Chal- lenges and Technical Design Report

The schedule for development of the CMS Computing Model, data analysis and Grid-related software systems is set by the need to be fully ready for physics in time for LHC startup in 2007. Intermediate major milestones (from 1999 on) are set by the need to support distributed production and analysis of simulated events in order to prepare and verify the necessary online trigger, DAQ and oﬄine systems (as will be documented in Technical Design Reports (TDRs)), and to verify the full range of the experiment’s capabilities to trigger and do the physics eﬃciently. A series of Data Challenges scheduled to be completed in 2006, progressing in steps with an increasing scale and complexity, will verify the design and functionality of the distributed computing systems and the associated Grid software. Figure 5.18 shows the Data Challenges time line.

Magnet, May 1997 1997 HCAL, June 1997 ECAL & Muon, Dec 1997 1998 Tracker, Apr 1998 Addendum: Feb 2000 1999

2000 Trigger, Dec 2000 2001

2002 TriDAS, Dec 2002 2003 DC04 Computing TDR, Oct 2004 (C-TDR validation) 2004 DC05 LCG TDR, Summer 2005 (LCG-3 validation) (Computing TDR as input) 2005 Physics TDR, Dec 2005 2006 DC06 (physics readiness) CMS Physics! Summer 2007 2007

Figure 5.18: Showing the schedule of CMS Data Challenges.

The DC04 Data Challenge

The DC04 data challenge (February 2004) is scaled to be 25run for one month. The target of 50,000,000 events to be reconstructed in DC04 is thus equivalent to a Luminosity of 2 × 1033cm−2s−1. The data rate will thus be equivalent to 25Hz. The events are to be reconstructed at the Tier0 center (CERN), and a few Tier1 and Tier2 centers. One quarter of the total events is allocated to each of the four PRS 86 5 CMS at LHC and L3 at LEP2 groups. DC04 will use OSCAR for simulation of the events, and ORCA for their reconstruction. ORCA will include the new POOL persistency mechanisms with ROOT I/O and basic services from SEAL (there will be no use of Objectivity in DC04). To prepare the events for DC04 reconstruction will require a massive OSCAR production in the months leading up to February 2004, starting in July 2003.

Computing Technical Design Report(TDR)

The Computing Technical Design report is due to be completed in October 2004. It will include the technical speciﬁcations of the computing and core software systems which will be used for the DC06 Data Challenge. In Figure 5.19 we show the strategy for gathering the requirements for simulation, reconstruction and physics analysis, that will govern the details of the Computing TDR.

Technologies Evaluation and evolution Estimated Physics Available Model Computing Model Resources •Architecture (grid, OO,…) (no cost book for •Data model computing) •Calibration •Tier 0, 1, 2 centres •Reconstruction •Networks, data handling Iterations / •Selection streams •System/grid software scenarios •Simulation •Applications, tools •Analysis •Policy/priorities… Required •Policy/priorities… resources Validation of Model

DC04 Data challenge Simulations Copes with 25Hz at Model systems & 2x10**33 for 1 month usage patterns

C-TDR • Computing model (& scenarios) • Specific plan for initial systems • (Non -contractual) resource planning

Figure 5.19: The Strategy for generating the Computing Technical Design Report, due in 2004. 5.4 CMS Software, Computing and Grid Systems 87

Grid Enabled Analysis challenges

With progress on Grid-enabled production well underway and attention now turning to the issues of providing for Grid-enabled data analysis (which we discuss in detail in the later sections of this report), it is expected that “Analysis Challenges” will in the future be added to the major milestones listed above.

5.4.3 Progress in the Development and Application of CMS Core Software

CMS Core Software is now well-advanced42. The Core Software team has completed the second major object-oriented software development cycle, the “functional prototype” phase, including the highly functional COBRA43) framework, and is well into the third phase which aims to provide “fully functional software”. The CMS reconstruction code ORCA (Object Reconstruction for CMS Analysis) is now in its sixth major release. Each major ORCA release is interleaved a large scale production “challenge” where several million events are simulated and reconstructed in support of the efforts of the Physics Recon- struction and Selection (PRS) groups. The interactive graphics analysis environment (IGUANA) is also being used increasing widely among the PRS community. A software QA process based on ISO standard IEC-15504 has been put in place to handle the complexity of CMS software, which now includes more than 1 million lines of C++ user code, managed in 15 repositories. Within the last year, the progress in developing the GEANT4-based CMS simulation OSCAR44 has accelerated. The full chain from generator to OSCAR to ORCA reconstruction, within the COBRA framework, is now working for all subdetectors. Many GEANT4 problems have been ironed out so that we now have a (technically) working full object-oriented simulation program for CMS. OSCAR validation of the geometry and response for the muon and tracker subsystems is ongoing, and validation for the calorimeters has started. Overall the goal is to replace the CMSIM (Fortran and GEANT3-based) simulation by OSCAR by the end of 2003. In the Fall of 2001 CMS decided to change its database strategy, moving away from Objectivity in favor of a hybrid solution. A project named POOL (Pool Of persistent Objects for LHC) was started at the end of 2002 to combine the efforts of the different LHC experiments to produce a common persistency layer. The implementation of POOL is well advanced with a public 1.0.0 release having been made on May 13. This hybrid persistency layer consists of Relational Database Management Systems (RDBMS) for metadata, and a combination of approaches for the bulk data, which includes the use of ROOT files and traditional RDBMS like Oracle 9i, SQLServer, MySQL and PostgreSQL. For example, objects stored in Oracle 9i will be interfaced to CMS software through an object-streaming service and will be accessed by users doing analysis through a well-developed front end such as ROOT. This is more fully described in Section 5.4.10. CMS is deeply involved in POOL as well as other joint LHC projects under the banner of the LHC Computing Grid (LCG)45, started to maximize LHC software development resources. CMS- developed software was also contributed to the SEAL Core Libraries and Services Group, which provides a consistent application library as a basis for experiment-specific software, while the Physics Interface (PI) project is headed by the CMS chief software architect.

42Although a shortage of software engineers, especially those outside the US and CERN, needs to be resolved if progress at the required pace is to continue. 43Coherent Object-Oriented Base for Reconstruction, Analysis and Simulation. See http://cobra.web.cern.ch/cobra 44Object Simulation for CMS Analysis and Reconstruction. 45see http://cg.web.cern.ch/LCG/ 88 5 CMS at LHC and L3 at LEP2

5.4.4 US CMS Software and Computing Project

The plans for a US CMS Software and Computing Project46 were initiated by Caltech, Northeastern and Fermilab in 1998. The first presentation on the need for an LHC Computing and Software Project was presented (by Caltech and Northeastern) to DoE and NSF in August 1998, with a focus on the need for software professionals. Caltech together with Fermilab led the efforts to form the Project and the Peer Reviews of CMS Software and Computing conducted jointly by DOE and NSF in November 1998, May 1999 and January 2000, along with periodic reports to the FNAL Project Management Group (PMG) and the DOE/NSF Joint Oversight Group (JOG). In January 2000 M. Kasemann (Head of FNAL Computing Division) stepped in as Acting Level 1 Project Manager, and Caltech (H. Newman) continued in a leading role as Acting Chair of the US CMS Software and Computing Board, up to the election of the permanent USASCB47 (the US Advisory Software and Computing Board) in the Fall of 2000. The Project formally began with the appointment of the Level 1 Project Manager at FNAL (L. Bauerdick). In November 2000 the project was “baselined” for FY 2001-2002. In November of 2001 DOE and NSF agreed to a baseline plan for the Project that was consistent with earlier agency funding guidance, covering the period up to the first year of LHC operations. In May 2002, Dan Green (FNAL) was appointed US CMS Research Program (RP) Manager, over- seeing the Software and Computing, Maintenance and Operations, and R&D for detector upgrades, as well as the US CMS Construction Project (CP). With this unified management, strongly supported by the US CMS Collaboration, a more coherent handoff between the Construction Project and the Opera- tions period of CMS became possible. By April 2003, funding profiles from DOE and NSF for Software and Computing through FY08 had been obtained. By stretching out the final stages of the Construction Project, the minimum Software and Computing Project support levels have been maintained during the critical transition period to rampup, in 2002-2004, as shown in Figure 5.20.

5.4.5 Summary of Caltech Contributions to CMS Software and Computing Model R&D

Caltech has established longstanding leading roles in many of the main activities in computing and software for CMS, the LHC program and the HENP community, some of them discussed in this section, and some in the LHCNet chapter of this Report. This has been made possible in part by the GIOD project funded by HP, the use of facilities and work in collaboration with Caltech’s Center for Advanced Computing Research (CACR), the development and operation of the ﬁrst CMS prototype Tier2 center together with UCSD, facility grants for computing power on a large scale received by our group from NSF’s National Partnership for Advanced Computing Infrastructure (NPACI), a grant from the NSF/KDI project (ALDAP) to develop novel database structures for eﬃcient data access and analysis in HEP and astrophysics, grants from the GriPhyN, iVDGL and PPDG Projects, and our group’s management and operation of the CERN-US network link funded partly by DoE and NSF. Examples of the contributions of the Caltech group to the CMS core software, development of the CMS reconstruction and analysis codes, as well as database and Grid system design, development, and prototyping include:

• Development of the distributed LHC Computing Model and deﬁning its scope and characteristics,

46Further details on the initiation, organization and structure of the US CMS Software and Computing Project, and references to associated documents, were presented in last year’s Annual Report. 47According to the US CMS Constitution, the Chair of the Collaboration Board cannot also serve in another elected oﬃce at the same time. 5.4 CMS Software, Computing and Grid Systems 89

US CMS RP Funding Profile 40 35 NSF 30 DOE 25 20 15 10 5 Funding (AY Funding (AY $M) 0 FY02 FY03 FY04 FY05 FY06 FY07 FY08

US CMS CP Funding Profile 40 35 NSF 30 DOE 25 20 15 10 5 Funding (AY $M) Funding (AY 0

FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07

Figure 5.20: The DOE and NSF funding profiles for the US CMS Research Program and the Construc- tion Project, showing the handoff that maintains the necessary S&C manpower effort in 2002-4.

since originating the Data Grid Hierarchy concept and model in 199948 • Development of CMS Core Software for the Hadron Calorimeter (V. Litvin) • Production of OO prototype tracking codes in CMS, including a prototype code for full tracking in CMS (J. Bunn, R. Wilkinson) • Development of the ﬁrst prototype of a networked Terabyte-scale object database, containing fully simulated CMS events and event-tags (Bunn, K. Holtman, Wilkinson)

48See l3www.cern.ch/ newman/uscmssw/nsfmre talkpapermay99.doc 90 5 CMS at LHC and L3 at LEP2

• Deployment of this prototype to institutes in the MONARC project, for testbed evaluations of distributed object databases (Bunn, Newman, I. Legrand) • Modeling and optimization of distributed data access scenarios in the GIOD (Bunn) and MONARC (Legrand) Projects, as well as the US CMS Software and Computing Project • Development of systems for efficient data file-level and object-level data access (Bunn, Holtman, Newman) • high throughput data transfers over wide areas networks (Newman, Galvez, Ravot, Bunn)in collaboration with SLAC and CERN • Shared responsibility for Object persistency (R. Wilkinson) as part of the CMS Object Oriented software effort; • Work on writing the OO tracking code for the forward muon system (Wilkinson), together with physicists at UCLA, UC Davis and Florida • Higgs → γγ detection studies including work on γ/π0 separation (S. Shevchenko, N. Wisniewski, Newman, Wilkinson) • Work on photon reconstruction (Shevchenko, A. Favara, Wisniewski and Caltech students J. Pino, and T. Lee). • Installation, development and operation of a Tier2 center, integrated with CACR, HEP, national and international high performance networks, participating in CMS simulation production, the US CMS iVDGL Grid testbed, and supporting the data analysis efforts by US CMS physics groups in California (Bunn, Singh, Newman, Ravot) • Development of the endcap muon DAQ software for the 2004 slice tests (Wilkinson). • Work on defining the first Grid-enabled Analysis Environment (GAE) for CMS and the LHC experiments, including development of the new CMS persistency strategy of extracting and delivering Object collections from an RDBMS (Newman, Bunn, Steenberg, Aslakson, Soedarmadji, Iqbal).

5.4.6 Development of the “Data Grid Hierarchy”

As the size of the LHC computing requirements came into focus in 1998-9, the concept of Computational Grids developed by Foster, Kesslman and others and began to have a major impact on computational science and engineering49. The CMS Caltech group then developed the “Data Grid Hierarchy” con- cept50 for LHC and HEP, where the resources are organized in “Tiers”, as shown for the CMS case in Figure 5.21. A key feature in this hierarchy are the Tier2 centers: relatively low-cost systems based on commodity rack-mounted PC systems, small data servers, and RAID arrays of disks interconnected by Ethernet that are dimensioned to be manageable by a small staﬀ at a university. The Data Grid concept, now recognized by CERN as the one to be used by all four LHC experiments51, led to the GriPhyN project supported by NSF, the DOE-supported PPDG project, and more recently the EU DataGrid, iVDGL and the LHC Computing Grid projects as well as many national Grid eﬀorts in the UK, Italy, France and many other countries involved in the LHC program. The rise of the tiered Data Grid concept also was a driving force behind the plans for Tier1 and Tier2 centers in the US, UK, France, Italy, Germany, and many other countries. Over the last four years, the Caltech group has worked with groups in China, India, Pakistan, Brazil and Korea to plan, fund, design, and deploy their

49See “The Grid: Blueprint for a New Computing Infrastructure”, Edited by I. Foster and C. Kesselman, Morgan Kaufman, San Francisco, 1999. 50In a White Paper presented by H. Newman at NSF headquarters in June 1999. 51See the Report of the Steering Group of the LHC Computing Review” (CERN/LHCC/2001-004) at http://lhc-computing-review-public.web.cern.ch/lhc-computing-review-public/Public/Report ﬁnal.pdf . 5.4 CMS Software, Computing and Grid Systems 91 regional centers.

CERN/Outside Resource Ratio ~1:2 ~PByte/sec Tier0/(Σ Tier1)/(Σ Tier2) ~1:1:1

Online System ~100-1500 MBytes/sec Experiment CERN Center Tier 0 +1 PBs of Disk; Tape Robot Tier 1 ~2.5-10 Gbps

IN2P3 Center RAL Center INFN Center FNAL Center 2.5-10 Gbps Tier 2 Tier2 CenterTier2 CenterTier2 CenterTier2 CenterTier2 Center Tier 3 ~2.5-10 Gbps

Institute Institute Institute Institute Tens of Petabytes by 2007-8. Physics data cache 0.1 to 10 Gbps An Exabyte ~5-7 Years later.

Workstations Tier 4

Figure 5.21: The Data Grid Hierarchy Computing Model for CMS. This model has been adopted by all four LHC experiments, and the concept emulated by other major HENP experiments.

The CMS Data Grid is planned to include the equivalent of 5 full-scale Tier1 national centers, including the US Tier1 at Fermilab and others in UK, France, Italy, and Germany, Russia. Other centers such as one in Brazil are being planned. We are also planning on approximately 25 Tier2 centers. The first prototype Tier2 center, funded by NSF, has been deployed and brought into production use in two halves, at Caltech’s Center for Advanced Computing Research (CACR) and the San Diego Supercomputer Center (SDSC), interconnected at OC12 (622 Mbps) over CALREN-2. The extensive R&D work carried out on our prototype Tier2 is summarized briefly in Section 5.4.11. The network link bandwidths shown in the figure are the baseline requirements for a single LHC experiment (CMS or ATLAS), corresponding to the US-CERN link reaching 10 Gbps by 2005, and approximately 40 Gbps (for all of HENP) by 2007. These requirements and the likely need for greater bandwidths on the major national and transoceanic links are discussed further in the LHCNet chapter of this report.

5.4.7 Maintaining a Physics Focus in the US: Collaboratories, VRVS, PAC and VCR

The US Tier1 and Tier2 centers will serve as natural nucleation points for physics analysis. This will keep the center of gravity of U.S. HEP within the US, and serve to mitigate the additional costs for students and postdocs working at CERN. We in US CMS consider it vital that the Tier1 and Tier2 92 5 CMS at LHC and L3 at LEP2

centers be structured so that a “critical mass” of US physicists will work together, enabling them to take leadership roles in the physics analysis. To support this, we are planning and engaged in several activities which include collaboratories, VRVS, a Physics Analysis Center, Virtual Control Rooms, and projects based on Collaboratories and global peer-group support for analysis teams.

Collaboratories and VRVS

It should be possible to accommodate the LHC accelerator community in the US in their desire to participate remotely in LHC accelerator experiments. By developing the Web-based tools of a “collaboratory”52 adapted to HENP’s needs, plus VRVS, we will aim to empower US physicists to participate fully in LHC physics, including from their home institutions. In this area, we are leading proponents of the ”GECSR” and ”DoVES/DAWN” project proposals submitted to the NSF/ITR program, which are described more fully in Section 5.4.13.

Physics Analysis Center and Virtual Control Room

In order to reinforce the focus of efforts in the US, it is planned to set up a remote control room at Fermilab, using the concept developed at KEK for remote participation in CDF operations as a starting point. Using Web-based based technology, the concepts of a collaboratory, and Caltech’s VRVS system including its extension to handle virtual spaces (shared desktops)53, we will be able to extend the remote control room idea to all the Tier1 and Tier2 centers jointly, as well to the desktop of individual experts. Making this model a success will require effort and ingenuity, and a clear and effective concept of the Grid-enabled Analysis Environment54. The Physics Analysis Center and Virtual Control Room are part of the Research Program in the M&O effort. Initially, the target customers for the VCR will be HCAL and ME since they will soon be performing ”slice” tests and electronics burn in - and will need remote monitoring. The customers for the PAC are the jet/met and muon groups who will be analyzing test beam data from the 40 MHz 2003 running, and preparing for DC04.

5.4.8 CMS Data Grid Architecture, Requirements and Software

During the past year we continued to make major contributions to the deﬁnition of Grid architecture for HEP experiments, and to the interface between CMS software and the Grid in particular. We also took a central role in helping to deﬁne the new strategy and implementation for large scale data storage and access using distributed relational databases (RDBMS) and object-collection extraction and/or delivery with the aid of Grid tools and Web services.

Architecture and Requirements

The CMS computing model as documented originally in 1996 in the CMS Computing Technical Proposal did not take the emergence of Grid technology into account. In 1998-2000, H. Newman led the MONARC project55 that developed the LHC distributed Computing Model that was adopted by the 4 LHC

52See for example: the Space Physics & Aeronomy Research Collaboratory at http://www.windows.ucar.edu/sparc/, and the recent work by our group on Web services discussed in Section 5.4.10. 53See the LHCNet chapter in this report. 54See Section 5.4.10 and http://pcbunn.cacr.caltech.edu/GAE/GAE.htm 55Together with L. Perini of ATLAS. 5.4 CMS Software, Computing and Grid Systems 93 experiments. In 1999 Caltech (Newman, Bunn) developed the Data Grid Hierarchy idea that is the Grid-based implementation of the Model, and the basis of planning for LHC Computing. Since then, the Caltech group has played a leading role in evolving the architectural concepts for how CMS software and computing will encompass Grid technology. The new vision sees CMS computing as an activity that is performed on the ‘CMS data grid system’. The properties of the CMS data grid system were defined in considerable detail in CMS note 2001/037 [74], which was authored primarily by the Caltech group. The document specifies a division of labor between the Grid projects and the CMS core computing project. This is crucial for detailed planning and manpower estimation, and also for steering the work of the Grid projects in which CMS is a partner. The CMS Grid vision is widely recognized as being the most detailed and complete vision of the use of Grid technology among the LHC experiments. During the past year we have continued to develop the concept of Grids for major physics experiments as (necessarily) end-to-end managed systems, due to the limited computing, data storage and network resources available, and Collaboration policies on resource sharing and task priorities that need to be taken into account. This is particularly important for Grid-enabled analysis, and our GAE efforts. We therefore developed the Dynamic Distributed Services Architecture56, and the MonALISA monitoring system57 to monitor and track the performance of the networks and Grid facilities in real time, as discussed at the last sections of this chapter. Higher level services being developed in MonALISA aim and to provide interactive decision support and eventually some automated management of the workflow through the system, so that physicists are able to develop effective strategies for carrying out their analysis tasks efficiently, while dealing with large data samples.

5.4.9 Demonstrations of Grid-Based, Object-Oriented Physics Analysis Soft- ware

GAE Demonstration at SC2002, Baltimore, November 2002

A distributed real-time physics analysis of simulated CMS Jet and Missing E T data was demonstrated at the SuperComputing 2002 conference, as part a wider Grid-based CMS data production run and demonstration being conducted at the time58. Data from the distributed production run were made available to analysis applications via the Cal- tech Clarens server software. On the show ﬂoor a Grid-enabled version of the ROOT analysis packages was used to download the production data and construct local collections to be analysed. The analysis results were graphically displayed in real time on two workstations as part of the Fermilab/SLAC and Caltech booths in Baltimore. A similar demonstration was given at the iGrid2002 meeting in Amsterdam in September 2002. Another part of the demonstration used distributed databases at Caltech, CERN, UCSD and other HEP institutes to distribute selection queries via the Clarens. These queries created virtual data collections at these sites, which were subsequently moved across the WAN using a specially enhanced TCP/IP stack (FAST TCP59), and rendered in real time on an analysis client workstation in Baltimore. This demonstration’s purpose was to preview two aspects Grid-enabled analysis of general interest to LHC: distributed data data access with local processing, and distributed processing with local access

56See “Data Intensive Grids for High Energy Physics” by H. Newman and J. Bunn in “Grid Computing: Making the Global Infrastructure a Reality”, edited by F. Berman, G. Fox and T. Hey, Wiley & Sons, March 2003. 57See http://monalisa.cern.ch/MONALISA 58There were also a total of 24 demonstrations from the Particle Physics Data Grid (PPDG) project that is led by Caltech (H. Newman), SLAC (Mount) and Wisconsin (Livny). 59See http://netlan.caltech.edu/FAST 94 5 CMS at LHC and L3 at LEP2 to results. The client hardware on the show floor consisted of a head node for display purposes (running the ROOT and MonALISA applications) and twelve late-generation dual processor servers each equipped with a SysKonnect SK9843 Gigabit fiber interface. These servers were running the Linux OS with the Caltech FAST TCP enhancements. Remotely, at Starlight, Caltech, CERN, Florida we made use of Linux and Windows-based servers that were hosting Oracle, SQLServer and PostgreSQL databases, all running the /em Clarens server software that acted as intermediary between the databases and file data, and the ROOT clients. During the course of the SC2002 demonstration the same hardware was used to break the Internet2 Land Speed Record60 thanks to the above mentioned FAST TCP enhancements.

5.4.10 Deﬁning the CMS and LHC Distributed Analysis Environment

Work on the Grid Analysis Environment (GAE) is now gathering momentum: its importance is hard to overestimate. While the utility and need for Grids has been proven in the production environment (by the Caltech group, US-CMS, CMS and the LHC experiments as a whole), their significance and critical role in the area of physics analysis has yet to be realized. The Caltech group is playing a leading role in the prototyping and development of a Grid-based global system for analysis. We have been the first of the LHC experiments’ groups to demonstrate with real world examples, the benefit that the integration of traditional and newer analysis tools brings. The work on GAE at Caltech is a natural progression from our pioneering project activities GIOD and ALDAP (now completed, and described in detail in previous Annual Reports), our recently funded project CAIGEE [ibid], our collaboration in PPDG, GriPhyN, iVDGL [ibid] and more recently the proposed GECSR and DAWN [ibid] projects. The development of the GAE is the acid test of the utility of Grid systems for science. The GAE will be used by a large, diverse community. It will need to support hundreds to thousands of analysis tasks with widely varying requirements. It will need to employ priority schemes, and robust authentication and security mechanisms. And, most challenging, it will need to operate well in what we expect to be a severely resource-limited global computing system. So we believe that the GAE is the key to success or failure of the Grid for physics, since it is where the critical physics analysis gets done, where the Grid end-to-end services are exposed to a very demanding clientele, and where the physicists themselves have to learn how to collaborate across large distances on challenging analysis topics. In the following Figure61 we show a schematic representation of how the dynamic process of supporting analysis tasks being carried out by peer-groups of varying sizes within a worldwide community of physicists might look. The diagram shows a ”snapshot” in time of analysis activities in the experiment. Groups of individuals, separated by large geographic distances, are working on specific analysis topics, Supersymmetry, for example. Resources in the Grid system are being shared between the active groups. The dashed line boundaries enclosing each of groups, move and change shape and size as the composition or requirements of the groups changes. In our work at Caltech, we are building an environment like that shown in the Figure, that consists of tools and utilities that integrate with existing analysis software, and which expose the Grid system functions, parameters and behavior at selectable levels of detail and complexity. This is achieved by the use of Web Services, which are accessed using standard Web protocols. A physicist is thus able to interact with the Grid to request a collection of analysis objects, to monitor the process of preparation and production of the collection and to provide ”hints” or control parameters for the

60See http://lsr.internet2.edu/ and http://www-iepm.slac.stanford.edu/lsr2/ 61From L. Bauerdick, FNAL. 5.4 CMS Software, Computing and Grid Systems 95

Figure 5.22: A set of dynamically changing analysis groups coexist in a global, Grid-enabled Analysis Environment. The schematic shows the sharing and geographic dispersion of people and resources.

individual processes. The GAE needs to provide various types of feedback to the physicist through a set of “Grid Views” (under development), such as the time to completion of a task, the evaluation of the task complexity, diagnostics generated at the different stages of processing, real-time maps of the global system, and so on. These are areas we are working on presently. We believe that only by exposing this complexity at the outset, can an intelligent user learn to develop reasonable work-strategies in dealing with the highly constrained global system we expect to have for the LHC computing tasks62. This work, which includes the development of Clarens and SOCATS [ibid], is already being adopted within CMS, and generating considerable interest from the other LHC experiments. In June 2003 we will hold a workshop whose topic will be the development of a detailed specification of GAE architecture, including the identification of existing components, missing pieces, and a roadmap

62In the longer term, as real-world experience with Grid-enabled analysis is gained, we hope to develop standardized guidelines for users, and then to progressively automate at least part of the work-planning, with an aim to reduce the turnaround time encountered by physicists for a variety of analysis tasks, by improving the ﬂow of work through the distributed system. This is discussed further in the last two sections of this chapter. 96 5 CMS at LHC and L3 at LEP2 for completing a ﬁrst-cut system for CMS as soon as possible. The attendees at this workshop will be drawn from CMS and ATLAS, as well as PPDG and the CERN LCG project, to ensure a balanced approach.

GAE Components

CAIGEE Architecture

Laptop ROOT Browser PDA

Web Client Web Client Web Client Desktop Peer Group

Clarens Grid Services Super Peer Group Web Sever DAGs CMS Apps

File Transfer

Others MonaLisa

From GridPhyn, iVDGL, Globus etc

Caltech/CMS Developments

Figure 5.23: The GAE components, as identiﬁed in CAIGEE, the current best candidate architecture for the Grid Analysis Environment. 5.4 CMS Software, Computing and Grid Systems 97

Web Services

One of the key aspects of our work on developing a distributed physics analysis environment for CMS is the use of Web Services. Since beginning this work in 2000, we have made rapid and sustained progress in showing the feasibility of using Web Services for physics analysis data access. Web services are computing services offered via the Web63. In a typical Web services scenario, an end-user application sends a request to a service at a given URL using the Simple Object Access Protocol (SOAP64) over HTTP. The service receives the request, processes it, and returns a response. An often-cited example of a Web service from the business world is that of a stock quote service, in which the request asks for the current price of a specified stock, and the response gives the stock price. This is one of the simplest forms of a Web service in that the request is satisfied almost immediately, with the request and response being parts of the same method call. Web services and consumers of Web services are typically different organizations with diverse software and hardware platforms, making Web services a potentially very viable means of accessing and gathering results from distributed computing and data handling resources. An institution or organization can be the provider of Web services and also the consumer of other Web services. For example, a group of researchers could be in the consumer role when it uses a Web service to read and analyze data from other service providers, and in the provider role when it supplies other researchers with the final product of the analysis. In our GAE work, different types of data ranging from detailed event objects stored in Objectivity ORCA databases, as well as Tag objects stored in Objectivity Tag databases have been converted into prototypical Web Services. We developed a set of tools that allowed lightweight access to detailed event objects through Web Services. These Web Services provided access to data ranging in granularity from the Federation metadata to the hits and tracks of individual events. The data accessed in this way could then be used by a variety of tools and software programs, with relative ease. In 2002 we successfully provided distributed access, via a Web Service, to FNALs JetMet Ntuple files, produced from the analysis of the High Level Trigger (HLT) data simulated and reconstructed in preparation for the DAQ TDR. This Web Service was implemented using both an SQLServer database backend running under Windows, and also using an Oracle9i database backend running under Linux. The user’s view of the interface was identical, so demonstrating the ease with which we were able to hide the heterogeneity and details of the databases at different sites.

Clarens

The Clarens project, started in the spring of 2001 as a remote analysis server, grew in 2002-3 into a Grid-enabled web services layer, with client and server functionality to support remote analysis by end-user physicists65. Clarens is currently deployed at CMS sites in the US, at CERN, as well as Pakistan. The server architecture was changed from a compiled CGI executable to an interpreted Python framework running inside the Apache http server, improving transaction throughput by a factor of ten. A Public Key Infrastructure (PKI) security implementation was developed to authenticate clients using certiﬁcates created by Certiﬁcate Authorities (CAs). All client/server communication still takes place over commodity http/https protocols, with authentication done at the application level. Authorization of web service requests is done using a hierarchical system of access control lists for

63With or without a Web browser. A modern definition (from the IEEE) of Web services is simply “services that talk to other services”. 64See http://www.w3.org/TR/SOAP/ 65The Clarens project web page with mailing lists, source archives (CVS) and bug tracking system is hosted at http://clarens.sourceforge.net. 98 5 CMS at LHC and L3 at LEP2 users and groups forming part of a so-called Virtual Organization (VO). As a side-effect, Clarens offers a distributed VO management system with the delegation of administrative authorization tasks away from a central all-powerful administrator, as is appropriate for a global physics collaboration. Server-side applications made available through Clarens include the obligatory file access methods, proxy certificate escrow, access to RDBMS data access through SOCATS, SDSC Storage Resource Broker (SRB) access, VO administration and shell command execution. Users on the Clarens-enabled servers are able to deploy their own web services without system administrator involvement. All method documentation and their APIs are discoverable through a standard interface. Access to web service methods is controlled individually through the ACL system mentioned above. The services described above are available from within Python scripts, C++, as well as standalone applications and web browser-based applets using Java. A Root-based client was used to demonstrate distributed analysis of CMS JetMET data at the Supercomputing 2002 conference in Baltimore, MD. Clarens was also selected to be part of the CMS 2004 data challenge (DC04) in 2004.

GroupMan

The GroupMan application was developed in response to a need for more user-friendly administration of current-generation LDAP-based virtual organizations. GroupMan can be used to populate the LDAP server with the required data structures and certificates downloaded from Certificate Authorities (CAs). Certificates may also be imported from certificate files in the case of CAs that do not offer certificate downloads. These certificates can then be used to create and manage groups of users using a platform-independent graphical user interface written in Python. The VO data is stored in such a way that it can be extracted using standard Grid-based tools to produce so-called gridmap files used by the Globus toolkit. These files map host system usernames to individuals or systems identified by their certificates, thereby providing a coarse-grained authorization mechanism.

SOCATS

We have begun development of a general purpose tool to deliver large SQL results-sets in a binary optimized form. This project is called SOCATS. SOCATS is an acronym for STL Optimized Caching and Transport System. The main purpose of SOCATS is to deliver results from heterogeneous relational databases to C++ clients, in the form of binary optimized STL vectors and maps. The data returned from the SOCATS server to the client will be described through standard web service Web Services Definition Language (WSDL66, but the data itself will be delivered in binary form. This will save the overhead of parsing large amounts of XML tags for large datasets. It will also reduce latency problems for WAN environments, in that large batches of rows which efficiently fill the network pipe will be transferred together. We intend to utilize Clarens as our rpc (Remote Procedure Call) layer for SOCATS.

GAE Tool Prototypes

We have been working on developing various tools as candidates for inclusion in the future GAE. One example (discussed further in the LHCNet chapter of this report) is the construction of a four- screen desktop analysis setup that works oﬀ a single graphics card. The 4-way graphics card used allows

66See http://www.w3.org/TR/wsdl 5.4 CMS Software, Computing and Grid Systems 99 an aﬀordable setup to be built that oﬀers enough screen space and pixels for most or all of:

• Traditional analysis tools (e.g. ROOT session)

• Software development windows: code, debug, execution, etc.

• Event displays (IGUANA)

• “Grid Views”: monitoring information like MonALISA displays or processed monitoring results

• Persistent collaboration – VRVS session(s); VNC sharing other’s desktops, etc.

• Online event or detector monitoring information from CMS (possibly more shared desktops)

• Web browsers, Email, etc.

Our prototype setup works on a desktop with four 20” displays for a cost of about $6-8 k. One can imagine variations where this works in small and large meeting rooms, by using diﬀerent displays. The cost of such a setup is certain to fall in the future. Another example tool is a handheld analysis client, targeted for the Pocket PC platform. We have been working closely with our colleagues at NUST in Pakistan on a prototype implementation, which is now in an alpha release. The analysis software is JAS (Java Analysis Studio), which was successfully ported to the Pocket PC by NUST. This software runs on the handheld device, and communicates as a client with a Clarens server. This enables the JAS client to fetch histogram and other data from the server, and manipulate and render it on the Pocket PC by use of the stylus. Connectivity with the Clarens server is over standard TCP/IP sockets, and Grid authentication is being built in. Either wireless or wired network connections to the Pocket PC are possible by use of an appropriate Compact Flash format network card. An example screen display from the alpha version of the JAS analysis client running on the Pocket PC is seen below.

5.4.11 Tier2 Center and Network-Based Systems Update

The Caltech Tier2 was conceived in the Fall of 2000, and developed, together with a similar installation at SDSC, as the ﬁrst prototype Tier2 center for the LHC, and CMS in particular. During the past year it has been used for:

• the Spring 2002 CMS production run (213,000 simulated events, 149,000 Hit formatted events, and 146,000 2 × 1033PUevents),

• Grid production runs as one of the US CMS ”Integrated Grid Testbed” (IGT) sites.

• shipping to both the Tier0 (CERN) and Tier1 (Fermilab) centers the data products from the above production runs, using both conventional and GridFTP protocols.

• development and deployment of various emerging technologies (new motherboards in computing nodes; ATA RAID arrays tuned for high performance; new network interfaces, drivers and protocol stacks) aimed at optimizing the performance of such facilities in the context of the LHC/CMS Data Grid

• successful investigations of grid deployment, cluster and network monitoring 100 5 CMS at LHC and L3 at LEP2

Figure 5.24: Showing the screen of a Pocket PC device running the prototype GAE handheld analysis client based on JAS. The user is viewing a stacked histogram of data, with error bars.

• serving an active physics analysis user community largely based in California, but also having members in other US regions and Europe: a total of around 60 physicists have accounts. Their home institutes are:

– Caltech HEP – Caltech CACR – Caltech CS – CERN – University of Florida – UC Davis – Romania – UC San Diego – FNAL – University of Wisconsin – ANL – UC Riverside – UCLA

In the following sections we detail the component parts of the Caltech Tier2. 5.4 CMS Software, Computing and Grid Systems 101

Tier2 Cluster

This is the first prototypical cluster system that contains nodes based on Intel Pentium III and Pentium IV processors. This cluster is also known as the Caltech IGT site. It is built around a Dell PowerEdge 4400 rack-mount server as the head-node, which sports dual Intel Xeon PIII 1 GHz processors and 2 GB of main memory. It has seven internal SCSI hard drives serving system and user home directory files. It has three Winchester FlashDisk RAID devices summing to three terabytes of storage capacity directly attached to it. These are being used for storing a physics data. The Dell server has both private and public network connections. Two Dell Powerconnect 5224 gigabit switches have been used as a private cluster network fabric. For the public network connection a fiber gigabit link is provided to the CACR switch via a Syskonnect card. The compute nodes consist of twenty 2U Pentium III systems and twelve 1U Intel Xeon 2.4 GHz systems. The cluster is protected by a 3kVA uninterruptable power supply unit. Although the cluster is originally based on Red Hat 6.2 (required for Objectivity support), the newer 1U nodes are operating with Red Hat 7.3, and are devoted to the IGT productions. The VDT toolkit is being used as the main Grid middleware.

AMD testbed Cluster

This Athlon AMD based cluster has been built with NPACI ROCKS 2.3 cluster building software. It consists of four Athlon nodes with dual 1.6 GHz AMD processors on Tyan S2466N Tiger MPX motherboards. The main objective of this cluster is to deploy and tests various grid related software tools before they are actually rolled over into the IGT cluster. The reason for using Athlon machines is to see how efficiently AMD processors perform in comparison to Intel processors. We have found that the AMD processors execute code faster for a given system clock speed than do the Intel processors. However, there are concerns about memory bandwidth and the operating temperature of the CPU. Asante gigabit switches have been used to create the private cluster network. The machines are in fact physically located in two different places on the Caltech campus, and interconnected with fiber links. The AMD testbed cluster ran the first successful WAN distributed Monte Carlo Production (MOP) for CMS.

Production Grid (PG)

Currently we are building a cluster termed the ”Production Grid” (PG) that will initially consist of ten 1U high, powerful dual Intel Xeon 2.8 GHz Pentium IV processors installed in Supermicro X5DPE-G2 motherboards. The head node will be a 4U disk server with 2 Terabytes of storage capacity. This cluster will be conﬁgured using NPACI ROCKS software, with some FNAL patches. Dell Powerconnect 5224 gigabit switches will be used for the private network. The main purpose of this cluster is provide a high availability (24x7) distributed Grid production service.

Network Servers

There are some machines set aside for network and disk I/O related tests and measurements. Three systems, two 4U dual Intel Xeon 2.2 GHz using Supermicro P4DP6 and one 2U dual Intel Xeon 2.2 GHz using Supermicro P4DP8-G2 motherboards have been used for FAST TCP related developments and tests. Recently, we acquired a 4U diskserver built using a Supermicro x5DPE-G2 motherboard 102 5 CMS at LHC and L3 at LEP2 with dual Intel Xeon 3.06 GHz processors and 2 GB of main memory. It consists of 16 Maxtor 200 GB hard drives and three 3Ware Raid controllers that provide storage capacity in excess of 3 Terabytes. This system is currently under test to see if it can achieve very high disk I/O rates. To this end we are trying various hardware and software conﬁgurations. Using the Bonnie++ benchmark, we have been able to achieve 225 MB/Sec writing and 350 MB/Sec reading speeds to and from the disk subsystem. Eventually this server will be connected to a high performance wide area network so as to carry out WAN disk to disk I/O tests. In summary, the Caltech Tier2 center is growing steadily in size and power in order to support our Grid computing and networking research objectives, our development goals, and a small but active and growing community of physicists doing analysis.

5.4.12 CMS and Caltech Distributed Production

Caltech Data analysis

Three different tasks were studied at Caltech in the last year. Firstly, the study of Higgs to gamma gamma decay channel was continued, with the simulation, reconstruction and analysis of 1.4M events. Two internal CMS notes, one official CMS Note, and one presentation at ACAT2002 were completed. Part of these results are being used in the DAQ TDR. Additionally, 0.85M events were analysed and then partially simulated and reconstructed, specifically for the DAQ TDR. This work was done at Caltech and at CERN. Secondly, a study of eta calibrations was started in mid 2002. This is ongoing, and 0.2M events have been simulated, reconstructed and analysed. Thirdly, a total of 2.45M events were simulated, reconstructed and analysed using the Caltech Tier2, CERN and Alliance (NCSA) resources. Specifically, we used the Caltech Tier2 and TeraGrid Itanium II TeraGrid prototypes, the NCSA Platinum IA32 cluster, and the CERN PIII cluster. We will shortly begin use of the Condor flock at Wisconsin, and Itanium II prototypes at NCSA and SDSC. In the longer term we hope to use the LosLobos cluster at the University of New Mexico if required. As part of these studies, the next round of calculations were started in February 2003. The main goal is to produce 10M of data for the Physics TDR due in 2005. We have already begun porting our codes to the TeraGrid Itanium II keeping in mind that the facility will be fully operational in Summer 2003. The FORTRAN part of CMS software has already been ported and the first 0.1M of events produced on the TeraGrid cluster at Caltech. Currently we are working with the SDSC and NCSA prototypes in the hope of using them soon for these applications, which have been selected as a TeraGrid Flagship effort. For storage we are using the Caltech HPSS and CERN CASTOR systems. We have slightly less than 9TB of data in HPSS at Caltech. This will increase to at least 40TB within the next two years. For the analysis itself, two different versions of the code were created. The Pixel reconstruction is being used as is the regional track finder. Unfortunately, the global track finder is not yet working correctly for high luminosity modes[109].

CMS Oﬃcial Monte-Carlo Production at Caltech

The Caltech Tier2 center has played a very important role in Spring 2002 DAQ TDR Production. At the same time Caltech Tier2 has made a signiﬁcant contribution by participating in USCMS Grid Pro- duction activities. In both cases we ran oﬃcial CMS production assignments and transferred produced event databases to CERN and Fermilab. Here is a brief overview of these production tasks. 5.4 CMS Software, Computing and Grid Systems 103

Caltech’s use of the Tier2 for the Spring 2002 CMS Monte Carlo production runs laid an excellent foundation for future production related activities. During these productions we made full use of our hardware resources and were able to evaluate emerging technologies such as Journalling Filesystem on RAID5 arrays, and gigabit networking. We were able to successfully complete the six different assignments given to us. The software used in the productions included IMPALA 2 0 8e, CMSIM123, CMSIM125-2, ORCA 6 0 2, BOSSv2 1 and PBS 2.3.8. A cluster monitoring tool developed at Caltech was used to check the health of the cluster nodes. We produced 213,000 simulated events, 149,000 Hit formatted events, and 146,000 2 × 1033PUevents. Another important achievement was the successful implementation of distributed Monte Carlo Pro- duction (MOP) using the USCMS Grid Testbed. MOP, which was primarily developed by Fermilab and supported by University of Wisconsin, is the key application used to facilitate the use of Grid technology for CMS simulation. The participating institutions in the USCMS Grid Testbed are Caltech, Fermilab, UCSD, the University of Florida and the University of Wisconsin. The MOP production was initially deployed and tested on the Development Grid Testbed (DGT). After achieving satisfactory results, MOP was rolled over to the Integrated Grid Testbed (IGT). In all instances, Fermilab hosted the MOP server and other sites acted as the MOP workers. At Caltech, we used four dual 1.6 GHz Athlon machines for the DGT, and the existing Tier2 cluster for the IGT. The Grid middleware used included Globus 2.x, Condor 6.4.x and GDMP (which come as a bundle in the Virtual Data Toolkit (VDT) packaged by University of Wisconsin). VDT is a product from the GriPhyn and iVDGL projects. The Testbed partners decided to use DOE Science Grid (DOESG) signed host and user certificates. As required, all the sites installed the CMSIM and ORCA packages and the required upgrades. Condor was used as the main batch queueing system and communicated with the MOP server through the Condor Jobmanager. GridFTP, as an integral part of Globus, was used to transport the final data products to Fermilab. This required very close coordination between MOP master and worker site managers. After tremendous debugging effort, we successfully achieved a stable MOP production! Plans for building a production quality Grid, termed the ”Production Grid” (PG) are well advanced. Such a Grid will allow us to run large CPU-intensive distributed CMS simulations with 24x7 dedication. Caltech will be using the PG nodes already described in the previous Section. The USCMS IGT has successfully produced 1,000,000 Egamma ”BigJets” events from the generation stage all the way through to analysis Ntuples. The production time is mostly dominated by the GEANT simulation [110].

5.4.13 Grid Projects

The Caltech group is active in several DoE and NSF projects that have been funded to carry out research and development in the area of Grids and distributed systems for physics. The group’s principal involvements in this area include the DOE-funded PPDG project, along with the NSF-funded GriPhyN, iVDGL and CAIGEE projects. Each of these projects has a diﬀerent emphasis: GriPhyN is concerned with long-term R&D on Grid-based solutions for, collectively, Astronomy, Particle Physics and Gravity Wave Detectors, PPDG is investigating short term infrastructure solutions for running particle physics experiments and for those in active development (such as CMS and ATLAS). Finally, CAIGEE is investigating, prototyping and developing a Grid-enabled analysis environment for CMS. The projects are described in more detail in the following sections. 104 5 CMS at LHC and L3 at LEP2

Particle Physics Data Grid (PPDG)

The Particle Physics Data Grid (PPDG67) project began in 1999, under the DOE/NGI program. Its goals are to provide Grid-based integrated systems for Particle Physics experiments, both those running, and those (like the LHC experiments) ramping up and due to run in the coming years. Running experiments, such as STAR, BaBar and D0, are being helped to ”Grid-extend” their software by incorporating new Grid functionality, while maintaining the stability of the systems for current data processing, storage management, and analysis activities. For the upcoming experiments, PPDG is being instrumental in helping to incorporate Grid services quickly, with a particular emphasis on the production and analysis of simulated data. The specific research involves designing, developing, and deploying a network and middleware infrastructure capable of supporting data analysis and data flow patterns common to the several particle physics experiments represented among the collaborators. Application-specific software is being adapted to operate in this wide-area environment and to exploit the infrastructure. The result of these collaborative efforts has been the delivery of an operating infrastructure for distributed data access, processing and analysis by participating physics experiments. Experiment and Computer Science teams, working in the PPDG framework, are agreeing on, and implementing, Grid software services for the experiments. One particular focus is the provision of robust file replication and information systems, which involves file transfer (GridFTP, bbftp), replica catalogs (Globus LDAP, SRB), intelligent replica management, and production deployment support. As a follow-on to PPDG, the SciDAC project (Particle Physics Data Grid Collaboratory Pilot), has the goals to develop, acquire and deliver vitally needed Grid-enabled tools for data-intensive requirements of particle and nuclear physics. Novel mechanisms and policies are being vertically integrated with Grid middleware and experiment-specific applications and computing resources to form effective end-to-end capabilities. SciDAC is a three-year program whose goals and plans are ultimately guided by the immediate, medium-term and longer-term needs and perspectives of the physics experiments.

Grid Physics Network (GriPhyN) and the International Virtual Data Grid Laboratory (iVDGL)

The NSF funded GriPhyN68 collaboration involves physicist and computer scientist researchers who are planning to implement Petabyte-scale computational environments for data intensive science over the next few years. Motivating the project are the unprecedented requirements for geographically dispersed extraction of complex scientiﬁc information from the very large collections of measured data expected from current and future experiments in Particle Physics, Astronomy and Gravitational Wave Physics. To meet these requirements the GriPhyN collaboration is deploying computational environments called Petascale Virtual Data Grids (PVDGs) that meet the data-intensive computing needs of the World-wide distributed physics communities involved. GriPhyN is carrying out the necessary computer science research and development on the PVDG concepts, through a series of staged software releases and testbed deployments. The GriPhyN research results are strengthening the LHC experiment’s computing models, and its Virtual Data Toolkit (VDT) is being well used in production tasks. The NSF funded iVDGL69 project is a sister project to GriPhyN, targeted towards provision and deployment of network and computing infrastructure for scientiﬁc Grids. The project was proposed, and funding agreed in 2001, at a level of more than 13M$. The two major LHC experiments, CMS and

67See http://www.ppdg.net 68See www.griphyn.org 69See http://www.ivdgl.org 5.4 CMS Software, Computing and Grid Systems 105

ATLAS, are participating, along with LIGO, SDSS and the NVO. The focus in iVDGL is on deployment of the infrastructure to support Grid-based scientific computing. There is naturally a special emphasis on networking, which is reflected in the strong ties that have been established to the TeraGrid project, Internet2, ESNET from the USA, Geant from Europe, DataTAG for TransAtlantic traffic, AMPATH for Latin America, and others. The GriPhyN VDT is being deployed at the iVDGL participating institutions, of which Caltech is one. Under the auspices of iVDGL, the Grid Operations Center has been set up at Indiana, and is working with the TeraGrid.

Trillium - Coordination of PPDG, GriPhyn and iVDGL

Due to the considerable overlap between the project leaderships and participants in the PPDG, GriPhyN and iVDGL projects, it was decided to form a oversight management group, called Trillium, where many of the common issues, requirements and deliverables could be more easily coordinated. This brought together not only the three project management groups, but also the relevant funding bodies within DoE and NSF. The Trillium team is thus ideally placed to ensure that the goals of the projects are being met, to initiate joint projects (e.g. on Grid systems monitoring), and promote common packaging, the use of the VDT, and other Grid software developed in the projects.

CMS Analysis - an Interactive Grid Enabled Environment (CAIGEE)

Until now, the Grid architecture being developed by the LHC experiments has focused on sets of files and on the relatively well-ordered large-scale production environment. Considerable effort is already being devoted to preparation of Grid middleware and services (this work being done largely in the context of the PPDG, GriPhyN, EU DataGrid and LHC Computing Grid projects). The problem of how processed object collections, processing and data handling resources, and ultimately physics results may be obtained efficiently by global physics collaborations has yet to be tackled head on. Developing Grid-based tools to aid in solving this problem within the next two to three years is essential if the LHC experiments are to be ready for the start of LHC running. The current view of CMSs computing and software model is well developed, and is based on use of the Grid to leverage and exploit a set of computing resources that are distributed around the globe at the collaborating institutes. CMS has developed analysis environment prototypes based on modern software tools, chosen from both inside and outside High Energy Physics. These are aimed at providing an excellent capability to perform all the standard data analysis tasks, but assume full access to the data, very significant local computing resources, and a full local installation of the CMS software. With these prototypes, a large number of physicists are already engaged in detailed physics simulations of the detector, and are attempting to analyze large quantities of simulated data. The advent of Grid computing, the size of the US-based collaboration, and the expected scarcity of resources lead to a pressing need for software systems that manage resources, reduce duplication of effort, and aid physicists who need data, computing resources, and software installations, but who cannot have all they require locally installed. The CAIGEE project (funded in 2002 under NSF’s ITR program) directly addresses the challenge described above. The project, led by Caltech (PI:Newman), involves collaborators at UC San Diego, UC Davis and UC Riverside. In it, we are developing an interactive Grid-enabled analysis environment (GAE) for physicists working on the CMS experiment. The environment is intended to be lightweight yet highly functional, and make use of existing and future CMS analysis tools as plug-in components. The environment is based on a plug-in Web Services architecture, with Grid authentication, global monitoring, and sophisticated middleware support that provisions object collection generation, management, streaming, 106 5 CMS at LHC and L3 at LEP2 caching and persistent storage, in support of analysis tasks. The GAE work is described more fully elsewhere in this report chapter.

A Global Grid-enabled Collaboratory for Scientiﬁc Research (GECSR)

GECSR is a proposal submitted to the ”medium” NSF/ITR program in 2003. Proponents on this proposal include CMS (PI:Newman), ATLAS, and colleagues from the seismic, Grid, and science of collaboratories communities. The target is to develop, prototype, test and deploy the first Grid-enabled collaboratory for scientific research on a global scale. The distinguishing feature of the proposal is the tight integration between the science of collaboratories, a globally scalable working environment built on the foundation of a powerful, fully functional set of working collaborative tools, and an agent-based monitoring and decision-support system that is intended to allow collaborating scientists to perform data analysis tasks efficiently. The proposal describes the need for a means of enabling persistent collaboration (from the desktop, in small and large conference rooms, in halls, and in virtual control rooms), a flexible and extensible structure of hierarchical (role-based) and ad hoc peer groups, and a suite of components that make the collaborations work efficiently and effectively. These include features such as a ”language of access” and control (for people, meetings, casual and formal conversations), which can be scheduled or occur ”on demand”. In addition, the proposal calls for constant evaluation, evolution and optimization of the collaboration system, by the means of proven, iterative evaluation methods. Also included is the concept of agent-based decision support, where the scheduling of data or compute intensive transactions is decided upon jointly by humans and the system itself. The overall purpose of the proposed GECSR work is thus to support communities of scientists who are working locally, but within a global context. An example of this is a small team of physicists working on a particular analysis channel, but who are physically separated from each other, and from the compute resources they require to accomplish their analysis.

Dynamic Global Virtual Environments for Science (DoVES/DAWN)

DAWN is a proposal submitted to the “large” NSF/ITR program in 2003, which focuses on the development of “Dynamic Analysis Workspaces” in support of the CMS and ATLAS scientiﬁc missions. The proposal places a strong emphasis on the “provenance” of the data and results, and on the importance of maintaining meta-data that precisely describes the location of event data, and the precise steps, software versions and environment used to produce that data. Another strong emphasis, following on from the GECSR proposal, is on the provision of Collaborative Environments and Information Systems for the experiments.

5.4.14 CMS Data Storage and Access Transition

During the past year, we developed and demonstrated proof-of-concept prototypes which implement grid-based architectures aimed at efficient data access, processing and analysis by a large community of physicists in CMS. We therefore focused on tools to analyze, insert and extract large amounts of simulation data from a set of heterogeneous relational database management systems (RDBMS). Relational databases are advantageous in this context in that they allow a user to search rapidly through large amounts of data using relational indexes. For example, a user may want to perform a cut using certain combinations of tag parameters to get a list of relevant events. Rather than read through millions of binary files to find the relevant events, tag (summary) data may be inserted into a database 5.4 CMS Software, Computing and Grid Systems 107 and efficiently searched using relational database techniques, delivering the list of desired events, or sub-collections of objects in events. Working with sub-collections of objects, accessed rapidly using RDBMS’s, also is important to reduce the I/O load on the storage facilities and networks in a large Data Grid. By dealing only with the data they need within an event, physicists will be able to more easily extract the sub-collection, and possibly transport it across wide area networks to local or regional facilities, for further processing and analysis. In order to link our development to the current activities of the PRS groups, and to lay the groundwork for our GAE developments (discussed in previous sections) we initially focused our tools on an 70 existing large dataset, used by the CMS “Jets and Missing ET ”(JetMET)group . For the JetMet analysis, we developed conversion utilities to aggregate large amounts of JetMet nTuple data (170 GB) into relational databases including SqlServer and Oracle. These tools were implemented in both Java and C++, so that we could support a particular site’s server platform preference. These tools also utilized relational database abstraction layers, so that almost any relational database could be employed according to a site’s database preference and/or local expertise. We also developed tools to extract selected portions of these large JetMET datasets for delivery in binary formats amenable to analysis, such as PAW FZ files and ROOT Trees. These tools were implemented in cross-platform C++ (Win32 and Linux) and were integrated into the Clarens rpc layer as plug-ins, so that users could remotely control data extraction. We believe providing these tools pre-integrated into Clarens as plug-ins will simplify site installation. We also developed web services which allowed users access to the relational tag data in self-describing XML formats. This form allows users that do not wish to use ROOT or PAW to do analysis. Any language for which an XML parser exists, which is practically all of them, may be utilized. We are in the process of implementing a general converter, usable on the client-side or server-side through Clarens, so that physicists may convert to/from any of (PAW FZ, ROOT Tree, SQL databases) for arbitrary binary input files. Our design goal is to have intermediate relational data abstraction layers which allow database brand pluggability (without recompile). This will also include dynamic relational database schema creation based on the input binary file. We are also developing additional tools to enable the user to specify which portions of the tag data should be included in the binary extract file, after the cut has been performed. This is important not only for efficient access to object collections, but will also reduce unnecessary I/O and data flow over the networks when large numbers of physicists are carrying out their analysis using the CMS Data Grid.

5.4.15 The Dynamic Distributed Services Architecture

We are developing a globally scalable “Dynamic Distributed Services Architecture” (DDSA)71 to serve large physics collaborations. As described below, this architecture incorporates many features that make it suitable for managing and optimizing workflow through Data Grids composed of hundreds of sites, with thousands of computing and storage elements, and thousands of pending tasks, such as those foreseen by the LHC experiments. In order to scale and operate robustly in managing global, resource-constrained Grid systems, the DDSA framework uses a set of Station Servers, one per facility or site in a Grid, that host a variety of dynamic, agent-based services. The services are registered with, and can be mutually discovered by a lookup service, and they are notified automatically in case of “events” signaling a change of state anywhere in a large distributed system. This allows the ensemble of services to cooperate in real time to gather, disseminate, and process time-dependent state and configuration information about the site facilities, networks, and many jobs running throughout the Grid. The monitored information is reported

70This dataset and the new structures we developed are described in detail in last year’s Annual Report. 71See “Grid Computing: Making the Global Infrastructure a Reality”, Chapter 39. 108 5 CMS at LHC and L3 at LEP2 to higher level services, that in turn analyze the information, and take corrective action to improve the overall efficiency of operation of the Grid (through load balancing, for example) or to mitigate problems as needed. The DDSA framework is inherently distributed, “loosely coupled” and self-restarting, making it scalable and robust. Cooperating services and applications are able to access each other seamlessly, to adapt rapidly to a dynamic environment (such as worldwide-distributed analysis by hundreds of physicists in a major HEP experiment). The services are managed by an efficient multithreading engine that schedules and oversees their execution, such that Grid operations are not disrupted if one or more tasks (threads) are unable to continue. The system design also provides reliable “non-stop” support for large distributed applications under realistic working conditions, through service replication, and automatic re-activation of services. These mechanisms make the system robust against the failure or inaccessibility of multiple Grid components (when a key network link goes down, for example). A service in the DDSA framework is a component that interacts autonomously with other services through dynamic proxies or agents that use self-describing protocols. By using dedicated lookup services, a distributed services registry, and the discovery and notification mechanisms, the services are able to access each other seamlessly. The use of dynamic remote event subscription allows a service to register to be notified of a selected set of event types, even if there is no provider to do the notification at registration time. The lookup discovery service will then automatically notify all the subscribed services, when a new service, or a new service attribute, becomes available. The code mobility paradigm (mobile agents or dynamic proxies) used in the DDSA extends the remote procedure call and the client server approach. Both the code and the appropriate parameters are downloaded to services as needed, from a distributed set of repositories (called “Gigaspaces”72). Several advantages of this paradigm are: optimized asynchronous communication and disconnected operation, remote interaction and adaptability, dynamic parallel execution and autonomous mobility. The combination of the DDSA service features and code mobility makes it possible build an extensible hierarchy of services capable of managing very large Grids, with relatively little program code. Effective and robust integrated applications require higher level service components able to adapt to a wide range of requests, and changes in the state of the system (such as changes in the available resources, for example). These services are capable of “learning” from previous experience, and apply “self-organizing neural network” (SONN [114]) or other heuristic algorithms to the information gathered, to optimize the system, by minimizing a set of “cost functions”. One example of this, reported in last year’s Annual Report, was the optimization of load-balancing among a set of regional centers, in a study that used the simulation system we developed for the MONARC project. We have built a prototype implementation of the DDSA based on JINI [115] technology. We have also included WSDL/SOAP [116] bindings for all the distributed objects, in order to facilitate a possible future migration to the Open Grid Services Architecture (OGSA73). The JINI architecture federates groups of devices and software components into a single, dynamic distributed system; functionality that the future OGSA will need to include. JINI enables services to find each other on a network and allows these services to participate and cooperate within certain types of operations, while interacting autonomously with clients or other services. This architecture simplifies the construction, operation and administration of complex systems by: (1) allowing registered services to interact in a dynamic and robust (multithreaded) way; (2) allowing the system to adapt when devices or services are added or removed, with no user intervention; (3) providing mechanisms for services to register and describe themselves, so that services can intercommunicate and use other services without prior knowledge of the services’ detailed implementation.

72See http://www.j-spaces.com/ 73See http://www.globus.org/ogsa/ 5.4 CMS Software, Computing and Grid Systems 109

5.4.16 MonALISA: A Distributed Monitoring Service

An essential part of managing a global Data Grid is a monitoring system that is able to monitor and track the many site facilities, networks, and the many task in progress, in real time. The monitoring information gathered also is essential for developing the required higher level services, and components of the Grid system that provide decision support, and eventually some degree of automated decisions, to help maintain and optimize workflow through the Grid. We therefore developed the agent-based MonALISA (Monitoring Agents in A Large Integrated Services Architecture [111]) system, based on the DDSA framework. MonALISA is an ensemble of autonomous multi-threaded, self-describing agent-based subsystems which are registered as dynamic services and are able to collaborate and cooperate in performing a wide range of monitoring tasks in large scale distributed applications, and to be discovered and used by other services or clients that require such information. MonaLisa services are organized in groups and this attribute is used for registration and discovery. The system monitors and tracks site computing farms and network links, routers and switches using SNMP, and it dynamically loads modules that make it capable of interfacing existing monitoring applications and tools (e.g. Ganglia, MRTG, Hawkeye, LSF). MonALISA provides flexible access to real-time or historical monitoring values, by using a predicate subscription mechanism or dynamically loadable filter agents. These mechanisms are used by any interested client to query and subscribe to only the information it needs, or to generate specific aggregate values in an appropriate format. MonALISA also provides a secure mechanism (SSL with X.509 certificates) for dynamic configuration of farms/network elements, and support for other higher level services that aim to manage a distributed set of facilities and/or optimize workflow. New monitoring modules can be dynamically loaded into the system as needed. Graphical user interfaces allow users to visualize global parameters from multiple sites, as well as detailed tracking of parameters for any component in the entire system. The graphical clients also use the remote notification mechanism, and are able to dynamically show when new services are started, or when services become unavailable. Dedicated filers are used to provide global views with real time updates for all the running services. Dynamically loadable alarm agents, and agents able to take actions when abnormal behavior is detected, are currently being developed to help with managing and improving the working efficiency of the facilities, and the overall Grid system being monitored. systems. The core of the monitoring service is based on a multi-threaded system used to perform the many data collection tasks in parallel, independently. The modules used for collecting different sets of information, or interfacing with other monitoring tools, are dynamically loaded and executed in independent threads. In order to reduce the load on systems running MonALISA, a dynamic pool of threads is created once, and the threads are then reused when a task assigned to a thread is completed. This allows one to run concurrently and independently a large number of monitoring modules, and to dynamically adapt to the load and the response time of the components in the system. If a monitoring task fails or hangs due to I/O errors, the other tasks are not delayed or disrupted, since they are executing in other, independent threads. A dedicated control thread is used to stop properly the threads in case of I/O errors, and to reschedule those tasks that have not been successfully completed. A priority queue is used for the tasks that need to be performed periodically. This approach makes it relatively easy to monitor a large number of heterogeneous nodes with different response times, and at the same time to handle monitored units which are down or not responding, without affecting the other measurements. As an example, we monitored 500 compute nodes performing a request for 200˜ metric values per node every 60 seconds. This provided a sustained rate of 1600˜ metric values per second collected, using an average of 20 active threads. The number of threads necessary to monitor a complete site is dynamically adjusted, and very dependent on the 110 5 CMS at LHC and L3 at LEP2 response time for each node, which is related to its load as well as to the quality of the network connections. The collected values are stored in a relational database, locally for each service. The JDBC framework offers the flexibility to dynamically load any driver and connect to virtually any relational database. A normalized scheme is used to store any result object provided by the monitoring modules in indexed tables, which are themselves generated as needed, dynamically. The monitored client services at a given site being monitored by MonALISA use the Lookup Dis- covery Services to find all the active MonALISA services running as part of one or several group “communities”. The discovery mechanism is used for notification when new services are started or when services are no longer available. The client application connects directly with each service for receiving monitoring information. To perform this operation it first downloads the proxies for the services it needs, from a URL specified as an attribute of each service. This procedure allows each service to correctly interact with other services running different software versions. Clients may subscribe to data using a predicate mechanism and may deploy filters to obtain processed information from the flow of measured values. A generic framework for building “pseudo-clients” for the MonALISA services was developed [117]. This has been used for creating a dedicated Web service repository with selected information from the groups of MonALISA services. The Web service uses a set of JINI Lookup Discovery Services to discover all the active MonALISA services from a specified set of groups and subscribes to these services with a list of predicates and filters. These predicates or filters specify the information the pseudo-client wants to collect from all the services. It stores all the values received from the running services in a local MySQL database, and uses procedures written as Java threads to compress old data (keeping the mean and the extreme values measured). The servlet engine of the Web service uses the local database to provide charts for current and customized historical values, on demand. It is also used to generate Wireless Access Protocol (WAP) pages containing the same information for mobile phone users. Multiple Web Repositories can easily be created to globally describe the dynamic services running in a distributed environment. MonALISA is now deployed and operating round the clock monitoring the US CMS Test Grid and an increasing number of other sites. The MonALISA Web repository is now accumulating historical data for the US CMS Tier1 and Tier2 centers at Fermilab, Caltech, UCSD, and the University of Florida, as well as the production farms at CERN, at the Academia Sinica in Taiwan (ATLAS), and at the Polytechnic University in Bucharest. We also monitor the network traffic on the US-CERN production link, and the distribution of the traffic into the major networks and links with which we peer: EsNet, Abilene, Mren, StarTAP, the US-CERN DataTAG link, the CERN-Geant link, Taiwan-Chicago, and Bucharest-Budapest. MonALISA was demonstrated at the iGRID2002 and SC2002 conferences last Fall. Following a presentation and demonstration at CHEP2003 last March, MonALISA was installed at SLAC and is used to provide a monitoring service for the IEPM-BW [120] project. As discussed in the LHCNet chapter of this report, it is also being used to monitor and optimize the interconnections among 20 of the VRVS reflectors. The VRVS system is thus used as a testbed for the development of generally useful agents that are capable of performing routing decisions, and of trying to restart services which are not working correctly. Through the Internet2 End-to-End Performance Initiative74 MonALISA is also going to be used to monitor and help manage the Internet2 Abilene backbone, and eventually some of the regional and local networks in Internet2 [122]. We are working to enhance the end-to-end measurements provided by MonALISA to meet the needs of Internet2, as well as the proposed UltraLight next-generation optical

74See http://www.internet2.edu/e2epi 5.4 CMS Software, Computing and Grid Systems 111 network [121]. (The UltraLight project is discussed further in the LHCNet chapter). In Figure 5.25 we show several views of a graphical client discovering and presenting real-time values (global and details) from the MonALISA services, as it monitors a set of regional centers in a prototypical Grid, including many of the sites mentioned above. Deploying these monitoring services on many sites and interfacing it with other monitoring tools (SNMP, Ganglia, MRTG, IEPM-BW and “Alien”(ALICE)-scripts) as well as with batch queuing systems (LSF, PBS, Condor) has provided very useful experience, and has enabled us to begin building reliable and scalable distributed services. This experience also has been important in enabling us to start building higher level services, to perform job scheduling and data replication tasks effectively; services that adapt themselves dynamically to respond to changing load patterns in large Grids. The new simulation framework we are developing [119], based on the experience from the MONARC project [118], is designed to dynamically use the information from the MonALISA monitoring service to simulate different Grid architectures, and the Computing Model that governs their use. These studies aim to evaluate the way such Grid systems will scale, and to develop strategies that will allow the LHC experiments to use these Grids effectively. 112 5 CMS at LHC and L3 at LEP2

Figure 5.25: MonALISA Monitoring Service: Examples of real time global and detailed views of the results of monitoring a set of regional centers and networks in a prototypical Grid, with sites in the US, Europe and Asia. BIBLIOGRAPHY 113

Bibliography

[1] The LEP Collaborations ALEPH, DELPHI, L3 and OPAL, CERN preprint CERN-EP/2000-153, hep- ex/0101027. [2] A. Straessner, “QCD and W Mass Determination at LEP”, talk presented at the 38th Rencontres de Moriond (QCD), March 2003. [3] L3 Collab., P. Achard et al., Phys. Lett. B 527 (2002) 29-38. [4] The LEP Collaborations ALEPH, DELPHI, L3 and OPAL, L3 Note 2766; A. G. Holzner, “Search for the Standard Model Higgs Boson at LEP”, hep-ex/0208045. [5] The LEP Electroweak Working Group, public page, http://lepewwg.web.cern.ch/LEPEWWG/ (updated March 2003). [6] L3 Collab., P. Achard et al., Phys. Lett. B 545, 30 (2002), hep-ex/0208042. [7] S. Ask, “Review of the SUSY Searches at LEP”, talk presented at the 38th Rencontres de Moriond (EW), March 2003. [8] “Search for Heavy Neutral and Charged Leptons in e+e− Annihilation at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B 517 (2001) 67-74. [9] “Search for Heavy Isosinglet Neutrino in e+e− Annihilation at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B 517 (2001) 75-85. [10] C. Coy, “Beyond the SM Model, Experimental Results from LEP”, talk presented at the 38th Rencontres de Moriond (QCD), March 2003. [11] S. Jadach, B. F. Ward and Z. Was, Comput. Phys. Commun. 130, 260 (2000). [12] L3 Collab., M. Acciari et al., Phys. Lett. B 431 (1998) 199. [13] K. Hagiwara et al., The Review of Particle Physics,Phys.Rev.D66 010001 (2002). [14] CDF Collab., F. Abe et al., Phys. Rev. Lett. 81 (1998) 1791. [15] N. Arkani-Hamed, S. Dimopoulos and G. Dvali, Phys. Lett. B 429 (1998) 263. [16] M. Gataullin, “Searches for Extra Dimensions at LEP”, hep-ex/0108008. [17] H.P. Nilles, Phys. Rep. 110 (1984) 1; J. Gunion and H. Haber, Phys. Rep. 117 (1985) 75. [18] G. ’t Hooft, in Recent Developments in Gauge Theories, ed. by G. ’t Hooft et al. (Plenum Press, 1980); L. Maiani, Proc. Summer School of Gif-sur-Yvette (1980); M. Veltman, Acta Phys. Polon. bf B 12 (1981) 437. [19] M. Dine and A. E. Nelson, Phys. Rev. D48(1993) 1277; M. Dine, A. E. Nelson and Y. Shirman, ibid. D51(1995) 1362; M. Dine et al., ibid. D53(1996) 2658. [20] “Interpretation of Neutralino and Scalar Lepton Searches in Minimal GMSB Model”, M. Gataullin, S. Rosier-Lees, L. Xia and H. Yang, L3 Note 2777 (2002). [21] H. Ma, H. Newman, R.Y. Zhu, and R. Hamm, Nucl.Instr.Meth. A281 (1989) 467; H. Ma, et al., Nucl.Instr.Meth. A274 (1989) 113. [22] R. Y. Zhu et. al., Nuclear Physics B (Proc. Suppl.) 44 (1995) 109. [23] U. Chaturvedi et. al., IEEE Trans. Nucl. Sci. 47 (2000) 2101-2105. [24] The CMS Electromagnetic Calorimeter Project, CERN/LHCC 97-33 (1997). [25] R.Y. Zhu and H. Yamamoto, GEM TN-92-126 and CALT 68-1802 (1992); and S. Mrenna et al., GEM TN-93-373 and CALT 68-1856 (1993). 114 5 CMS at LHC and L3 at LEP2

[26] N. Ellis and J. Virdee, Ann. Rev. of Nucl. and Part. Sci. 44 (1994) 609; C. Seez, Proc. of the 2nd Int. Conf. on Calorimetry in High Energy Physics, ed. A. Ereditato, World Scientiﬁc (1992) 359; CMS TN/93-115 (1993) and CMS TN/94-288, (1995); E. Clayton, C. Seez and T.S. Virdee, CMS TN/92-50, (1993). [27] R.Y. Zhu, Nucl. Instr. and Meth. A413 (1998) 297; R.Y. Zhu et al. IEEE Trans. Nucl. Sci. NS-45 686 (1998) and Nucl. Instr. and Meth. A376 (1996) 319.

[28] R.Y. Zhu et al., PbWO4 Monitoring and Calibration, a talk given in CMS ECAL Meeting (May 1996). [29] L.Y. Zhang et al., Monitoring Light Source for CMS Lead Tungstate Crystal Calorimeter at LHC, IEEE Trans. Nucl. Sci. NS-48 (2001) 372. [30] L.Y. Zhang et al., Installation of the ECAL Monitoring Light Source, CMS Internal Note 2001/008. [31] R,Y. Zhu et al. IEEE Trans. Nucl. Sci. NS-43 (1996) 1585 and Proc. of the 6th International Conference on Calorimetry in High Energy Physics, ed. A. Antonelli et al., Frascati Physics Series 577 (1996). [32] R.Y. Zhu, IEEE Trans. Nucl. Sci. NS-44 468 (1997) and Proc. of the 7th International Conference on Calorimetry in High Energy Physics,ed.E.Cheuet al., World Scientific 469 (1998). [33] X.D. Qu et al., Nucl. Instr. and Meth. A480 (2002) 468 and Nucl. Instr. and Meth. A469 (2001) 193, and Q. Deng et al., Nucl. Instr. and Meth. A438 (1999) 415. [34] Q. Deng et al., Proc. of the 10th International Conference on Calorimetry in Particle Physics,ed.R.- Y. Zhu, World Scientific 190 (2002). [35] R.Y. Zhu et al., Proc. of the 9th International Conference on Calorimetry in High Energy Physics,ed. B. Aubert et al., Frascati Physics Series 709 (2000) and R.Y. Zhu, Proc. of the 8th International Conference on Calorimetry in High Energy Physics, ed. G. Barreira et al., World Scientific 226 (1999).

[36] T. Hu and R.Y. Zhu, Absolute Calibration of PbWO4 Crystal ECAL by using Physics Processes,CMS Internal Note 2001/034, published in Proc. of the 10th International Conference on Calorimetry in Particle Physics, ed. R.-Y. Zhu, World Scientiﬁc 459 (2002). [37] The LHC Study Group, The LHC Conceptual Design Report, CERN/AC/95-05 46 (Oct 1995). [38] X.D. Qu et al., Radiation Induced Color Centers and Light Monitoring for Lead Tungstate Crystals, IEEE Trans. Nucl. Sci. NS-47 (2000) 1741. [39] L.Y. Zhang, K.J. Zhu and R.Y. Zhu, Monitoring Light Source for CMS Lead Tungstate Crystal Calorimeter at LHC, IEEE Trans. Nucl. Sci. NS-48 (2001) 372. [40] CMS TN/92-56, C. Seez and J. Virdee, ”Detection of an intermediate mass Higgs boson at LHC via its two photon decay mode”. [41] CERN/LHCC 97-33, CMS Collaboration, ”The Electromagnetic Calorimeter Project - Technical De- sign Report”. [42] CMS Note 1997/101, M.N. Dubinin et. al., ”Light Higgs Boson Signal at LHC in the Reactions pp → γγ + jet and pp → γγ + lepton”. [43] CMS Note 2001/022, M. Dubinin, ”Higgs Boson Signal in the Reactions pp → γγ +2forwardjet”. [44] V. Tisserand, ”The Higgs to two photon decay in the ATLAS Detector”, ATLAS Internal Note, PHYS- NO-90 (1996). [45] See Section 5.4.12 of this report. [46] D. Rainwater and D. Zeppenfeld, JHEP 12 (1997) 5. [47] D. Rainwater, D. Zeppenfeld and K. Hagiwara, Phys. Rev. D59(1999) 014037. [48] CMS Note 2002/030, V. Litvin, H. Newman, S. Shevchenko, N. Wisniewski, ”The rejection of background to the H → γγ process using isolation criteria based on information from electromagnetic calorimeter and tracker”. BIBLIOGRAPHY 115

[49] CMS IN 2002/034, V. Litvin, H. Newman, S. Shevchenko, N. Wisniewski, ”Isolation studies for photons from Higgs decay based on the Tracker”. [50] CMS IN 2002/023, V. Litvin, H. Newman, S. Shevchenko, ”Isolation studies for electrons and photons based on ECAL”. [51] CMS Note 1993/092, C. Seez and T.S. Virdee, ”Using Tracks to Locate the two Photon Vertex”. [52] CMS Note 1993/115,C.Seez,”An Algorithm Using Tracks to Locate the two Photon Vertex”. [53] CMS Note 1995/115,D.J.Graham,”An algorithm using tracks to locate the two photon vertex at high luminosity”. [54] CMS IN 2001/010, S. Shevchenko, ”Number of hard tracks in the H→ γγ and minimum bias events for diﬀerent models of multiple interaction”. [55] CMS IN 2003/023, T. Lee, V. Litvin, H. Newman, S. Shevchenko, ”Search for intermediate mass Higgs boson, decaying into two photons”. [56] L. Randall and R. Sundrum, Phys. Rev. Lett. 83, 3370 (1999). [57] W.D. Goldberger and M.B. Wise, Phys. Rev. Lett. 83, 4922 (1999). [58] CERN/LHCC 2002-26, CMS Collaboration, ”The Trigger and Data Acquisition Project, Volume 2, Data Acquisition and High-Level Trigger - Technical Design Report”. [59] http://cmsdoc.cern.ch/Physics/egamma/transparencies/m90-3.pdf, C. Shevchenko, Using η → γγ process for ECAL calibration. [60] The XDAQ framework for distributed data acquisition: http://xdaq.web.cern.ch [61] http://www.cacr.caltech.edu/projects/tier2support [62] http://baldrick.cacr.caltech.edu/TagServer/TagServices.asmx [63] http://www/apache.org [64] http://clarens.sourceforge.net [65] http://www.w3.org/2002/ws [66] http://www.xmlrpc.org [67] http://sourceforge.net/projects/spheon-jsoap/ [68] http://hep-proj-database.web.cern.ch/hep-proj-database/workshop-July2001/workshop agenda.htm [69] http://lhcgrid.web.cern.ch/LHCgrid/SC2/RTAG1/default.htm [70] Saima Iqbal. Evaluation Of Oracle9i To Manage CMS event store: Oracle Architecture To Store Petabyte Of Data (PART ONE). CMS Internal Note 2002/002 http://cmsdoc.cern.ch/documents/02/in02 002.pdf [71] http://kholtman.home.cern.ch/kholtman/sc2001/sc2001.html [72] http://www.ppdg.net/docs/presentations/sc2001 demos.htm [73] http://www.objy.com/ [74] Koen Holtman, on behalf of the CMS collaboration. CMS Data Grid System Overview and Requirements. CMS Note 2001/037. http://kholtman.home.cern.ch/kholtman/cmsreqsweb/cmsreqs.html [75] Koen Holtman. Views of CMS Event Data: Objects, Files, Collections, Virtual Data Products. CMS NOTE-2001/047. http://kholtman.home.cern.ch/kholtman/note01 047.ps [76] K. Holtman. HEPGRID2001: A Model of a Virtual Data Grid Application. Proc. of HPCN Europe 2001, Amsterdam, p. 711-720, Springer LNCS 2110. http://kholtman.home.cern.ch/kholtman/hepgrid2001/ 116 5 CMS at LHC and L3 at LEP2

[77] Kurt Stockinger. Multi-Dimensional Bitmap Indices for Optimising Data Access within Object Oriented Databases at CERN. Ph.D. Thesis, University of Vienna, Austria, November 2001. http://kurts.home.cern.ch/kurts/RESEARCH/diss.ps [78] The Monarc Simulation Tool http://www.cern.ch/MONARC/sim tool/ [79] I.C. Legrand, Multi-threaded, discrete event simulation of distributed computing systems Computer Physics Communications 140 (2001) 274 http://clegrand.home.cern.ch/clegrand/MONARC/CHEP2k/sim chep.pdf [80] I.C. Legrand, H.B. Newman, Simulating Large Network-Distributed Processing Systems Proc. of the 2000 Winter Simulation Conference Editors: J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick http://clegrand.home.cern.ch/clegrand/MONARC/WSC/wsc monrc.pdf [81] Y. Wu, I. C. Legrand, I. Gaines and V. O’Dell Simulating the Farm Production System Using the MONARC Simulation Tool CHEP 2001, Beijing, Sept 2001 http://clegrand.home.cern.ch/clegrand/CHEP01/chep01 fnal sim.pdf [82] Web site with the simulation results for the FNAL production farm http://www.cern.ch/MONARC/sim tool/Publish/FNAL/ [83] H.B. Newman, I.C. Legrand Simulating Distributed Systems, ACAT 2000, Chicago http://clegrand.home.cern.ch/clegrand/MONARC/ACAT/IAST505 paper.pdf [84] Web site with the results for the study in Using Tapes http://www.cern.ch/MONARC/sim tool/Publish/TAPE/publish/ [85] Web site for simulations in data replications between regional centers http://www.cern.ch/MONARC/sim tool/Publish/CMS/ [86] H.B. Newman, I.C. Legrand Simulation Studies in Data Replication Strategies CHEP 2001, Beijing, Sept 2001 http://clegrand.home.cern.ch/clegrand/CHEP01/chep01 10-048.pdf [87] H.B. Newman, I.C. Legrand A Self-Organizing Neural Network for Job Scheduling in Distributed Systems CMS NOTE 2001/009, January 8, 2001 http://clegrand.home.cern.ch/clegrand/SONN/note01 009.pdf [88] H.B. Newman, I.C. Legrand Self-Organizing Neural Network for Job Scheduling in Distributed Systems ACAT 2000, Chicago http://clegrand.home.cern.ch/clegrand/MONARC/ACAT/AI406 paper.pdf [89] Web site with the simulation results in the evaluation of SONN http://www.cern.ch/MONARC/sim tool/Publish/SONN/ [90] I.C. Legrand, A Distributed Server Architecture for Dynamic Services http://clegrand.home.cern.ch/clegrand/lia/DistributedServices D2.pd [91] I.C. Legrand, Presentation at the SUN-High Performance Computing http://clegrand.home.cern.ch/clegrand/lia/SUN-HPC June-01 il.pdf BIBLIOGRAPHY 117

[92] H.B. Newman, I.C. Legrand, J.J. Bunn A Distributed Agent-based Architecture for Dynamic Services CHEP 2001, Beijing, Sept 2001 http://clegrand.home.cern.ch/clegrand/CHEP01/chep01 10-010.pdf [93] I.C. Legrand, The Architecture of the Monitoring Tool http://clegrand.home.cern.ch/clegrand/CMS Monitor/MonitorTool.doc http://clegrand.home.cern.ch/clegrand/CMS Monitor/cms mar 02 il.ppt [94] http://www.sdss.jhu.edu/˜ szalay/kdi/ [95] http://www.sdss.jhu.edu/ScienceArchive/doc.html [96] Terabyte Analysis Machines for Observational Astronomy. http://sdss.fnal.gov:8000/ an- nis/astroCompute.html [97] K. Holtman, H. Stockinger. Building a Large Location Table to Find Replicas of Physics Objects. Pro- ceedings of CHEP 2000, Padova, Italy. http://kholtman.home.cern.ch/kholtman/olt long.ps [98] CMS Virtual Data Requirements. Work in progress. Version 9. Available from http://kholtman.home.cern.ch/kholtman/ [99] http://www.cacr.caltech.edu/ppdg/meetings/ppdg collab/holtman/objrepl.pdf [100] http://kholtman.home.cern.ch/kholtman/globusretreat objrepl.ppt [101] http://www.globus.org/datagrid/deliverables/gsiftp-tools.html [102] K. Holtman. Object level physics data replication in the Grid ACAT’2000, October 16-20, 2000, Fermi National Accelerator Laboratory http://conferences.fnal.gov/acat2000/program/presentations/vlsc302.ppt [103] http://cmsdoc.cern.ch/orca/ [104] Kurt Stockinger, Dirk Duellmann, Wolfgang Hoschek, Erich Schikuta. Improving the Performance of High Energy Physics Analysis through Bitmap Indices. 11th International Conference on Database and Expert Systems Applications, London - Greenwich, UK, Sept. 2000. Springer-Verlag. http://kurts.home.cern.ch/kurts/RESEARCH/Bmi dexa2000.ps [105] http://wwwinfo.cern.ch/db/objectivity/docs/hepodbms/ [106] http://cmsdoc.cern.ch/cms/grid/ [107] http://research.microsoft.com/ Gray/ [108] Kurt Stockinger, Design and Implementation of Bitmap Indices for Scientiﬁc Data, to appear in Interna- tional Database Engineering & Applications Symposium, Grenoble, France, July 2001, IEEE Computer Society Press. [109] CMS Notes, Internal Notes and publications: http://www.cacr.caltech.edu/∼litvin/in02 023.ps http://www.cacr.caltech.edu/∼litvin/in02 034.ps http://www.cacr.caltech.edu/∼litvin/note02 030.ps http://www.cacr.caltech.edu/∼litvin/nim.v2.0.3pages.ps http://www.cacr.caltech.edu/∼litvin/article.pdf http://cmsdoc.cern.ch/cms/TDR/DAQ/daq.html - CERN/LHCC/2002-26, CMS TDR 6.2, 15 December 2002 Presentations: http://www.cacr.caltech.edu/∼litvin/19apr2002.ppt http://www.cacr.caltech.edu/∼litvin/15apr2002- agenda.ppt http://www.cacr.caltech.edu/∼litvin/15apr2002.ppt http://www.cacr.caltech.edu/∼litvin/14jun2002- agenda.ppt http://www.cacr.caltech.edu/∼litvin/14jun2002.ppt http://www.cacr.caltech.edu/∼litvin/acat2002.ps http://www.cacr.caltech.edu/∼litvin/acat2002.ppt http://www.cacr.caltech.edu/∼litvin/27sep2002.ppt http://www.cacr.caltech.edu/∼litvin/28oct2002.ppt http://www.cacr.caltech.edu/∼litvin/09dec2002.ppt 118 5 CMS at LHC and L3 at LEP2

http://www.cacr.caltech.edu/∼litvin/27jan2003.ppt http://www.cacr.caltech.edu/∼litvin/10feb2003.ppt http://www.cacr.caltech.edu/∼litvin/10mar2003.ppt http://www.cacr.caltech.edu/∼litvin/talk260203.ps http://www.cacr.caltech.edu/∼litvin/sergey.eta.ps [110] http://www.cacr.caltech.edu/projects/tier2-support/ http://cmsdoc.cern.ch/cms/production/www/html/general/ http://computing.fnal.gov/cms/Monitor/cms production.html http://www.lsc- group.phys.uwm.edu/vdt/ [111] MonaLisa web page http://monalisa.cacr.caltech.edu/ [112] H.B. Newman, I.C. Legrand, J.J. Bunn A Distributed Agent-based Architecture for Dynamic Services CHEP 2001, Beijing, Sept 2001 http://clegrand.home.cern.ch/clegrand/CHEP01/chep01 10-010.pdf [113] Julian Bunn and Harvey Newman Data Intensive Grids for High Energy Physics Grid Computing: Making the Global Infrastructure a Reality edited by Fran Berman, Geoﬀrey Fox and Tony Hey, March 2003 by Wiley [114] H.B. Newman, I.C. Legrand A Self-Organizing Neural Network for Job Scheduling in Distributed Systems CMS NOTE 2001/009, January 8, 2001 http://clegrand.home.cern.ch/clegrand/SONN/note01 009.pdf [115] Jini web page http://www.jini.org [116] World Wide Web Consortium http://www.w3.org [117] MonaLisa WEB Repository for US-CMS http://monalisa.cacr.caltech.edu:8080/ [118] MONARC Project http://www.cern.ch/MONARC/sim tool/ [119] MonaLisa WEB Repository for US-CMS http://monalisa.cacr.caltech.edu/MONARC/ [120] Internet End-to-end Performance Monitoring http://www-iepm.slac.stanford.edu/ [121] The Ultralight Project http://www.phys.uﬂ.edu/ avery/ein2003/Ultralight proposal all.doc [122] The iInternet2 Web Page http://www.internet2.org/ [123] The Net-Snmp Web Page http://www.net-snmp.org/ [124] The Ganglia Project Web Page http://ganglia.sourceforge.net/ [125] The MRTG Project Web Page http://www.mrtg.org/ [126] The Openwings Project Web Page http://www.openwings.org/ 6. BaBar

J. Albert, E. Chen, A. Dvoretskii, R. Erwin, D. Hitlin, G. Dubois-Felsmann, I. Narsky, T. Piatenko, F. Porter, A. Ryd, A. Samuel

Overview

The Caltech HEP group, due to the interests of Hitlin and Porter, has a long history of involvement in the effort to study CP violation in the B system, culminating in the BABAR experiment at the PEP-II asymmetric collider. The BABAR experiment is now completely constructed [1], commissioned, has taken a significant dataset in e+e− collisions near bottom threshold, and is in the midst of a very productive program for physics results [2]–[86], including the observation of CP violation in B decays[2]–[4]. The main focus of this experiment is on understanding CP violation and searching for new phenomena in rare B decays. A portion of the BABAR group has completed a participation in the BES experiment. This group has also embarked on an investigation into a future higher luminosity B factory. Personnel involved in the BABAR effort currently include, in addition to Hitlin and Porter: graduate students E. Chen, A. Dvoretskii, R. Erwin, T. Piatenko, and A. Samuel; Member of the Professional Staff G. Dubois-Felsmann; Senior Postdoctoral Scholar A. Ryd; Postdoctoral Scholar I. Narsky; Robert A. Millikan Postdoctoral Fellow J. Albert; as well as undergraduate students at various times. Senior Postdoctoral Scholar S. Yang left in February 2003, for a consulting position in industry. Senior Post- doctoral Scholar M. Weaver left in 2001, taking a staff position at SLAC. Senior Postdoctoral Scholars Y. Kolomensky and S. Metzler left during 2000. Kolomensky has taken an Assistant Professor position at UC Berkeley, and Metzler a postdoctoral position in medical imaging. Member of the Professional Staff R. Zhu participated in the BABAR construction. G. Eigen took a faculty position in Norway several years ago, but maintains a visiting position at Caltech and spends a small amount of time on campus. Dubois-Felsmann and Ryd came to Caltech in 1996, Narsky joined the BABAR effort at Caltech in Au- gust 2001, finally replacing Metzler. Justin Albert arrived at Caltech in July 2002 as a new Millikan Prize Postdoctoral Scholar, effectively replacing Kolomensky. We currently have an offer out to replace Weaver. Alexander Samuel returned in October 2001 as a fourth-year (now fifth) graduate student, following a period on leave of absence to work in the software industry. Chen is a sixth-year graduate student, and Dvoretskii is a fifth-year graduate student. Matthew Dorsten briefly joined the group as a second-year graduate student, but has made a transition to theory. We were joined by two new (entering) graduate students, Rebecca (Joan) Erwin and Timofei Piatenko in the summer of 2002. Ryd and Kolomensky were initially partially supported by Caltech funding, and Albert currently is. The BABAR budget formerly contained support for R&D and construction as well as for salaries, travel and miscellaneous supplies. The R&D (M&S) and construction effort was supported by funding coming through SLAC, as awarded by the SLAC management in consultation with BABAR in response to R&D proposals and to WBS construction items. As the R&D and construction are complete, the BABAR group is now supported on operating funds. In the early stages of the B factory project, we were aided as well by other funds from Caltech (which supported J. Tennyson while he was doing his beam-beam studies here [87], the study of beam pipe cooling [88], the accelerator workshop we held [89], and a physics & detector workshop). We were also awarded funds over a two-year period by IBM to study computing issues. In September 1997, Caltech hosted the concluding physics workshop for BABAR [90], and we received partial support from the Institute for this. In support of a new R&D effort on liquid xenon, we have recently received funding from Caltech. As mentioned above, we have terminated involvement, besides papers in progress, in the BES exper-

119 120 6 BaBar

iment, studying e+e− collisions in the region near tau and charm threshold, at the BEPC storage ring at IHEP in Beijing. We were heavily involved in several areas of physics in this experiment, including the tau mass measurement [91]-[94], D and Ds physics [95]-[102], and ψ physics [103]-[115]. Other involvements of Caltech in BES included the construction of a luminosity monitor [116], work on the core software, work on the particle identiﬁcation, and development and maintenance of the UNIX side of the BES software management system, CODEMAN. We also had a small involvement in the detector R&D for a TeV-scale e+e− linear collider [117], [118]. Porter served an early stint as co-coordinator of the North American Calorimetry Working group, with Ray Frey of the University of Oregon and Andre Turcot of Brookhaven National Laboratory. Finally, Hitlin is leading a new eﬀort to study the physics potential and technical challenges of a future higher luminosity B factory [119], with participation by a majority of the group.

BaBar Experiment

The b quark was discovered to have a long lifetime in 1983 [120]. The long lifetime suggests that the B meson has great potential as a laboratory to look for rare processes, including possible violations of the standard model. The motivation for B physics was further boosted in 1987 with the discovery of large B0B¯0 mixing [121]. This opened the way for particularly clear studies of the mechanism of CP violation, which had only been observed in the neutral kaon system. It was this possibility which led to the suggestion of an asymmetric e+e− B factory [122]. Hitlin and Porter of the Caltech group were heavily involved in the effort to establish an asymmetric e+e− B factory, starting in earnest with the Snowmass Summer Study in 1988 [123]. Following the Summer Study, we devoted considerable effort to studying the feasibility of using “conventional” storage ring technology for such a collider. Having convinced ourselves that the required luminosity could be achieved, and was to be favored over possible “new technology” approaches, we convened an informal workshop at Caltech in April of 1989 to get together various experts in the community to study the accelerator issues in more depth [89]. Together with interested people at LBL and SLAC, this work continued, culminating in the first feasibility study for a B factory sited at the PEP storage ring at SLAC [124]. This study formed the evolutionary basis for the current design for PEP-II [125], which was approved in October 1993, has completed construction and commissioning, and has been producing collisions for BABAR since May 1999. In parallel with the accelerator work, we considered the physics potentials, and the detector issues. Contemporary with the accelerator feasibility study was a survey of the physics program of such a machine [126]. At that time, Alan Weinstein wrote the parametric Monte Carlo simulation program (ASLUND) that was used in many of our initial CP violation sensitivity studies. Subsequently, studies of various detector options, including several R&D efforts, were carried out [127]-[156]. We note that Caltech continues to maintain a substantial presence in this experiment: Hitlin completed his terms as spokesperson, served in the publication board, which is the final authority on what gets published, currently serves as a U.S. representative on the Executive Board, and is leading the study of a higher luminosity (1036 cm−2s−1) B factory. Porter was elected to the Vice-Chairmanship of the BABAR Collaboration Council (the governing organization), serves as system manager for the computing system, and as chair of the BABAR statistics working group. Weaver performed R&D on a promising source calibration technology, demonstrated its feasibility, led the construction activity, and led its operation until leaving Caltech. He also served as head of the online calibrations for BABAR. Dubois-Felsmann leads a major Caltech responsibility in BABAR, the “Online Event Processing” compute farm which occupies a substantial fraction of the group, including Narsky and all of the graduate 6.1 Computing 121 students at various times. He is also the chair of the BABAR CKM Working Group. Chen has completed service in charge of the fast monitoring of the detector data quality. Ryd has built a new physics event generator package, has served as the Monte Carlo Generators Physics Tool Group Convenor, and currently is co-convenor of the Radiative Penguin Analysis Working Group. He held the major responsibility of Reconstruction Coordinator, and continues to be consulted. Ryd and Yang took on substantial responsibilities in the development of the Level 3 trigger, and more recently the responsibility to contribute large Monte Carlo samples using the Caltech computing resources. Samuel has now taken over the responsibility for the Caltech Monte Carlo production. Prior to departing, Kolomensky developed a framework and algorithms for particle identification and was responsible for the particle identification information from the silicon vertex tracker. Kolomensky also served as the physics-computing contact person. In response to requests from SLAC, we have recently returned to helping with accelerator issues, embarking on a design for beam spot measurements, and on beam-beam studies. We summarize below the current activities of the Caltech group in the operation of BABAR and in the physics analysis activities. We also discuss briefly the higher luminosity planning activities.

6.1 Computing

There are a number of computing challenges for the BABAR experiment. The data rates are high for an e+e− collider, and there is a large quantity of useful data which must be stored and analyzed. Indeed, the BABAR database already contains over 3/4 PB, making it possibly the largest in the world! Convenient interactive access must be available for the efficient extraction of physics results. In addition, the wide geographical distribution of the collaboration implies significant issues for remote access to the data and code, collaboration communication, and for remote computing. The BABAR Computing System includes several aspects: (1) the online data acquisition, control, and monitoring hardware and software; (2) the offline computing hardware—CPUs (“farm” and desktop), storage, X servers, and networking—whether local to SLAC or at remote locations; (3) the software environment in which the computing work is done; and (4) the code itself used by the collaboration. The overall design of the system is described in the Technical Design Report [157], and is not repeated here. The Caltech group involvement with the B factory computing has been partly R&D, partly organization, partly implementation, and partly operation. On the organizational side, Porter is system manager for the BABAR computing system. Gregory Dubois-Felsmann is the leader of the online event processing system, a major Caltech responsibility, and serves as the principal liason between the collaboration’s online and offline computing groups. Anders Ryd had until recently the major responsibility of reconstruction manager. Caltech is among the remote institutions which contribute significant capacity for Monte Carlo production and DST analysis. Below we describe the areas of especial emphasis of the Caltech group in BABAR computing.

6.1.1 Online Computing

The Caltech BABAR group maintains a major role in the development of the experiment’s online computing systems and software, centering around the “online farm”, a collection of commercial UNIX workstations that provides event building, software triggering, data quality monitoring, and calibration services to BABAR during data-taking. Speciﬁcally, the online farm: 122 6 BaBar

1. Hosts the final stage of event building — assembling the contributions to an event from the O(28) data acquisition VME crates with their O(160) embedded processors at a Level 1 trigger event rate of up to 5000 triggers per second (Tps) with 100 MB/s data throughput; 2. Applies a software (“Level 3”) trigger algorithm to these events, reducing the rate of events to be logged to permanent storage to 100–150 Tps, at the present PEP-II luminosity; 3. Provides a rapid-feedback data quality monitoring facility, dubbed “Fast Monitoring”, accumulating statistics and histograms on the data, with a basic level of monitoring of data acquisition and trigger quantities for all Level 1 events and more detailed detector-system-specific monitoring for a subset of the data flow; 4. Acts as a server for event display clients; 5. Ensures that the data passing Level 3 are logged to permanent storage at the SLAC computing center; 6. Performs the final stages of the online calibrations, including validation and logging of the resulting calibration constants to the “conditions database”.

Much of the work done in the online farm is included in the so-called “Online Event Processing” (OEP) component of the BABAR online system software. This component provides the software infrastructure within the online farm for performing the tasks enumerated above, including the means to control and coordinate the operations of the farm machines and interact with the online Run Control subsystem. The tasks themselves are performed by “modules” provided by the core OEP group and by, among others, the Level 3 trigger group and the detector subsystems. The entire system is coded in the C++ language, using object-oriented design techniques throughout. OEP must:

• Provide an execution environment — the “OEP Framework” — which coordinates and sequences the operation of modules and the flow of event data into and out of them, across the set of online farm processors devoted to OEP for a partition. • Provide a set of common services needed by modules, such as configuration and control, histogramming, access to databases, and the like. This requires attention to the needs of the subsystems and other users of the experiment, with an ongoing effort to identify common needs as they arise and implement them in common when feasible. Several of these service components have entailed major software development projects in their own right within the Caltech group, some of which are still in progress and are covered in additional detail elsewhere in this report:

– a distributed histogramming system and automated histogram display and monitoring tools, and the associated integration of Java and CORBA technologies into the online system; – interfaces from both UNIX and VxWorks to the BABAR object-oriented database for online conﬁguration and calibration information; and – a client-server protocol for event data monitoring and display.

• Be integrated with the online system as a whole, respecting common programmatic and user interface conventions as appropriate. This means, for example, operating under the control of the Run Control subsystem of the online, and using a common error reporting and logging system 6.1 Computing 123

with the rest of the online. Moreover, OEP must also be able to run in a variety of “standalone” modes for debugging and for the support of detector subsystems’ development and commissioning eﬀorts, at collaborator’s home institutions and at SLAC, away from the resources of the online farm and the infrastructure of the main BABAR installation at SLAC.

In addition to providing the framework and common tools themselves, the OEP group must assist in coordinating the development of modules from all contributing groups and integrating them into a system that meets the needs of BABAR as a whole. While the event builder software itself is provided by the DataFlow group, based at SLAC, as a set of libraries, the application that actually invokes this code and manages the buffering, distribution, and lifetime of events, is provided as part of OEP. Undertaken beginning in late 1996, the OEP effort has been one of the two principal institutional responsibilities of Caltech within the BABAR collaboration, led by Gregory Dubois-Felsmann, currently supervising Postdoctoral Scholars Justin Albert and Ilya Narsky and graduate student Edward Chen. In previous years, Yury Kolomensky, Scott Metzler, and Songhoon Yang, and graduate students Brian Naranjo and Alex Samuel participated in this effort, with occasional assistance from additional summer and work-study students. Narsky has taken over Metzler’s important role in the support and development of the histogramming system. The Level 3 trigger is implemented as a set of modules running in the OEP environment. Devising the algorithms and providing the module are responsibilities of the Trigger group, within which the Level 3 effort is led by Larry Gladney of the University of Pennsylvania (Penn). Anders Ryd and Songhoon Yang were major contributors to this effort in previous years, together with members of the Penn, University of California, Santa Cruz, and Lawrence Berkeley National Laboratory groups. Alexei Dvoretskii was heavily involved in upgrades to the drift chamber tracking component of Level 3. Dubois-Felsmann continues to serve as the contact point between the OEP and Level 3 groups, and so is involved in architectural design of Level 3 and its upgrades. Otherwise Caltech has now phased out its contributions to Level 3. The data acquisition and analysis required to support online calibrations are split across the DataFlow and OEP subsystems of the online, taking place, respectively, in the embedded processors and the online farm. Dubois-Felsmann, Kolomensky, and Weaver have in the past made major contributions in this area, and Dubois-Felsmann continues to provide architectural leadership for ongoing upgrades to the calibration system.

Performance achieved

Over the year covered by this report, the OEP system has gone into routine production as BABAR has − begun physics data acquisition, and the BABAR experiment has now acquired more than 121 fb 1 of colliding beam data using this system, at remarkably high efficiency. Under normal circumstances data acquisition efficiency, expressed as the fraction of available PEP-II luminosity that is recorded, is better than 98%. The entire combination of DataFlow event builder, core OEP infrastructure, and Level 3 trigger has been demonstrated to be capable of acquiring data at rates beyond the original 2000 Tps design point of the system, and logging data, in concert with the Logging Manager software provided by the Prompt Reconstruction group, at rates far in excess of the 100 Tps design point. Presently the Level 1 trigger rate limit is close to 5000 Tps and the instantaneous logging rate limit approaches 1000 Tps. The average Level 1 trigger rate in normal operation remains below 2000 Tps, and the logging rate around 150 Tps, so there is a great deal of headroom in the system and it is ready to be used at future higher luminosities. These improvements in performance have been achieved by upgrading the online farm from its initial 124 6 BaBar configuration of thirty-two Sun Ultra-5 computers, first to sixty of these, and then, at the start of Run 3 in the fall of 2002, to forty-seven Dell 1650 dual-CPU (1.4 GHz Pentium-III) Intel/Linux systems. The move to Intel/Linux required some work from the Caltech OEP group in 2001 and 2002 on supporting the difference in intra-word byte ordering between the data acquisition embedded processors and the Intel Pentium architecture. At this time the OEP/Level 3 system appears capable of handling data flow up to the maximum rate supported by the other components of the data acquisition system; i.e., it is not presently a bottleneck.

Work remaining

The main goals of remaining work on the core OEP components, in addition to maintenance, are:

• Improvement of automation and interface with Run Control. We continue to make progress in this area. The improvements in the control of calibrations mentioned in the previous report have now been applied to all the detector systems, and further fine-tuning takes place on a regular basis. The most important step, however, has been the extension, as planned, of these tools to cover of complete sets of many OEP nodes — the group of 47 used for Level 3 triggering, and, now for the first time, a group of 14 used for data quality monitoring, replacing the single-node system previously in use. The multi-node control system was designed by Dubois-Felsmann and was implemented and re- fined by a 2001 hire to the SLAC BABAR online computing group. It was deployed in the summer of 2002 for Level 3.

• Improvement of data quality monitoring (“Fast Monitoring”) throughput and user interface. The above control system was extended in early 2003 to Fast Monitoring by Albert and Dubois- Felsmann. The present group of 14 nodes provides more than a factor of ten improvement in throughput from the single (slightly faster) node used for the last few years. In the near future we will move Fast Monitoring to a larger group of Intel/Linux computers, providing a further improvement in performance. At the same time, a graphical user interface for the control of Fast Monitoring was introduced and has greatly simpliﬁed the task of the shift crew who operate it.

• Support of symmetric multiprocessor hardware in the online farm. The new Intel/Linux farm is composed of dual-CPU computers. In order to extract the maximum performance from these units, it is necessary to be able to distribute the load equally between the two processors. In the present online farm, the majority of the computation performed on each farm node is in the Level 3 trigger code. This runs as a single computational “thread” and so cannot beneﬁt from the availability of the second CPU. In order to make full use of both CPUs, we must modify the OEP software to allow two (or more) simultaneous Level 3 processes to consume the event data. This possibility was foreseen in the original design. The additional performance from this upgrade was not required in 2002 and the work was deferred. It has now become more important and the work is resuming in May 2003. It will be available for Run 4 and should be ready for a prototype test in June.

• Support for “trickle injection”. An ongoing project is the development of software to support a new mode of operation of PEP-II, using continuous, or “trickle”, injection of at-energy particles into the beams. This mode avoids the need to stop collisions and data acquisition periodically to “reﬁll” the storage rings. Continuous injection has the possible drawbacks of causing radiation 6.1 Computing 125

damage to the detector and problems in data acquisition, either due to large overlaid backgrounds in events or increases in event rates. These may cause data acquisition deadtime and problems in reconstructing events for Level 3 triggering or for physics analysis. The latter problems can in principle be minimized by taking advantage of the fact that even during “continuous” injection, the actual particle pulses are injected into the storage ring at well-known times, and to cause their most severe effects on backgrounds for only very brief intervals. OEP software work in support of continuous injection has been in two parts: first, to provide diagnostic tools to study the effects of running in this mode, and allow event processing and trigger decisions to take into account the time since the last injection pulse; and second, to improve the ability of the system to handle the additional non-uniform data acquisition load produced by the periodic injection pulses. This requires a new facility for deep buffering of events just before the Level 3 processing. It has been observed that the excess load from the injection pulses can exceed the instantaneous throughput limits of the system, but does not come close to exceeding its maximum average throughput. The addition of buffering at this stage should permit the system to run through the periodic spikes in load by averaging them out over the full interval between injection pulses. The first part was completed in early 2002 in time for an initial test of trickle injection in PEP-II. Further tests were deferred until other pending machine development projects were completed, and are expected to resume now in June 2003. Thus the second half of the OEP project was not required in the past year and we are only now resuming development. A prototype will be used in the June 2003 tests, and a final version will be in place for Run 4 in the fall, when it is expected that continuous injection will become routine.

Distributed Histogramming and Fast Monitoring

New development on the Distributed Histogramming tools described in previous years’ reports, sus- pended for a time after Metzler’s departure, were able to be resumed following Narsky’s joining the Caltech group in 2001. The Distributed Histogramming Package (DHP) for OEP has been reimplemented for improved speed and reliability. This work involved mostly redesign of low-level implementation routines without noticeable change of existing high-level interfaces. The core idea was to make shared memory segments containing information about histograms easily searchable and movable. To accomplish this, absolute pointers were replaced with relative offsets and linear containers were replaced with balanced binary trees. In parallel, high-level interfaces were extended to give DHP users more flexibility. The new DHP has been deployed in the online system in January 2003. Its performance has been reliable, with a significant increase in speed over the old implementation. The reduction of CPU cycles allowed use of DHP for the new multi-node Fast Monitoring. Chen has recently transferred his daily operational responsibilities in the Fast Monitoring system to a graduate student from SUNY Albany, but remains involved in the support of the software infrastructure for this work.

BABAR online data monitoring

With BABAR’s steadily-increasing rate of data taking, reliable real-time monitoring of the vast quantities of information recorded is critical in order to spot any problems that occur. We have improved BABAR’s “online fast monitoring” system to allow it to handle over 10 times the data it could handle previously. As described in an earlier section, online event processing (OEP) is performed using a farm of workstations that combine the information from the diﬀerent detector subsystems and run the level 126 6 BaBar

3 trigger algorithm. Previously, the monitoring of raw data information (online fast monitoring) was performed using a single CPU that received a random sample of data from just one of the nodes in the OEP farm. We have modiﬁed this system so that multiple CPUs are used to monitor the raw data in real-time. Each monitoring CPU receives a stream of data from an OEP farm node, and the information from each of the monitoring CPUs is combined so that BABAR shifters can easily determine if anything is amiss. This new system has now been in production for 3 months and is being used to monitor all incoming BABAR data.

Figure 6.1: The status panel for the multiple CPUs used in the new online fast monitoring system.

6.1.2 BABAR Reconstruction

Anders Ryd was in charge of the BABAR reconstruction software development from July 2001 until July 2002. The major projects during this period was the removal of the Rogue Wave tools.h++ foundation class library and a change to the way calibrations are performed in our prompt reconstruction. For the Rouge Wave migration the primary motivation was the cost of licensing this product. Rouge Wave changed their licensing and the cost to continue using this library became prohibitly high. At SLAC alone, ignoring remote sites, the cost would have been several $M per year. Most of the tools.h++ classes has straight forward replacements in the C++ standard library, though a few classes were more involved to migrate. However, the largest challenge in this migration is just in the very large number of places that we had to change code and the possibility of introducing new bugs. The second project was a change to the way calibrations are performed in the prompt reconstruction. The original design for the reconstruction and calibrations system was to do it in one pass. Runs are processed sequentially, when processing run N a calibration is performed and these constants are used for the reconstruction of run N + 1. There are two problems with this approach. First, the constants as determined in run N are only applied to run N + 1 and hence there is a delay in applying the calibrations. Second, the need to process the runs sequentially lead to severe problems in scaling to higher luminosity. In essence we 6.2 BaBar Monte Carlo Production 127 needed very large farms (300 CPU’s) to keep up with the instantaneous data rate, however these large farms had substantial overheads at the start end ends of runs due to conditions access etc. Hence we restructured the way the calibrations are done. It is now a two pass system. In the ﬁrst pass (prompt calibration) runs are processed sequentially. However, only a small fraction of the events are needed for the calibrations so we can use a much smaller farm of about 20 CPU’s to do the calibration. In the second pass (event reconstruction) all events are fully reconstructed using the constants determined from the same run. As no constants are determined in the second pass these runs do not need to be processed sequentially. The two pass system was put into production in the fall of 2002 and was used to reprocess all the Run 1 and Run 2 data as well as processing the data taken in Run 3. Overall the two pass system has performed well and has solved the scaling problems that we had during the earlier processings. The Rouge Wave migration was not completed before we started the production in the fall of 2002. However, in early 2003 this migration was completed and the BABAR software is now free of dependences on Rouge Wave.

6.2 BaBar Monte Carlo Production

Producing the Monte Carlo samples that are needed to support physics analysis in BABAR is a major undertaking. Caltech was the first BABAR institution to use Linux in production. Anders Ryd was involved in the porting of the BABAR code to Linux, in particular the simulation code. Caltech participates in BABAR’s Monte Carlo simulation production effort as a remote production site. The production process consists of generating random physics events, simulating the response of the BABAR detector to particles in these events, adding machine backgrounds to these events using measured machine background data, and finally performing the standard event reconstruction on the simulated events. In 2002, we participated in the “SP4” production series. Starting in 2003, we tested and converted to the “SP5” production series. In SP4, simulation is performed by three separate programs. The first performs event generation and detector simulation, and stores the simulated events in an Objectivity database. The second mixes background events with the simulated events, and again writes the results to the database. The third performs event reconstruction, and stores the reconstructed event in the same database. Failures occur frequently at each of these stages, due to software bugs, intermittent hardware problems, or server overloading. The sprite program accommodates this by detecting program failures and rerunning the appropriate stage as necessary. After a full allocation of production runs is complete, the reconstructed events are exported from the Caltech database, transferred to SLAC over the network, and imported into a database there. At Caltech, simulation production is performed on a farm of two-processor Pentium III rack systems running Linux. One farm node is dedicated to running the batch system that coordinates jobs running on the farm. For SP4, we used the DQS batch system software. The remaining thirty farm nodes execute simulation programs, two processes per node. The Objectivity database itself resides on a separate server equipped with a 500 MB RAID subsystem. Database access is via NFS. From the beginning of 2002 until January 2003, Caltech produced more than 15 million SP4 Monte Carlo events. During this time, we addressed a variety of problems which impeded production at maximum efficiency. In particular, NFS configuration problems and problems with the server hosting the Objectivity database led to significant downtime and job crashes. By monitoring the NFS servers and clients, we were able to gain an understanding of some of the problems, and alleviate them by spreading NFS load to additional servers. 128 6 BaBar

In early 2003, we ended SP4 production running and began testing SP5, the next incarnation of BABAR simulation production. Caltech was the ﬁrst remote (from SLAC) site to work with SP5. Among its improvements, SP5 uses Moose, the Monolithic Object-Oriented Simulation Executable. Moose combines the event generation, detector simulation, background mixing, and reconstruction programs into a single executable. This obviates the need for writing intermediate event data to the database. Not only does this speed up simulation by drastically reducing network I/O, it diminishes the NFS load on the database server and the Objectivity lock server. In late 2002, ten additional nodes were added to the production farm. However, high utilization on the database server prevented us from using these nodes in simulation production. In early 2003 we also deployed a new system for use as the Objectivity server. This server is faster, includes more storage (1 TB), and features a more reliable RAID implementation. Between the added capabilities of the server and the reduction in storage load because of Moose, NFS load on the server was low enough that we were able to begin using the additional nodes for production without increasing the failure rate. In fact, failure rates are much lower since we have begun SP5 production, and NFS load on the database server is low enough that we anticipate adding additional nodes to the farm. Design limitations in DQS prevented us from submitting large numbers of jobs to the batch queue at once. To work around this, the sprite program monitored the batch queue and submitted additional jobs as necessary. In late 2002, we replaced the DQS batch system with PBS, the Portable Batch System. PBS allows us to submit an entire allocation of several thousand runs at one time to the queue. We no longer need to run sprite, and the requirement for manual supervision of simulation production is greatly reduced. In the ﬁve months since the transition to SP5, Caltech has produced more than 50 million Monte Carlo events, about a sixth of the total BABAR SP5 production so far. As we continue to tune our setup and address problems, and add nodes to the farm, we expect our rate to increase still further. Initially, the Monte Carlo production was run by Anders Ryd and Songhoon Yang. John Estes, an undergraduate student, then continued running the production for a time. Currently, it is managed by Alex Samuel.

6.3 Physics

The Caltech group actively participated in the preparation for the physics studies to be performed by BABAR, and is now pursuing several physics interests. These include several topics besides the well- known CP-violation via mixing studies. In this section, we summarize some current eﬀorts in this area.

∗ − 6.3.1 Search for B → K( )+ at BABAR

The rare B decays B → K + − and B → K∗ + − proceed via the ﬂavor-changing neutral current transition b → s + −. In the Standard Model, this transition is forbidden at tree level, but can occur at higher order, in loop diagrams known as “penguin” diagrams. These decays are sensitive to properties of the top quark; also, many extensions to the Standard Model, such as SUSY models, predict additional contributions. Thus, by measuring these decays, we hope to test the Standard Model and constrain extensions to it. Samuel, Ryd, and Porter have been working on these measurements, in collaboration with Berryhill and Richman at UCSB. ∗ − The search for B → K( ) + at BABAR attempts to reconstruct eight exclusive decay modes: 6.3 Physics 129

± ± + − 0 + − 0 ∗0 + − ∗0 + − ± ∗± + − B → K , B → KS , B → K where K → K π ,andB → K where ∗± ± + − + − + − K → KSπ ; is either e e or µ µ . For this analysis the KS is reconstructed in the mode + − KS → π π . We reconstruct candidates from charged tracks, and reduce backgrounds by selecting candidates based on kinematic and particle identification quantities. We perform a two-dimensional maximum likelihood fit in mES, the beam energy-substituted mass of the reconstructed B, and ∆E, the difference between its energy and the beam energy. We fit the sum of signal and background distribution functions to determine the signal yield. We use signal-mode efficiencies estimated from Monte Carlo simulation to compute branching fractions from our signal yields. In selecting candidates, special attention is paid to removing background modes in which all tracks which form the candidate come from a single real B meson, such as B → J/ψ K(∗);andB → Dπ, D → Kπ where both pions are misidentified as muons. These background modes give rise to candidates whose values of mES and ∆E are similar to those of signal events, and therefore bias our fits. We performed a blind analysis by defining our candidate selection procedure using Monte Carlo simulations of signal and background modes, and validated it using three control samples:

• B → J/ψ K(∗) and B → ψ(2S)K(∗) events

• candidates with values of mES and ∆E outside the range used in our ﬁts • candidates formed with e±µ∓ lepton pairs

Only after the selection procedures were ﬁnalized did we examine our signal candidate events. − We have published a completed a search for B → K(∗) + − using 22.7fb 1 of data collected in 1999 and 2000. We observed no evidence for a signal, and determined 90% C.L. upper limits: B(B → K + −) < 0.51 × 10−6 B(B → K∗ + −) < 3.1 × 10−6

This analysis have been updated to use the Run 1 plus Run 2 data with an integrated luminosity of 77.8 fb−1 corresponding to a sample of (84.4 ± 0.9) × 106 Υ(4S) → BB¯ decays. Preliminary results of the new analysis has been presented at HEP conferences in the summer of 2002. In the new data set, we observe a signal in the combined B → K + − modes with a signiﬁcance of about 4.4σ, and in the B → K∗ + − modes with a signiﬁcance of 2.8σ.

Figure 6.2 shows the projections of our ﬁts in ∆E and mES for each mode. Table 6.1 summarizes the yield, eﬃciency, and measured branching fractions for each mode. Figure 6.3 shows the projections summed over lepton and isospin modes. We calculate the branching fractions, summed over lepton and isospin modes, B → + − +0.24+0.11 × −6 (B K )=(0.78−0.20−0.18) 10 B → ∗ + − +0.68 ± × −6 (B K )=(1.68−0.58 0.28) 10 .

As the signal is not signiﬁcant in the B → K∗ + − modes we quote an upper limit at 90% C.L.

B(B → K∗ + −) < 3.0 × 10−6

Since this preliminary result was made public we have made some further improvements to the analysis. E.g., we are using bremsstrahlung recovery in the electron channels to improve the eﬃciency 130 6 BaBar

andintheK∗ modes we are now also including the Kπ mass in the likelihood ﬁt. This analysis is almost complete and we intend to publish this based on the full Run 1 plus Run 2 data sample.

10 + + + + - +µ+µ- (a) K e+e- (b) K µ+µ- B A B AR (a) K e e (b) K B A B AR 5 5

100 0 0 + - 0 µ+µ- 2 (c) KS e e (d) KS 0 + - 0 µ+µ- (c) KS e e (d) KS

c 5 5

100 0 (e) K*0e+e- (f) K*0µ+µ- *0 + - *0µ+µ- Entries/3 MeV/ 5 (e) K e e (f) K 5 Entries/20 MeV

100 *+ *+ + - 0 (g) K e+e- (h) K µ µ *+ + - *+µ+µ- 5 (g) K e e (h) K 5

0 0 5.2 5.22 5.24 5.26 5.285.2 5.22 5.24 5.26 5.28 -0.2 -0.1 0 0.1 0.2 -0.2 -0.1 0 0.1 0.2 2 2 ∆ E ∆ E m ES (GeV/ c ) mES (GeV/ c ) (GeV) (GeV)

Figure 6.2: Projections in mES (left plots) and ∆E (right plots) of signal (red) and background (blue) distributions resulting from ﬁt to candidate events (histograms).

6.3.2 B → K1(1400)γ

We have begun preliminary investigations for a potential measurement of the decay B → K1(1400)γ in BABAR. This process is an example of a flavor-changing neutral current b → sγ transition, which proceeds in the one-loop level in the Standard Model via a radiative penguin diagram. ∗ BABAR and other experiments have measured the exclusive b → sγ process B → K γ, as well as exclusive radiative penguin processes B → K + − and B → K∗ + −. Analyses of decays involving higher kaon resonances are under way; higher kaon resonances are also included in inclusive B → Xsγ measurements. ∗ The K1(1400) (henceforth K1) resonance decays with a branching fraction of 94% to K π. While the resulting final state, Kππ, is staightforward to identify, it is difficult to isolate intermediate resonances such as this K1. Its large width, 174 MeV, prevents us from separating it by mass alone from the ∗ ∗ nearby resonances K1(1270), K (1410), and K2 (1430). Belle has measured the branching fraction of B+ → K+π+π−γ, but has not isolated intermediate resonances from each other or from nonresonant production, and no other measurement of B → K1γ has been performed to date. (CLEO and Belle have observed B → K∗(1430)γ.) Nevertheless, with adequate statistics, it should be possible to separate the resonances based on angular distributions or Dalitz anlysis. A measurement of this decay will add to our understanding of radiative penguin processes, and help to understand inclusive B → Xsγ measurements as well.

In addition, the three-body decay of the K1 allows an indirect measurement of the photon polariza- + → + 0 → 0 tion in B K1 γ and B K1 γ decays, as described in Gronau, Grossman, Pirjol, and Ryd (2002). The orientation of the angle between the Kππ decay plane in the K1 rest frame and the K1 ﬂight 6.3 Physics 131

(a) Kl +l - (b) Kl +l - 10 10 BABAR

2 5 5 c

0 0 (c) K *l+l- (d) K *l+l- Entries/20 MeV

Entries/3 MeV/ 10 10

0 0 5.2 5.22 5.24 5.26 5.28 -0.2 -0.1 0 0.1 0.2 2 ∆ mES (GeV/c ) E (GeV)

Figure 6.3: Projections in mES (left column) and ∆E (right column) of signal distributions resulting from ﬁt to candidate events (histograms). The top row shows the sum of all four B → K + −, and the bottom row shows the sum of all four B → K∗ + − modes. 132 6 BaBar

Signal (∆B/B) (∆B)ﬁt B Mode yield (%) (%) (10−6) (10−6) + → + + − +5.0 ± +0.14 +0.34+0.16 B K e e 14.4−4.2 17.5 6.8 −0.21 0.98−0.28−0.22 + → + + − +2.3 ± +0.09 +0.30+0.09 B K µ µ 0.5−1.3 9.2 6.6 −0.08 0.06−0.17−0.08 0 → 0 + − +2.6 ± ± +0.49+0.14 B K e e 1.3−1.7 18.6 7.9 0.14 0.24−0.32−0.15 0 → 0 + − +2.9 ± ± +1.07± B K µ µ 3.6−2.1 9.4 7.7 0.24 1.33−0.78 0.26 0 → ∗0 + − +5.2 ± +0.46 +0.87+0.48 B K e e 10.6−4.3 10.6 7.6 −0.47 1.78−0.72−0.49 0 → ∗0 + − +3.9 ± ± +1.14± B K µ µ 3.4−2.8 6.1 9.3 0.38 0.99−0.82 0.39 + → ∗+ + − +3.7 ± +0.69 +1.87+0.69 B K e e 0.3−2.3 10.3 9.5 −0.72 0.15−1.16−0.72 + → ∗+ + − +3.9 ± ± +3.91± B K µ µ 3.6−2.5 5.2 11.1 1.80 3.61−2.51 1.84

Table 6.1: Results from the fits to B → K(∗) + − modes. The columns are, from left to right: fitted ∗ 0 signal yield; the signal efficiency, (not including the branching fractions for K , K ,andKS decays); the systematic error on the selection efficiency, (∆B/B) ; the systematic error from the fit, (∆B/B)fit; the branching fraction central value (B); and the 90% C.L. upper limit on the branching fraction, including systematic uncertainties. direction relative to the B meson is correlated with the photon polarization, and is measureable when ∗ the K1 decays to Kππ via two isospin-related K resonances. In the Standard Model, weak interactions in the radiative penguin diagram cause the s quark in the b → s transition to be left-handed; the photon from the decay of pseudoscalar B must therefore also be left-handed. This leads to a large asymmetry in this decay plane angle. With moderate statistics, it may be possible to measure the distribution of this angle, a sensitive test of the Standard Model. Howewer, the analysis is complicated not only by other kaon resonances decaying to the same final state, but also by the poorly-understood D-wave contribution to the K1 decay, as well as the smaller (4%) decay mode K1 → Kρ, ρ → ππ.

Our eﬀorts have focused on understanding the phenomenology of the B → K1π decay. We have constructed analytical and numerical models to estimate the sensitivity to the photon polarization in various regions of the K1 Dalitz plot. We have implemented a decay model in the EvtGen event ∗ generator to incorporate the interference between K1 decays via isospin-related K resonances. We are beginning to formulate an analysis to identify and measure the B → K1γ process, and hope to perform a measurement of the photon polarization as well.

6.3.3 Radiative Leptonic Decays of the B Meson

Chen, Dubois-Felsmann, and Hitlin are working on measurements of the branching fractions of the + + radiative leptonic decay modes B → l νlγ, where l = e, µ. These measurements are relevant to the determination of one of the basic theoretical parameters of B decays, the pseudoscalar decay constant fB. This parameter, in turn, is an important input to the global ﬁt procedures that are used to assess the consistency of weak decay data with the Cabibbo-Kobayashi-Maskawa (CKM) model of ﬂavor mixing. Thus far it has not been possible to measure it directly. In the Wolfenstein parameterization of the CKM matrix, Eqn. 6.3, the parameters λ and A are relatively well measured, while ρ and η, which determine one vertex of the standard unitarity triangle, are less so. One constraint on ρ and η comes from semileptonic B decays:

2 2 ρ + η = |VudVub| (6.1) 6.3 Physics 133

+ + Table 6.2: Branching fractions of B → l νl

Leptonic mode Branching Fraction e 6.9 × 10−12 µ 2.9 × 10−7 τ 5.5 × 10−5

Another comes from neutral B-meson mixing:

2 2 −2 (1 − ρ) + η = |VtdVtb|∝∆mB · fB (6.2) where Vxy are elements of the CKM matrix and ∆mB is the mass diﬀerence between the B mixing eigenstates. The power of this constraint manifestly depends on the precision of our knowledge of fB.

    − λ2 − λ4 3 − Vud Vus Vub 1 2 8 λAλ(ρ iη)    2 4  6 Vcd Vcs Vcb = − 2 5 1 − − − λ − λ 2 2 + O(λ ). λ + A λ ( 2 ρ iη)12 8 (1 + 4A ) Aλ Vtd Vts Vtb 3 − − − 2 4 1 − − − 1 2 4 Aλ (1 ρ¯ iη¯) Aλ + Aλ ( 2 ρ iη)12 A λ (6.3)

The B meson decay constant fB has not been directly measured, and its theoretical value can range from 175 to 370 MeV, depending on the method of calculation [158, 159, 160, 161]. The purely leptonic B decay modes offer a direct way to measure fB with no additional theoretical uncertainties: 2 | |2 2 2 B + → + GF Vub 2 2 − ml (B l ν)= fBτBmBml 1 2 (6.4) 8π mB Unfortunately, as is evident in the above equation, these modes are helicity-suppressed, resulting in the estimated branching fractions listed in Table 6.2. Thus, the electronic mode may well never be observable. The muonic mode is the subject of active analysis efforts in both BABAR and Belle, but is not expected to be observable above background with the current integrated luminosities. The tau decay mode, also being actively pursued, presents the added difficulty of one or more extra missing neutrinos and again is not expected to lead to a measurement in the near future.

A less direct means of measuring fB is through the radiative leptonic decay mode. The presence of the photon removes the helicity suppression, but introduces some theoretical uncertainty into the determination of fB, which would no longer be directly observed, but rather could be calculated via heavy quark symmetry. Theoretical estimates of the radiative leptonic B branching fraction put it between 1.0 × 10−6 and 4.0 × 10−6 [162]. − In 1997, CLEO published results based on 2.5fb 1 of data and set upper limits of 2.0 × 10−4 and 5.2 × 10−5 on the branching fractions of the electron and muon channels, respectively [163]. They performed a cut-based analysis that looked for a signal lepton and photon, assigned all other visible particles to the recoil B, and then reconstructed the energy and momentum of the signal neutrino. Some examples of their cut variables were: Fox-Wolfram 2nd moment (R2), total visible charge, tag lepton energy, recoil B mass and energy, and lepton-photon opening angle. Our plan is to perform a blind analysis on the full BaBar dataset collected as of Summer 2002 − (≈ 81.9fb 1). Assuming a branching fraction of 3 × 10−6, we would expect about 6 events per mode at the 2% eﬃciency achieved in the CLEO analysis. Thus, a key goal of our work is to improve the sensitivity of the analysis procedure. 134 6 BaBar

The first step in our analysis was to generate a Monte Carlo sample of signal events. Using the tree- level matrix element given by Korchemsky, Pirjol, and Yan [164], we wrote an event generator model which provided events for the BaBar detector simulation. This model was added to BABAR Monte Carlo production, so samples of signal events continue to be generated on a regular basis, weighted by luminosity and reflecting changes in detector conditions. Next, starting with the electron mode, we performed a CLEO-style analysis on our Monte Carlo signal events, as well as on generic B, charm, uds, and tau Monte Carlo. After manually optimizing the CLEO event selection for BABAR, we got a Figure-of-Merit(FOM) (≡ S/ (S + B)) = 0.86, with a signal efficiency of 3%, using a cut-based electron particle ID (PID) for the signal electron. In order to improve upon this, we have focused our efforts on improving both cut optimization and analysis technique. Since we have more than 10 selection criteria, we’ve concluded that it would be better not to tune our cuts manually. The typical techniques used in BABAR analyses are Fisher discriminants and neural networks. We’ve decided to try something more novel: binary decision trees. In particular, we have explored the capabilities of CART, a commercial data-mining package which uses binary recursive partitioning to classify a training sample of data and create a predictive model. The predictive model exploits the correlations between different variables so that we can make more varied and efficient cuts. The use of this technique in high energy physics was pioneered by Jonathan Dorfan and other physicists on Mark II for e/π separation. In Mark III, Daniel Coffman, a former graduate student at Caltech, developed a tree-based algorithm for e/π separation [165]. Our studies have shown that a crude optimization using CART allows us to more than double our signal efficiency to about 6.9%, while increasing the FOM to 1.1. Still more work needs to be done in order to adapt CART to our particular analysis. We also plan to compare results from CART with those from a neural network. In parallel with our studies of cut optimization we have made other efforts to refine our analysis technique. In particular, we hope to improve signal event reconstruction and selection in order to better separate signal from background. Some potential areas of improvement include: electron PID, recoil ◦ 0 track PID, π rejection, and KLPID:

• We have switched from a cut-based electron selector to a likelihood-based selector for the signal electron PID, resulting in a slightly better signal to background separation. • We used to assign the charged pion mass to all charged tracks in the recoil, but have now taken advantage of kaon, proton, and electron PID lists in order to assign more accurate mass hypotheses to all charged tracks. • Our analysis attempts to exclude events where the radiative photon candidate in fact arises from a π◦decay by rejecting candidates that, when combined with any other photon, have an invariant mass near that of the π◦. Unfortunately, due to the combinatorics resulting from the large number of photons in typical B decays, this criteria rejects a signiﬁcant fraction of signal events. Therefore, we’ve looked at ways to improve the identiﬁcation of true π◦’s by, for example, combining the non-signal π◦photon with another photon to see if that combination also forms a π◦.

0 • We cannot, in general, correctly account for KL’s produced in the decay of the recoil B; they either escape detection altogether or when they do interact in the EMC or IFR, their energy is not well measured. In our original analysis any neutral energy clusters in the EMC arising from 0 0 KLinteractions were treated as photons. Algorithms for identifying KL’s in the EMC and IFR 0 0 have been developed by BABAR for use in the B → J/ψ K L CP analysis. We have now included these algorithms in our analysis code and will investigate whether the information is useful in improving the reconstruction of the recoil B. 6.3 Physics 135

We have recently finished producing analysis ntuples from the entire Monte Carlo dataset, including the above improvements. We hope to finalize the analysis procedure in the near future, compare the results from continuum Monte Carlo with off-resonance data, and then process the on-resonance data, unblinding when all checks have been completed. We will also re-tune our procedure for the muon − mode. Based on the results of simulations to date, we expect this analysis, using the 81.9fb 1 dataset, to produce an upper limit somewhat above the standard model predictions for the branching fraction.

6.3.4 Dalitz Plot Analysis of Three Body Charmless Decays B+ → h+h+h− h =(K/π)

As more and more BABAR data became available there has been an increased interest in the collaboration in Dalitz plot analyses of B decays into three body charmless final states. Dvoretskii has developed models for simulating such decays in the BABAR EvtGen simulation framework. The code properly simulates the interference between different subchannels in Dalitz plot decays. Importance sampling is used to correctly simulate narrow resonance structures. Dvoretskii has also developed RooFitDalitz - an extension to the BABAR RooFit fitting framework. RooFitDalitz is now used in several groups within the collaboration for performing kinematics simulation and fitting of three body B decays. Dvoretskii, Dubois-Felsmann and Hitlin are currently working on the full Dalitz plot analysis of charged charmless B decays which will become the basis for Dvoretskii’s PhD thesis. The tools developed by Dvoretskii make it possible to generate a library of functions that can be used to create a fit model which includes the effects of detector efficiency and resolution. A study of different background sources is under way. Using the models for signal and background BABAR charmless data will be analyzed to determine the fractional contributions of different subchannels and their relative phases. The analysis will be documented in BABAR analysis document (BAD) 643.

6.3.5 Measurement of the branching fraction and CP -violating asymmetries in B0 → D∗±D∓

In the Standard Model, the time-dependent CP -violating asymmetries in B → D∗±D∓ decays are ≡ − ∗ ∗ related to the angle β arg[ VcdVcb/VtdVtb]. We have measured the branching fraction and CP - violating asymmetries in B → D∗±D∓ decays using a sample of 87.9 ± 1.0 million BB decays. 0 0 Decays where b → ccs¯ , such as B → J/ψ K S , can be used to measure sin2β, however the Standard Model predicts that the time-dependent CP violating asymmetries in b → ccd¯ decays such as B0 → D(∗)D¯(∗) can also measure sin2β. An independent measurement of sin2β in these modes therefore provides a test of CP -violation in the Standard Model. This is especially imperative because very reasonable choices of SUSY parameters (˜b and gluino masses in the range 100-300 GeV) can produce measurable diﬀerences in the values of sin2β obtained from b → ccs¯ and b → ccd¯ [166]. This is one of the more theoretically promising currently available avenues to search for supersymmetry.

− ∗± ∓ Last year, using 56 fb 1of BABAR data, we reconstructed 59 B0 → D D events and measured the time-dependent CP asymmetries for B0 → D∗±D∓ to be:

− ± ± S−+ = 0.38 0.88(stat.) 0.12(syst.), 136 6 BaBar

b) a) d¯

c ¯b W d¯ W c ¯b c¯ t¯ c¯ d d d d Figure 6.4: The leading-order Feynman graphs for B0 → D∗±D∓ decay: a) tree diagram and b) penguin diagram.

− ± ± C−+ = 0.30 0.50(stat.) 0.13(syst.), ± ± S+− =+0.43 1.41(stat.) 0.23(syst.), − ± ± C+− = 0.53 0.74(stat.) 0.15(syst.).

which were presented at Moriond and at APS, and included as part of Justin Albert’s Ph.D. dissertation prior to his arrival at Caltech. This year, with 81 fb−1of data and using a more eﬃcient signal selection, we reconstruct a signal yield of 113 B0 → D∗±D∓ events. We have measured the time-dependent CP asymmetries, as well as the branching fraction, and also the time-integrated direct CP asymmetry between the rates to D∗+D− and D∗−D+ (which is physically independent of the time-dependent CP asymmetries). The results have recently been presented at Moriond, Lake Louise, and APS. The results are

− ± ± S−+ = 0.24 0.69(stat.) 0.12(syst.), − ± ± C−+ = 0.22 0.37(stat.) 0.10(syst.), − ± ± S+− = 0.82 0.75(stat.) 0.14(syst.), − ± ± C+− = 0.47 0.40(stat.) 0.12(syst.).

The branching fraction is measured to be

B(B → D∗±D∓)=(8.8 ± 1.0(stat.) ± 1.3(syst.)) × 10−4

and the time integrated direct CP asymmetry between rates to D∗+D− and D∗−D+ is

A = −0.03 ± 0.11(stat.) ± 0.05(syst.).

Plots of the decay time difference (∆t) between the reconstructed and tag B, as well as the raw CP -violating asymmetry as a function of the decay time difference, may be seen on the following pages. We submitted these results to Physical Review Letters, and our paper was recently accepted for publication. The paper is currently available at hep-ex/0303004. These results are presently limited by small statistics, but with BABAR’s rapidly increasing data set, these modes are beginning to give us a window in the search for CP violation beyond the Standard Model. Later this year, we intend to make the first measurements of and search for direct CP violation in the B± → D(∗)±D(∗)0 modes, and also to find limits on (or branching fractions of) the yet-undiscovered color-suppressed B0 → D(∗)0D¯ (∗)0 modes. 6.3 Physics 137 ) 2 30 signal a) signal b) combinatoric bkgd. combinatoric bkgd. 25 peaking bkgd. peaking bkgd. 20

Events / (2.5 MeV/c 10

0 5.2 5.22 5.24 5.26 5.28 5.3 5.2 5.22 5.24 5.26 5.28 5.3 2 2 mES (GeV/c ) mES (GeV/c )

∗− + ∗+ − Figure 6.5: The mES distributions of a) B → D D and b) B → D D candidates with |∆E| < 18 MeV. The ﬁt includes Gaussian distributions to model the signal and a small peaking background component, and an ARGUS function to model the combinatoric background shape.

6.3.6 A search for the beyond-Standard-Model process B0 → invisible (+ gamma)

The search for new physics is one of the primary goals of the BABAR experiment. In addition to searching for new sources of CP violation, which have so far been fairly consistent with Standard Model predictions, BABAR must ensure that other manifestations of new physics in B decays are not able to elude notice. Less than 50% of the total width of the B is explained by known branching fractions; few constraints exist on decays beyond what is expected from the Standard Model. There are no significant experimental constraints on invisble decays of heavy flavor. The Standard Model predicts infinitesimally small branching fractions for these decays. However, without causing any inconsistency with all current published experimental results, such decay rates could in principle be up to the order of 10% [167]. In the Standard Model, the lowest-order decay processes for B0 → invisible (+ gamma) are second- order weak decays: Each of these diagrams are highly suppressed within the Standard Model. For the ννγ¯ channel, the expected Standard Model branching fraction should be of similar order to other Z-mediated second- order weak decays, for example K+ → π+νν¯, which has been measured by BNL-787 to be at the 10−10 level [168]. The νν¯ channel has an additional helicity suppression and thus for all intents and purposes should never occur at all. Thus there should be no visible Standard Model contamination in these channels; an observation at BABAR would necessarily imply the existance of new physics. Significant rates for invisible B0 decays can occur in several physical models, ranging from phenomenological models motivated by inconsistencies in neutrino experimental data with the Standard Model, to theoretical models motivated by attempts to resolve fundamental open questions, such as the 138 6 BaBar

0 *+ - 10. B → D D a) B0 tags

b) 10. − B0 tags

1.0 c)

-1.0 10. B0 → D*-D+ d) B0 tags

e) 10. − B0 tags

1.0 f)

-1.0 -5 0 5 ∆t (ps)

∗+ − 0 Figure 6.6: Distributions of ∆t for B → D D candidates in the signal region with a) a B tag (NB0 ) 0 − and b) with a B tag (NB0 ), and c) the raw asymmetry (NB0 NB0 )/(NB0 + NB0 ). The solid curves are the ﬁt projections in ∆t. The shaded regions represent the background contributions. Figures d), e), and f) contain the corresponding information for D∗−D+. 6.3 Physics 139

a) b) c) γ γ γ W + ¯ ν b ¯b t¯ ν ¯b W + ν t l W t Z Z d ν¯ t − W − d ν¯d W ν¯ Figure 6.7: The lowest-order Standard Model Feynman graphs for B0 → invisible (+ gamma) decay: a) box diagram, b) qq¯ weak annihilation diagram, and c) W +W − weak annihilation diagram.

∗ 0 0 d ν˜i χ˜1 d χ˜1 d ν¯i d˜L ˜bR 0 ¯b ν¯i ¯b ν¯i ¯b χ˜1

(a) (b) (c)

0 −→ 0 Figure 6.8: From reference [169]: light neutralino production in B-meson decays: (a-c) Bd ν¯iχ˜1. hierarchy problem. An example of the former is described in reference [169]. This attempt to explain NuTeV’s observation of an anomalous excess of dimuon events provides a model for the production of long-lived heavy neutral particles consistent with the NuTeV data [170]. They propose a supersymmetric model with a neutralino LSP that avoids tight LEP contraints on neutralino production by coupling to decays of B mesons. Their model predicts invisible B decays with a branching fraction − − in the 10 7 to 10 5 range, which is just below visibility with the current BABAR data sample. The SUSY production mechanism for invisible B0 decays is shown above in Figure 6.8. Figure 6.9 shows the MSSM phase-space corresponding to this model, which is completeley consistent with LEP limits on neutralino production. Figure 6.10 shows the impact on the B0 → invisible branching fraction compared with the expected number of dimuon events seen at NuTeV. In addition, models using large extra dimensions to solve the hierarchy problem also can have the eﬀect of producing signiﬁcant, although small, rates for invisible B decays. Examples of such models, and their predictions, may be found in references [171, 172, 173].

6.3.7 Analysis overview

The analysis takes advantage of the fact that, at the Υ(4S), when one reconstructs a B decay, one can be certain that there was another B on the other side. The essence of the analysis is to reconstruct a B decay and, in the rest of the event, look for consistency with “nothing” or single gamma hypotheses. Similar to most rare decay analyses, this analysis is limited by statistics and thus it is critical to get as high an efficiency as possible. The efficiency is entirely dependent on the choice of tag algorithm, as the signal-side selection efficiency (for “nothing” or just a gamma) is nearly independent of the tag that is used for the opposite B. We decided on the tag strategy used by the semileptonic-tag B → Kνν¯ analysis [174] and, more recently, by the semileptonic-tag B → τν analysis [175], due to its very high efficiency and its well-understood properties. The semileptonic tag approach relies on identifying a D(∗)±lν candidate in one of three D0 modes (D0 → Kπ, Kπππ,orKππ0) and one D± mode (D± → Kππ). Due to the very high branching fractions of these modes, and the background rejection due to both the lepton and the fully reconstructed D(∗), this is an efficient and clean tag. 140 6 BaBar

Figure 6.9: From reference [169]: solutions in (M1, M2, µ, tan β) giving 4.5GeV ≤ Mχ0 ≤ 5.5GeV ˜1 in the cross-hatched region. Points below the horizontal hatched line are excluded by the requirement that Mχ+ > 100 GeV. ˜1

We use a neural network to identify potential B0 → invisible (+ gamma) events from the tagged sample. The selection criteria are primarily based on two things: the number of tracks on the opposite side (signal will have 0, or at most one from leftover pions from D∗∗ on the tag side or from beam background), and the amount of EMC clusters and energy on the opposite side (minus the cluster from the highest-momentum photon in the case of B0 → invisible + gamma). Other information, such as the 0 number of reconstructed KLin the event, the event shape variables, and the quality of the reconstructed tag B0 also factor in. Preliminary plots of neural net output from signal Monte Carlo events and − from all BABAR background types in 25 fb 1of background Monte Carlo are shown in figure 6.11. We perform two independent analyses to validate the analysis and ensure we have not fallen victim to any unnoticed differences in data and MC that would affect the results of the analysis. One is to do a similar search for B± → invisible (+ gamma); this should always produce a result consistent with a zero branching fraction, as this would be a charge-nonconserving decay. Another is to use a cut-based selection rather than the neural net; this should, of course, always produce results consistent with (but not quite as sensitive as) the neural net selection. The primary systematic uncertainties are due to potential differences in data and Monte Carlo; studies are performed to estimate these uncertainties. This analysis is still in progress. We have tuned the selection criteria on Monte Carlo samples, done validation comparisons of data and Monte Carlo, and run over the entire BABAR dataset and Monte Carlo production to create the analysis fit files. We expect this analysis to be released for the 6.3 Physics 141

Figure 6.10: From reference [169]: number of events in the NuTeV detector for neutralino production in B-meson decays as a function of the neutralino lifetime.

10 3

10 2 10 2

10 10

1 1

0 0.25 0.5 0.75 1 0 0.25 0.5 0.75 1 neural net output neural net output

Figure 6.11: Plots of analysis neural network output for (left) signal B0 → invisible Monte Carlo events − and (right) from all background types in 25 fb 1of BABAR background Monte Carlo. 142 6 BaBar conferences this summer, and to be published shortly thereafter.

6.3.8 Studies of quark mixing matrix and heavy ﬂavor decay parametriza- tions

One of the major goals of present research on heavy flavor physics, and in particular on the properties of B mesons and their decays, is an understanding of whether CP violation and other observations can be accounted for within the Standard Model, using the three-generation Cabbibo-Kobayashia-Maskawa (CKM) quark mixing matrix. Assessing this requires the combination of a great deal of experimental data with corresponding theoretical predictions. The Caltech high energy physics group is heavily involved in both sides of the collection of this data, through the BABAR experimental collaboration as well as the heavy quark decay research in the theory group under Prof. Wise. A number of methods for performing global fits to this collection of information have been presented in the literature over the past 5–10 years, included the so-called “scanning method” described in the “BaBar Physics Book” [90]. Several of the more recently developed techniques aim to produce quantitative measures of the overall consistency of the experimental data and theoretical model, and on preferred values for the CKM matrix elements. A key challenge in the development of these procedures has been the representation in the fits of non- statistical uncertainties, especially on theoretical parameters such as the B meson pseudoscalar decay constant fB and the “bag factor” BK arising in K meson decay calculations. Calculations of these parameters are generally published with “uncertainty ranges” reflecting the authors’ degree of belief in the correctness of model assumptions or the effects of ignoring higher-order terms in expansions. These do not in general have a precise statistical definition (except for those portions arising from statistics- limited Monte Carlo or lattice calculations), and there is no clear consensus in the community for the meaning to be attached to their precise numerical value. Nevertheless, in order to perform a “global fit” that incorporates inputs with such uncertainties, in the framework of a standard minimization-of-deviations fitting procedure, the goodness-of-fit metric to be used must somehow be constructed to give them quantitative effect. Because of the lack of a precise statistical meaning for these uncertainties or a consensus on how to interpret them, the schemes used to do this have been a subject of considerable debate in the literature, and the procedural value judgments involved inevitably color the interpretation of any quantitative results obtained from such analyses. Differences arising from these value judgments in the results obtained from the various global CKM fitting techniques in the literature have been shown to be of a scale comparable to those arising from the other uncertainties in the input data and therefore cannot be ignored.[176] In the fitting procedures published to date, these choices tend to be hidden in the details of the analysis, and their effects are not readily made manifest in a study of the outputs of the fit. We believe that this obscures the importance of these choices and lends an undue patina of statistical precision to the quantitative results obtained. This may result in inappropriate conclusions on the significance of results obtained from CKM fits, such as statements about the consistency of data with the Standard Model. Dubois-Felsmann, Hitlin, and Porter, in collaboration with visiting physicist Eigen, have developed new techniques for studying and visualizing the sensitivity of global CKM fits to non-statistical uncertainties and to these choices for their parametrization, as well as techniques for visual evaluation of the consistency of experimental and theoretical inputs that minimize the implicit use of these value judgments, while illuminating their effects. Our approach was discussed in some detail in last year’s annual report. These techniques have been presented within the BaBar collaboration, and, by Dubois-Felsmann 6.4 PEP-II, SuperPEP-II, and SuperBABAR 143 and Eigen, at the February 2002 CKM workshop held at CERN [176]. An updated paper was submitted to the ICHEP conference in Amsterdam [177]. We are currently refining the analysis and updating it to include recent results and are preparing a paper for publication.

6.3.9 Statistics Working Group

Recognizing the diﬃculty experimenters often have with statistical issues, especially as our analysis methods have become more sophisticated, BABAR put together a small group to think about these issues and prepare a set of recommendations on how to deal with them. Porter chairs this “Statistics Working Group”, and Hitlin is also a member from Caltech. The issues include such matters as:

1. How to quote conﬁdence intervals, e.g., when to quote one-sided, and when to quote two-sided intervals.

2. What ﬁtting procedures and hypothesis tests are appropriate in various circumstances.

3. How to treat systematic uncertainties.

4. How best to display results graphically, including the representation of uncertainties.

5. How to deal with the interpretation of results, e.g., when to claim a “signal”.

The Statistics Working Group held several meetings to discuss these issues. These discussions were augmented by online interchanges using BABAR’s hypernews forum. The group has a web page [178] to aid in this communication. A report has been produced [179] which contains a set of recommendations, in addition to a good deal of pedagogical material. The recommendations have been released to the collaboration, and there has been a good eﬀort by the physics organization to apply them in the production and reporting of our physics results. As we get feedback and additional questions, the Statistics Working Group formulates informal responses, and collects items for addition to the formal report.

6.4 PEP-II, SuperPEP-II, and SuperBABAR

Over the next four years or so, the luminosity of the PEP-II asymmetric B Factory will increase to 2 × 1034cm−2s−1. The Caltech group is taking on responsibilities in PEP-II to ensure the best possible performance of the machine, as it is crucial for PEP-II to remain fully competitive with KEK-B over this period. These responsibilities include the development of a laser wire scanner to allow the direct measurement of vertical beam size in at least one ring and the simulation of beam-beam interactions in the high bunch charge, low β regime. Planning exercises done by the BABAR Collaboration have shown that only relatively minor detector upgrades are required to cope with the planned increased luminosity. This program will yield a total data sample of ∼1000 fb−1 by 2008. KEK-B/Belle are expected to have similar data samples on this time scale. At that point, the LHC will begin producing physics, and the B physics data samples from LHCb, ATLAS and CMS will in short order outpace those produced at the current B Factories. An asymmetric B Factory has powerful and unique advantages over a hadronic B physics experiment, but these advantages cannot outweigh a large disparity in sample size. The obvious question then is whether 144 6 BaBar

the luminosity of an asymmetric B Factory can be increased to produce data samples comparable to those at LHC experiments. John Seeman of SLAC has inititated study of an advanced asymmetric B Factory, SuperPEP- II, which would have a luminosity of 1036cm−2s−1. It appears that such a machine can indeed be built. SuperPEP-II would produce B0 and B± meson samples quite comparable to those at hadronic experiments, but with unique advantages. The Caltech group has been active in organizing studies of the physics capabilities of such a machine and the design requirements for a detector, SuperBABAR, that would be capable of doing physics at 1036, commencing in the period 2009 to 2011. An initial look at physics opportunities, accelerator considerations and the design of a new detector can be found in the Snowmass 2001 proceedings [180]. SLAC submitted information on SuperPEP-II and SuperBABAR to the recent DOE Facilities Panel. A1036 machine can produce 10 ab−1 in a running year, thus increasing then-existing data samples by more than an order of magnitude in a single year. With such large samples of heavy quark and τ decays it is possible to obtain remarkable sensitivity to physics beyond the Standard Model [181]. The approach to finding new physics is two-fold: 1) push the precision of overconstrained tests of the CKM sector of the Standard Model to the few percent level, and 2) search for effects, such as CP violation in rare decays, that are sensitive to new physics. BABAR has formed a 1036 Study Group, co-chaired by David Hitlin and by Francesco Forti of INFN Pisa. This group is actively looking at the new physics discovery potential of a 1036 machine. In March, 2003 there were a series of presentations, under the rubric B-Day, within the ongoing SLAC Scenarios Study. In May, 2003 there was a Workshop on the Discovery Potential of an Asymmetric B Factory at 1036 Luminosity. This Workshop, with 123 registrants, was the start of an ongoing study which will culminate in October, 2003 with a concluding workshop at SLAC. There will then be a joint workshop with KEK in Hawaii in January, 2004. The purpose of these study group and workshop activities is to sharpen our understanding of the role of flavor physics in clarifying any discoveries, such as supersymmetry, that may be made at the LHC. The discovery of supersymmetric particles at the LHC would be a remarkable achievement, but a full understanding of the squark sector of any newly proposed theory will require a detailed study of the effect of squarks on rare loop-dominated B decay processes. Observation of squarks at the LHC will provide information on the diagonal terms in the squark mass matrix, but an understanding of the off-diagonal matrix elements requires a detailed study of CP -conserving and CP -violating rare B decays. In general, theoretical predictions of inclusive rare B decays are more precise than those of exclusive decays. A high luminosity asymmetric B Factory is the only venue for the study of such inclusive decays; this cannot be done at a hadronic experiment. Thus, the main thrust of the workshop activities will be to refine our understanding of the precision of Standard Model predictions of rare B decay rates and kinematic distributions, as well as to evaluate in detail the limits of the measurement precision obtainable with 10 to 50 ab−1 samples. The Snowmass 1036 study showed that technologies exist to cope with rates and backgrounds at this new accelerator. We are now developing a specific upgrade path for the existing BABAR detector to a device capable of doing physics at 1036 [182], [183]. The key to such an upgrade is an electromagnetic calorimeter capable of functioning in the radiation and backround environment of 1036. The existing CsI(Tl) calorimeter, while it performs extremely well in the current environment, is not usable at the target luminosities. The chosen calorimeter technology determines the scale of a colliding beam detector. The Snowmass strawman detector used an LSO crystal calorimeter, leading in a natural way to a small device with a 3 Tesla field. A calorimeter based on detection of scintillation light in liquid xenon provides the basis for an upgraded detector based on the BABAR flux return and solenoid. The upgraded detector also includes a silicon tracking system with two pixel layers and seven double-sided strip layers, as well as a new BIBLIOGRAPHY 145

DIRC particle identification system that reads out the Cherenkov light without the need for the large water-filled stand-off box of the current detector, a major source of background. Several attempts have been made to develop a practical calorimeter using liquid xenon scintillation light; none has been particularly successful. The scintillation light is at the far UV wavelength of 175 nm. It has proved difficult to combine adequate photon detection efficiency with uniformity of light collection efficiency. The design considered here converts the 175 nm light to the optical region using tetraphenyl butadiene on the walls of optically-isolating compartments made of expanded ptfe. This light, at 350 nm is absorbed in a coil of standard Kuraray Y11 fiber and shifted to 510 nm. The WLS fibers are readout by a pixilated RMD avalanche photodiode placed in the cryostat. The projected energy resolution is expected to exceed that of a high quality crystal calorimeter. Position resolution, important for isolating high momentum π0’s, should be superior. In addition, it is easily possible to provide longitudinal segmentation, which is difficult to do with a crystal, thereby improving π/e discrimination. Most importantly, radiation damage is not a factor in a liquid xenon-based device. Using funds provided by the Caltech PMA Division, we are constructing a prototype device to allow us to optimize the various components of the calorimeter. This will be done using radioactive sources and cosmic rays. We expect first results by the Fall of 2003. We then expect to develop a beam test prototype on the time scale of two years or less. This test will be greatly facilitated by equipment developed for liquid xenon R&D at the SSC that we have obtained from MIT.

Bibliography

[1] B. Aubert et al. [BABAR Collaboration], “The BaBar detector,” Nucl. Instrum. Meth. A 479,1 (2002).

[2] B. Aubert et al. [BABAR Collaboration], “Measurement of the CP-violating asymmetry amplitude sin 2β,” Phys. Rev. Lett. 89, 201802 (2002).

[3] B. Aubert et al. [BABAR Collaboration], “Improved measurement of the CP-violating asymmetry amplitude sin(2β)”, arXiv:hep-ex/0203007.

[4] B. Aubert et al. [BABAR Collaboration], “Observation of CP violation in the B0 meson system,” Phys.Rev.Lett.87, 091801 (2001).

[5] B. Aubert et al. [BABAR Collaboration], “Measurement of the CKM matrix element |V(ub)| with B → rho e nu decays,” Phys. Rev. Lett. 90, 181801 (2003).

[6] B. Aubert et al. [BABAR Collaboration], “Simultaneous measurement of the B0 meson lifetime and mixing frequency with B0 → D*- l+ nu/l decays,” Phys. Rev. D 67, 072002 (2003).

[7] B. Aubert et al. [BABAR Collaboration], “A study of the rare decays B0 → D/s(*)+ pi- and B0 → D/s(*)- K+,” Phys. Rev. Lett. 90, 181803 (2003).

[8] B. Aubert et al. [BABAR Collaboration], “A measurement of the B0 → J/ψπ+π− branching fraction,” Phys. Rev. Lett. 90, 091801 (2003).

[9] B. Aubert et al. [BABAR Collaboration], “Study of inclusive production of charmonium mesons in B decay,” Phys. Rev. D 67, 032002 (2003).

[10] B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fraction for inclusive semileptonic B meson decays,” Phys. Rev. D 67, 031101 (2003). 146 6 BaBar

[11] B. Aubert et al. [BABAR Collaboration], “Measurements of branching fractions and CP-violating asymmetries in B0 → pi+ pi-, K+ pi-, K+ K- decays,” Phys. Rev. Lett. 89, 281802 (2002).

[12] B. Aubert et al. [BaBar Collaboration], “Measurement of the branching fraction and CP content for the decay B0 → D*+ D*-,” Phys. Rev. Lett. 89, 061801 (2002).

[13] B. Aubert et al. [BABAR Collaboration], “Search for T and CP violation in B0 - anti-B0 mixing with inclusive dilepton events,” Phys. Rev. Lett. 88, 231801 (2002).

[14] B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 lifetime with partially recon- 0 ∗− + structed B → D l nul decays,” Phys. Rev. Lett. 89, 011802 (2002) [Erratum-ibid. 89, 169903 (2002)].

+ ∗+ [15] B. Aubert et al. [BABAR Collaboration], “Measurement√ of Ds and Ds production in B meson decays and from continuum e+e− annihilation at s =10.6 GeV,” Phys. Rev. D 65, 091104 (2002).

[16] B. Aubert et al. [BABAR Collaboration], “A study of time dependent CP-violating asymmetries and ﬂavor oscillations in neutral B decays at the Upsilon(4S),” Phys. Rev. D 66, 032003 (2002).

[17] B. Aubert et al. [BABAR Collaboration], “Search for the rare decays B → Kl+l− and B → K∗l+l−,” Phys. Rev. Lett. 88, 241801 (2002).

[18] B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fractions for the exclusive decays of B0 and B+ to anti-D(*) D(*) K,” arXiv:hep-ex/0305003.

[19] B. Aubert et al. [BABAR Collaboration], “A search for B- → tau- anti-nu recoiling against a fully reconstructed B,” arXiv:hep-ex/0304030.

[20] B. Aubert et al. [BABAR Collaboration], “Observation of a narrow meson decaying to D/s+ pi0 at a mass of 2.32-GeV/c**2,” arXiv:hep-ex/0304021.

[21] B. Aubert et al. [BABAR Collaboration], “A Search for the decay B- → K- nu anti-nu,” arXiv:hep- ex/0304020.

[22] B. Aubert et al. [BABAR Collaboration], arXiv:hep-ex/0304014.

[23] B. Aubert et al. [BABAR Collaboration], arXiv:hep-ex/0304007.

[24] B. Aubert et al. [BABAR Collaboration], “Measurements of the branching fractions and charge asymmetries of charmless three-body charged B decays,” arXiv:hep-ex/0304006.

[25] B. Aubert et al. [BABAR Collaboration], “Measurements of CP-violating asymmetries and branching fractions in B meson decays to eta’ K,” arXiv:hep-ex/0303046.

[26] B. Aubert et al. [BABAR Collaboration], “Limits on the lifetime diﬀerence of neutral B mesons and on CP, T, and CPT violation in B0 anti-B0 mixing,” arXiv:hep-ex/0303043.

[27] B. Aubert et al. [BABAR Collaboration], “Observation of B meson decays to omega pi+, omega K+, and omega K0,” arXiv:hep-ex/0303040.

[28] B. Aubert et al. [BABAR Collaboration], “Observation of B meson decays to eta pi and eta K,” arXiv:hep-ex/0303039.

[29] B. Aubert et al. [BABAR Collaboration], “Evidence for B+ → J/psi p anti-Lambda and search for B0 → J/psi p anti-p,” arXiv:hep-ex/0303036. BIBLIOGRAPHY 147

[30] B. Aubert et al. [BABAR Collaboration], “A search for B+ → tau+ nu(tau) recoiling against B- → D0 l- nu/l X,” arXiv:hep-ex/0303034. [31] B. Aubert et al. [BABAR Collaboration], “Branching fractions in B → Phi h and search for direct CP violation in B+- → Phi K+-,” arXiv:hep-ex/0303029. [32] B. Aubert et al. [BABAR Collaboration], “Observation of the decay B+- → pi+- pi0, study of B+- → K+- pi0, and search for B0 → pi0 pi0,” arXiv:hep-ex/0303028. [33] B. Aubert et al. [BABAR Collaboration], “Measurements of the branching fractions of charged B decays to K+ pi- pi+ final states,” arXiv:hep-ex/0303022. [34] B. Aubert et al. [BABAR Collaboration], “Rates, polarizations, and asymmetries in charmless vector-vector B decays,” arXiv:hep-ex/0303020. [35] B. Aubert et al. [BABAR Collaboration], “Study of time-dependent CP asymmetry in neutral B decays to J/psi pi0,” arXiv:hep-ex/0303018. [36] B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fraction and CP-violating asymmetries in neutral B decays to D*+- D-+,” arXiv:hep-ex/0303004. [37] B. Aubert et al. [BABAR Collaboration], “Measurement of B0 → D/s(*)+ D*- branching fractions and B0 → D/s(*)+ D*- polarization with a partial reconstruction technique,” arXiv:hep- ex/0302015. [38] B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 meson lifetime with partial reconstruction of B0 → D*- pi+ and B0 → D*- rho+ decays,” arXiv:hep-ex/0212012. [39] B. Aubert et al. [BABAR Collaboration], “A measurement of the B0 → J/psi pi+ pi- branching fraction,” arXiv:hep-ex/0203034. [40] B. Aubert et al. [BABAR Collaboration], “Measurement of branching fractions of color suppressed decays of the anti-B0 meson to D0 pi0, D0 eta, and D0 omega,” arXiv:hep-ex/0207092. [41] B. Aubert et al. [BABAR Collaboration], “Dalitz plot analysis of D0 hadronic decays D0 → K0 K- pi+, D0 → anti-K0 K+ pi- and D0 → anti-K0 K+ K-,” arXiv:hep-ex/0207089. [42] B. Aubert et al. [BABAR Collaboration], “Measurement of branching ratios and CP asymmetries in B- → D0(CP) K- decays,” arXiv:hep-ex/0207087. [43] B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fractions for the exclusive decays of B0 and B+ to anti-D(*) D(*) K,” arXiv:hep-ex/0207086. [44] B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 → D*- a1+ branching fraction with partially reconstructed D*,” arXiv:hep-ex/0207085. [45] B. Aubert et al. [BABAR Collaboration], “Measurement of the first hadronic spectral moment from semileptonic B decays,” arXiv:hep-ex/0207084. [46] B. Aubert et al. [BABAR Collaboration], “Search for decays of B0 mesons into pairs of leptons,” arXiv:hep-ex/0207083. [47] B. Aubert et al. [BABAR Collaboration], “Evidence for the flavor changing neutral current decays B → Kl+l-andB→ K* l+ l-,” arXiv:hep-ex/0207082. [48] B. Aubert et al. [BABAR Collaboration], “Measurement of the inclusive electron spectrum in charmless semileptonic B decays near the kinematic endpoint,” arXiv:hep-ex/0207081. 148 6 BaBar

[49] B. Aubert et al. [BABAR Collaboration], “Measurement of the CKM matrix element |V(ub)| with charmless exclusive semileptonic B meson decays at BABAR,” arXiv:hep-ex/0207080. [50] B. Aubert et al. [BABAR Collaboration], “Measurement of B0 → D/s(*)+ D*- branching fractions and polarization in the decay B0 → D/s*+ D*- with a partial reconstruction technique,” arXiv:hep- ex/0207079. [51] B. Aubert et al. [BaBar Collaboration], “Determination of the branching fraction for inclusive decays B → X/s gamma,” arXiv:hep-ex/0207076. [52] B. Aubert et al. [BABAR Collaboration], “b → s gamma using a sum of exclusive modes,” arXiv:hep-ex/0207074. [53] B. Aubert et al. [BABAR Collaboration], “Search for the exclusive radiative decays B → rho gamma and B0 → omega gamma,” arXiv:hep-ex/0207073. [54] B. Aubert et al. [BABAR Collaboration], “Measurement of time-dependent CP asymmetries and the CP-odd fraction in the decay B0 → D*+ D*-,” arXiv:hep-ex/0207072. [55] B. Aubert et al. [BABAR Collaboration], “Simultaneous measurement of the B0 meson lifetime and mixing frequency with B0 → D*- l+ nu/l decays,” arXiv:hep-ex/0207071. [56] B. Aubert et al. [BABAR Collaboration], “Measurement of sin(2beta) in B0 → Phi K0(S),” arXiv:hep-ex/0207070. [57] B. Aubert et al. [BABAR Collaboration], “A search for B+ → K+ nu anti-nu,” arXiv:hep- ex/0207069. [58] B. Aubert et al. [BABAR Collaboration], “Search for CP violation in B0 / anti-B0 decays to pi+ pi- pi0 and K+- pi-+ pi0 in regions dominated by the rho+- resonance,” arXiv:hep-ex/0207068. [59] B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fraction for B+- → chi/c0 K+-,” arXiv:hep-ex/0207066. [60] B. Aubert et al. [BABAR Collaboration], “Measurements of branching fractions and direct CP asymmetries in pi+ pi0, K+ pi0 and K0 pi0 B decays,” arXiv:hep-ex/0207065. [61] B. Aubert et al. [BABAR Collaboration], “A search for the decay B0 → pi0 pi0,” arXiv:hep- ex/0207063. [62] B. Aubert et al. [BABAR Collaboration], “A study of time dependent CP asymmetry in B0 → J/psi pi0 decays,” arXiv:hep-ex/0207058. [63] B. Aubert et al. [BABAR Collaboration], “A study of the rare decays B0 → D/s(*)+ pi- and B0 → D/s(*)- K+,” arXiv:hep-ex/0207053. [64] B. Aubert et al. [BABAR Collaboration], “Measurements of charmless two-body charged B decays with neutral pions and kaons,” arXiv:hep-ex/0206053. [65] B. Aubert et al. [BABAR Collaboration], “Measurements of the branching fractions of charmless three-body charged B decays,” arXiv:hep-ex/0206004. [66] B. Aubert et al. [BABAR Collaboration], “Evidence for the b → u transition B0 → D/s+ pi- and asearchforB0→ D/s*+ pi-,” arXiv:hep-ex/0205102. [67] B. Aubert et al. [BABAR Collaboration], “Measurements of branching fractions and CP-violating asymmetries in B0 → pi+ pi-, K+ pi-, K+ K- decays,” arXiv:hep-ex/0205082. BIBLIOGRAPHY 149

[68] B. Aubert et al. [BABAR Collaboration], “A measurement of the neutral B meson lifetime using partially reconstructed B0 → D*- pi+ decays,” arXiv:hep-ex/0203038.

[69] B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 lifetime with partial reconstruction of anti-B0 → D*+ rho-,” arXiv:hep-ex/0203036.

[70] B. Aubert et al. [BABAR Collaboration], “Rare B decays to states containing a J/psi meson,” arXiv:hep-ex/0203035.

[71] B. Aubert et al. [BABAR Collaboration], “Study of B± → J/ψπ± and B± → J/ψK± decays: Measurement of the ratio of branching fractions and search for direct CP-violating charge asymmetries,” Phys. Rev. D 65, 091101 (2002).

[72] B. Aubert et al. [BABAR Collaboration], “Direct CP violation searches in charmless hadronic B meson decays,” Phys. Rev. D 65, 051101 (2002).

[73] B. Aubert et al. [BABAR Collaboration], “Measurement of B → K∗γ branching fractions and charge asymmetries,” Phys. Rev. Lett. 88, 101805 (2002).

[74] B. Aubert et al. [BABAR Collaboration], “Study of CP-violating asymmetries in B0 → π+π−, K+π− decays,” Phys. Rev. D 65, 051502 (2002).

[75] B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fractions for ψ(2S) → e+e− and ψ(2S) → µ+µ−,”Phys.Rev.D65, 031101 (2002).

[76] B. Aubert et al. [BABAR Collaboration], “Measurements of the branching fractions of exclusive charmless B meson decays with η or ω mesons,” Phys. Rev. Lett. 87, 221802 (2001).

[77] B. Aubert et al. [BABAR Collaboration], “Search for the decay B0 → γγ,” Phys. Rev. Lett. 87, 241803 (2001).

[78] B. Aubert et al. [BABAR Collaboration], “Measurement of the B → J/ψK∗(892) decay amplitudes,” Phys. Rev. Lett. 87, 241801 (2001).

[79] B. Aubert et al. [BABAR Collaboration], “Measurement of branching fractions for exclusive B decays to charmonium ﬁnal states,” Phys. Rev. D 65, 032001 (2002).

[80] B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 and B+ meson lifetimes with fully reconstructed hadronic ﬁnal states,” Phys. Rev. Lett. 87, 201803 (2001).

+ − [81] B. Aubert et al. [BABAR√ Collaboration], “Measurement of J/psi production in continuum e e annihilations near s =10.6 GeV,” Phys. Rev. Lett. 87, 162002 (2001).

[82] B. Aubert et al. [BABAR Collaboration], “Measurement of branching fractions and search for CP- violating charge asymmetries in charmless two-body B decays into pions and kaons,” Phys. Rev. Lett. 87, 151802 (2001).

[83] B. Aubert et al. [BABAR Collaboration], “Measurement of the decays B → φK and B → φK∗,” Phys.Rev.Lett.87, 151801 (2001).

[84] A. Ryd [BABAR Collaboration], “Studies of CP violation at BaBar,” Nucl. Instrum. Meth. A 462, 57 (2001).

[85] B. Aubert et al. [BABAR Collaboration], “Measurement of CP violating asymmetries in B0 decays to CP eigenstates,” Phys. Rev. Lett. 86, 2515 (2001). 150 6 BaBar

[86] D. G. Hitlin [BABAR Collaboration], “First CP violation results from BaBar,” hep-ex/0011024. [87] J. Tennyson, “Asymmetric B Factory: Luminosity Optimization and Constraints on the Round Beam Parameter Base”, CALT-68-1632, Oct 1991. [88] An Investigation of Beam Pipe Cooling Techniques for a Proposed Particle Accelerator, H. Schem- ber and P. Bhandari, JPL D-7151, Feb 1990. [89] Proceedings of the Workshop on High Luminosity Asymmetric Storage Rings for B Physics,April 25-28, 1989, Caltech, D. Hitlin and F.C. Porter, eds.

[90] [BABAR Collaboration], The BABAR Physics Book, P.F. Harrison and H.R. Quinn, eds., SLAC-R-504 (1998). [91] J.Z. Bai et al. [BES Collaboration], “Measurement of the mass of the tau lepton,” Phys. Rev. D53, 20 (1996). [92] J.Z. Bai et al. [BES Collaboration], “Measurement of the mass of the tau lepton,” Phys. Rev. Lett. 69, 3021 (1992). [93] F.C. Porter [BES. Collaboration], “Measurement of the mass of the tau lepton,” In *Dallas 1992, Proceedings, High energy physics, vol. 1* 422-426. [94] L. Jones, “A Measurement of the Mass of the Tau Lepton”, Ph.D. thesis, Caltech, CALT-68-1920 (1995). [95] J. Z. Bai et al. [BES Collaboration], “Measurement of the inclusive charm cross-section at 4.03-GeV and 4.14-GeV,” Phys. Rev. D 62, 012002 (2000) [hep-ex/9910016]. [96] J. Z. Bai et al. [BES Collaboration], “Direct measurement of B(D0 → φX0)andB(D+ → φX+),” Phys.Rev.D62, 052001 (2000) [hep-ex/9907052].

[97] J.Z. Bai et al., “Evidence for the leptonic decay D → µνµ,” Phys. Lett. B429, 188 (1998).

+ + [98] J.Z. Bai et al. [BES Collaboration], “Direct measurement of branching ratio Ds → φX ,” Phys. Rev. D57, 28 (1998).

[99] J.Z. Bai et al. [BES Collaboration], “A Measurement of the branching fraction of Ds inclusive + + semileptonic decay Ds → e X,” Phys. Rev. D56, 3779 (1997).

[100] J.Z. Bai et al. [BES Collaboration], “Measurement of the pseudoscalar decay constant, fD,” SLAC-PUB-7147 (1996). [101] J.Z. Bai et al. [BES Collaboration], “A Direct measurement of the pseudoscalar decay constant, f(Ds),” Phys. Rev. Lett. 74, 4599 (1995).

[102] J.Z. Bai et al. [BES Collaboration], “A Direct measurement of the Ds branching fraction to φπ,” Phys. Rev. D52, 3781 (1995). [103] J. Panetta, “Decays of the ψ(2S) Meson to Baryonic Final States”, Ph.D. thesis, Caltech, CALT- 68-2216 (1999). [104] J. Z. Bai et al. [BES Collaboration], “A measurement of ψ(2S) resonance parameters,” Phys. Lett. B 550, 24 (2002). [105] J. Z. Bai et al. [BES Collaboration], “ψ(2S) two- and three-body hadronic decays,” Phys. Rev. D 67, 052002 (2003). BIBLIOGRAPHY 151

[106] J. Z. Bai et al. [BES Collaboration], “Measurement of ψ(2S) decays to baryon pairs,” Phys. Rev. D 63, 032002 (2001).

[107] J. Z. Bai et al. [BES Collaboration], “ψ(2S) → π+π−J/ψ decay distributions,” Phys. Rev. D 62, 032002 (2000).

[108] J. Z. Bai et al. [BES Collaboration], “First measurement of the branching fraction of the decay ψ(2S) → τ +τ −,” hep-ex/0010072.

[109] J.Z. Bai et al. [BES Collaboration], “Charmonium decays to axial vector plus pseudoscalar mesons,” Phys. Rev. Lett. 83, 1918 (1999).

[110] J.Z. Bai et al. [BES Collaboration], “Study of the P wave charmonium state chi(cJ) in psi(2S) decays,” Phys. Rev. Lett. 81, 3091 (1998) hep-ex/9807001.

[111] J.Z. Bai et al. [BES Collaboration], “Determination of J / psi leptonic branching fraction via psi(2S) → pi+ pi- J / psi,” Phys. Rev. D58, 092006 (1998).

[112] J.Z. Bai et al. [BES Collaboration], “Branching fractions for psi(2S) → gamma eta-prime and gamma eta,” Phys. Rev. D58, 097101 (1998).

[113] J.Z. Bai et al. [BES Collaboration], “Search for psi(2S) production in e+ e- annihilations at 4.03 GeV,” Phys. Rev. D57, 3854 (1998).

[114] J.Z. Bai et al. [BES Collaboration], “ψ(2S) hadronic decays to vector-tensor ﬁnal states,” Phys. Rev. Lett. 81, 5080 (1998).

[115] J.Z. Bai et al. [BES Collaboration], “Study of the hadronic decays of χc states,” hep-ex/9812016.

[116] J. Z. Bai et al., “The BES upgrade,” Nucl. Instrum. Meth. A 458, 627 (2001).

[117] J. Bagger et al. [American Linear Collider Working Group Collaboration], “The case for a 500-GeV e+e− linear collider,” hep-ex/0007022.

[118] T. Abe et al. [American Linear Collider Working Group Collaboration], “Linear collider physics resource book for Snowmass 2001,” SLAC-570 Resource book for Snowmass 2001, 30 Jun - 21 Jul 2001, Snowmass, Colorado.

[119] Z. G. Zhao et al., “Report of Snowmass 2001 working group E2: Electron positron colliders from the Phi to the Z,” in Proc. of the APS/DPF/DPB Summer Study on the Future of Particle Physics (Snowmass 2001) ed. R. Davidson and C. Quigg, arXiv:hep-ex/0201047.

[120] E. Fernandez et al., (MAC Collaboaration), Phys.Rev.Lett.51, 1022 (1983); N. Lockyer et al., (Mark II Collaboaration), Phys. Rev. Lett., 51, 1316 (1983).

[121] C. Albajar et al., (UA1 Collaboaration), Phys. Lett. 186B, 247 (1987); H. Albrecht et al.,(AR- GUS Collaboaration), Phys. Lett. 192B, 245 (1987).

[122] P. Oddone, in Proceedings of the UCLA Workshop: Linear Collider BB¯ Factory Conceptual Design, D. Stork, ed., World Scientiﬁc, p 243 (1987).

[123] Report of the B Physics Group: I. Physics and Techniques, G. Feldman et al., p 561, and II. Accelerator Technology, R. H. Siemann et al., p 572, in Proceedings of the 1988 DPF Summer Study, High Energy Physics in the 1990s (Snowmass ’88) (1989). 152 6 BaBar

[124] Feasibility Study for an Asymmetric B Factory Based on PEP, LBL PUB-5244, SLAC-352, CALT- 68-1589, October 1989. This study was changed to a version with equal circumference rings in: Investigation of an Asymmetric B Factory in the PEP Tunnel, LBL PUB-5263, SLAC-359, CALT- 68-1622, March 1990. [125] PEP-II – An Asymmetric B Factory, Conceptual Design Report, LBL-PUB-5379, SLAC-418, CALT-68-1869, UCRL-ID-114055, UC-IIRPA-93-01, June 1993. [126] “The Physics Program of a High Luminosity Asymmetric B Factory at SLAC”, D. Hitlin, ed., SLAC-353, LBL-27856, CALT-68-1588, October, (1989). [127] Workshop on Physics and Detector Issues for a High-Luminosity Asymmetric B Factory at SLAC, D. Hitlin, ed., SLAC-373, LBL-30097, CALT-68-1697, March 1991; Proceedings of B Factories: The State of the Art in Accelerators, Detectors and Physics, Stanford, CA, 6-10 Apr 1992. [128] “Automated Data Quality Monitoring in the BaBar Online and Oﬄine Systems.” S. Metzler et. al. for the BaBar Computing Group. Presented at CHEP 2000, Padova, IT, 7-11 February 2000. CALT-68-2261. [129] “Production Experience with CORBA in the BaBar Experiment.” S. Metzler et. al. for the BaBar Computing Group. Presented at CHEP 2000, Padova, IT, 7-11 February 2000. [130] “BABAR Fast Monitoring Fast-Feedback System.” S. Metzler et. al. for the BaBar Computing Group. Presented at HEPVis99, Orsay, FR, 6-10 September 1999. [131] Ed Frank, Cathy Cretsinger, Gregory Dubois-Felsmann, Alan Eisner, Larry Gladney, Frederic Kral, Anders Ryd, Vivek Suraiya, Songhoon Yang, “Architecture of the BABAR Level 3 Software Trigger”, BABAR-Note-463, 21-OCT-98. [132] R.Y. Zhu et al., “CsI(Tl) Radiation Damage and Quality Improvement”, BaBar Note 166, CALT- 68-2117 (1998). [133] R.Y. Zhu, “Electron Identiﬁcation and Pion Rejection with BaBar CsI(Tl) Crystal Calorimeter”, BaBar Note 151 and CALT 68-2146 (1998). [134] R.Y. Zhu, “Radiation Damage in Scintillating Crystals”, Nucl. Instr. and Meth. A413 297 (1998). [135] R.Y. Zhu, “Precision Crystal Calorimetry in High Energy Physics”, Proceedings of DPF99 (1999).

[136] S. Yang and T. Glanzman, “CORBA Evaluations for the BABAR Online System”, SLAC-PUB- 7988, Nov 1998; Contributed to International Conference on Computing in High-Energy Physics (CHEP 98), Chicago, IL, 31 Aug - 4 Sep 1998. [137] K. Arisaka et al., Nucl. Instr. Meth. A385 (1997) 74; D. Boutigny et al., IEEE Trans. Nucl. Sci. 44 (1997) 44; K. Arisaka et al., Nucl. Instr. Meth. A379 (1996) 74; J. Y. Oyang, SLAC-BABAR- NOTE 137 (1994). [138] G. Eigen, D. G. Hitlin, and J. Y. Oyang, in Proceedings of the Workshop on Scintillating Fiber Detectors (SCIFI 93), (1993) 590.

[139] The Calorimeter Calibration Task Force, “Use of Radioactive Photon Sources with the BABAR Electromagnetic Calorimeters”, SLAC-BABAR-NOTE 322 (1996); M. Weaver, “A Circulating Calibration Source for the CsI Calorimeter”, SLAC-BABAR-NOTE 288. [140] D. Hitlin and G. Eigen, in Heavy Scintillators, Editions Frontieres, ed. F. Nataristefani et al.,p. 467 (1992); Z.Y. Wei and R.Y. Zhu, Nucl. Instr. and Meth. A326 508 (1993). BIBLIOGRAPHY 153

[141] D. R. Quarrie and F. C. Porter, “An Application Framework and Data Model Prototype for the BABAR Experiment”, in Proceedings of the CHEP 96 conference, Rio de Janiero, 1996.

[142] R. Aleksan et al. [TOPAZ White Paper Committee Collaboration], “TOPAZ magnet evaluation committee: Report,” SLAC-BABAR-NOTE-159.

[143] T. Glanzman et al., “SCS support of the B factory,” SLAC-BABAR-NOTE-145.

[144] D. Boutigny et al. [BABAR Collaboration], “Letter of intent for the study of CP violation and heavy ﬂavor physics at PEP-II,” SLAC-0443.

[145] F. Porter and A. Snyder, “The CP sensitivity table in the status report,” SLAC-BABAR-NOTE- 140.

[146] D. Aston et al., “B Factory computing decisions to be made,” SLAC-BABAR-NOTE-115.

[147] F. Porter, “Variation of PEP-II lifetimes with vacuum,” SLAC-PEP-II-AP-NOTE-36-93.

[148] R. Becker-Szendy, D. Briggs and F. Porter, “Level 1 trigger rate estimates,” SLAC-BABAR- NOTE-112.

[149] F. Porter, “BABAR event size estimate,” SLAC-BABAR-NOTE-108.

[150] F. Porter, “B factory computing issues,” In Proceedings of B Factories: the State of the Art in Accelerators, Detectors, and Physics, Stanford, CA, 6-10 Apr 1992 SLAC-400.

[151] A. Breakstone et al., “Computing for a B factory,” SLAC-BABAR-NOTE-082.

[152] D. Hitlin et al., “A Comparison of CP violation measurement programs at e+e− and hadronic accelerators,” SLAC-BABAR-NOTE-081.

[153] M. Dickopp, S. Gowdy, M. Kelsey, P. Raines, A. Romosan and A. Ryd, “Elephant, meet Penguin: Bringing up Linux in BaBar,” Published in CHEP 2000, Computing in High Energy and Nuclear Physics, 406-9, Padova, IT, 7-11 February 2000.

[154] D. Quarrie et al., “Operational experience with the BaBar database,” Published in CHEP 2000, Computing in High Energy and Nuclear Physics, 398-401, Padova, IT, 7-11 February 2000.

[155] G. Zioulas, Y. Kolomensky, S. Metzler and V. Miftakov, “The BaBar online databases,” Published in CHEP 2000, Computing in High Energy and Nuclear Physics, 242-5, Padova, IT, 7-11 February 2000.

[156] G. P. Dubois-Felsmann, E. Chen, Y. Kolomensky, S. Metzler, A. Samuel and S. Yang, “Flexible processing framework for online event data and software triggering,” IEEE Trans. Nucl. Sci. 47, 353 (2000).

[157] BABAR Technical Design Report, SLAC-R-95-457 (1995).

[158] J.M. Flynn, In Proceedings of the 28th international conference on high-energy physics, Warsaw, Poland, 25-31 July 1996, World Scientiﬁc, Singapore (1997).

[159] M. Neubert, Phys. Rev. D45, 2451 (1992).

[160] C.R. Allton et al, Nucl. Phys. B349, 598 (1991).

[161] C. Alexandrou et al, Phys. Lett. B256, 60 (1991). 154 6 BaBar

[162] G. Burdman, T. Goldman, and D. Wyler, Phys. Rev. D51, 111 (1995). [163] T.E. Browder et al (CLEO collaboration), Phys. Rev. D56, 11 (1997). [164] G.P. Korchemsky, D. Pirjol, and T. Yan, Phys. Rev. D61, 114510 (2000). [165] D.M. Coﬀman, The properties of semileptonic decays of Charmed D Mesons, Ph.D. dissertation, Caltech (1987). [166] Y. Grossman and M. Worah, Phys. Lett. B 395, 241 (1997). [167] Particle Data Group, K. Hagiwara et al.,Phys.Rev.D66, 010001 (2002). [168] BNL-787 Collaboration, S. Adler et al., Phys. Rev. Lett. 84, 3768 (2000). [169] A. Dedes, H. Dreiner, and P. Richardson, hep-ph/0106199, Phys. Rev. D65, 015001 (2002). [170] NuTeV Collaboration, T. Adams et al., hep-ex/0104037, Phys. Rev. Lett. 87, 041801 (2001). [171] K. Agashe, A.G. Deshpande, and G.-H. Wu, hep-ph/0006122, Phys. Lett. B489, 367 (2000). [172] K. Agashe and G.-H. Wu, hep-ph/0010117, Phys. Lett. B498, 230 (2000). [173] H. Davoudiasl, P. Langacker, and M. Perelstein, hep-ph/0201128, Phys. Rev. D65, 105015 (2002).

[174] BABAR Collaboration, B. Aubert et al., “A Search for B+ → Kνν¯ with BABAR,” hep-ex/0207069, submitted to ICHEP2002. − [175] BABAR Collaboration, B. Aubert et al., “A Search for B+ → tau nu recoiling against B → D0 −νX,” hep-ex/0303034, submitted to 2003 Recontres de Moriond. [176] Proceedings of the Workshop on the CKM Unitarity Triangle, CERN, February 13-16, 2002, http://ckm-workshop.web.cern.ch/ckm-workshops/Default2002.htm; M. Battaglia et al., “The CKM matrix and the unitarity triangle,” arXiv:hep-ph/0304132. [177] G. P. Dubois-Felsmann, D. G. Hitlin, F. C. Porter, and G. Eigen, “Sensitivity of CKM ﬁts to theoretical uncertainties and their representation”, submitted to International Conference on High Energy Physics, Amsterdam, 2002, ABS959, CALT 68-2396, June 2002.

[178] The BABAR Statistics Group web page: http://www.slac.stanford.edu/BFROOT/www/Statistics.

[179] The BABAR Statistics Group report: http://www.slac.stanford.edu/BFROOT/www/Statistics/- Report/report.pdf. Also, BaBar Analysis Document 318, Version 2. [180] Physics at a 1036 Asymmetric B Factory. SLAC-PUB-8970, contributed to the APS/DPF/DPB Summer Study on the Future of Particle Physics (Snowmass 2001), Snowmass, Colorado, 30 Jun - 21 Jul 2001. [181] D. Hitlin. Physics at a 1036 e+e− Asymmetric B Factory. Proceedings of the 9th International Symposium on Heavy Flavor Physics, F. Porter and A. Ryd, ed. Pasadena, California, 10-13 Sept. 2001. [182] D. Hitlin. Detector Issues for a 1036 e+e− Asymmetric B Factory. Proceedings of the 8th Interna- tional Conference on Instrumentation for Colliding Beam Physics. Novosibirsk, Russia, 28 Feb.-6 March 2002. [183] D. Hitlin. The Super B Factory. Proceedings of International Symposium on Flavor Physics and CP Violation (FPCP1), Philadelphia, Pennsylvania, 16-18 May 2002. 7. MINOS

B. C. Barish, B. C. Choudhary, J. Hanson, P. Harris, R. Knapp, D. G. Michael, H. Newman, C. W. Peck, P. Randall, C. Smith, E. Tardiﬀ, J. Trevor, H. Zheng

7.1 Overview

With the interests of the reader in mind, this year’s report on MINOS is written as much as possible as incremental to those of previous years. It is assumed that the reader is already familiar with the experiments or has the Caltech report/grant renewal proposal for previous years available. This year, we provide sufficient information to remind the reader of Caltech’s role in MINOS, report on the work in the last year and give an update on our plans for the coming year(s). Caltech work on MACRO is complete. A couple of final papers are still being prepared but no Caltech personnel are now directly involved in that process. Hence, we provide no further update or discussion regarding MACRO. The group’s research effort is now focussed on the MINOS experiment. This experiment is rapidly nearing the end of the construction phase. The far detector is almost done, the near detector is assembled on the surface waiting to go underground and the beamline is in the final phase of civil construction. The main current MINOS activities are:

• Nearing completion of the construction of the far detector.

• Well into construction of the near detector.

• Collecting and analyzing data from the calibration detector.

• Analyzing cosmic ray and atmospheric neutrino data from the far detector.

• Forming analysis groups for the beam data.

• Working to develop the Monte Carlo and Analysis code base.

• Working to complete the NuMI Beamline.

• Working to have the highest possible proton intensity when beam running starts.

The Caltech group is involved in many of these activities. In July 2002, Doug Michael was elected as MINOS Co-Spokesperson, and hence has some interaction with all of these activities. Construction of the scintillator system is complete, including the modules for the far detector which were built at Caltech. The far detector is actively collecting cosmic-ray data for calibration purposes and atmospheric neutrino and anti-neutrino data. Caltech postdocs have continued to contribute to installation work at Soudan. MINOS is the ﬁrst detector capable of identifying whether each event is neutrino or anti-neutrino induced and hence can measure oscillation properties for each independently. In order to permit a clean analysis of such events with the interaction point located within the MINOS detector, we are installing a cosmic-ray veto shield above the detector. At this time, the shield is installed and active on the ﬁrst half of the detector. The remaining shield will be installed on the second half of the detector in June 2003, following completion of the plane construction. Doug Michael has led the

155 156 7 MINOS effort to design the veto shield and collect the necessary approvals and funds for its construction. The Caltech group is starting to work on analysis of atmospheric neutrino events where we plan to focus on reconstruction efficiency issues for contained-vertex events in the short term. Another of the MINOS detectors which has been taking data is the Calibration Detector at the CERN PS. The Caldet has now had two data runs. One in the summer/fall of 2001 and another in summer/fall of 2002. A final run is scheduled for summer/fall of 2003. The data from the calibration detector is now being analyzed and starting to be used to understand the calibration for the Near and Far detectors. Chris Smith, who joined the Caltech group as a postdoc last October, wrote his thesis on MINOS calibration using data from the Calibration detector. He continues his participation in the operation of Caldet and analysis of the data. An important development in the last year has been the formation of physics analysis groups for beam neutrinos. The three groups for the oscillation physics are focussed on measurement of the νµ CC events (νµ disappearance), νe CC events (νe appearance) and NC events (νµ to νsterile). The goal of setting up these analysis groups now is to further develop the necessary tools for the analysis and be ready to rapidly analyze data once the beam turns on. Chris Smith is working on νµ CC analysis efforts and Hai Zheng is working on νe appearance efforts. The Caltech group now has a significant effort aimed towards increasing the proton intensity available for the NuMI beamline. This is the single most important identifiable issue at this time for the future success of MINOS beam measurements. The previously published sensitivities for MINOS assumed two years of running with 4 × 1020 protons per year to the NuMI target. However, the current Fermilab planning for NuMI is aimed to deliver only 2.0 − 2.5 × 1020 protons per year for a total of 4 years of running. Of course, real intensities could be lower for a number of reasons. On the other hand, the technical means of significantly increasing the proton intensity for reasonable investment levels have been identified. A committee co-chaired by Doug Michael and Phil Martin issued a report in August 2002 which outlined a possible path to roughly doubling of the NuMI proton intensity from the nominal plan by 2008. But at essentially all levels of intensity, it is clear that a large and dedicated effort will be required to improve the proton intensity for NuMI/MINOS. The greater the improvement, the greater the required effort. With the completion of the construction of the far detector and the scintillator system, the Caltech group is now making its primary hardware focus the increase in proton intensity. Doing this requires a good deal of development of innovative means of collaborating with the accelerator groups at Fermilab. In order to help keep things moving in a positive direction, we have established a MINOS working group on proton intensity with bi-weekly meetings. Various people involved in work, both from within and outside of Fermilab attend these meetings to discuss work of most relevance to MINOS. Of course, the real work is carried out in collaboration with Fermilab experts on the Booster and Main Injector. Some of the specific efforts that Caltech group members have been involved with over the last year include:

• The joint Fermilab/MINOS report on proton intensity for NuMI, Aug. 02 [1] (Michael).

• Continued direction and deﬁnition of MINOS roles in accelerator work (Michael).

• Director’s Committee on Proton Intensity, report due June 03 (Michael).

• LINAC to Booster steering control (Zheng).

• Prototype solid-state barrier RF driver (Zheng).

• Fabrication of new larger inner-diameter RF cavities for the Booster (Smith, Michael, Hanson). 7.2 Personnel 157

We plan to continue work on these efforts as well as develop new responsibilities as reasonable given other activities. We believe that these efforts on increasing the proton intensity are well aimed at the physics issues which MINOS will start addressing in the next few years. The recent results from Kamland have resulted in a great deal of interest in new sensitivity to a possible small admixture of νµ to νe oscillation with ∆m2 the same as for atmospheric neutrinos. The nominal MINOS proton intensity will extend this sensitivity, but a higher intensity of protons may allow an unambiguous discovery and not just a newer limit. In addition, the fact that the best-fit value for ∆m2 from Super-Kamiokande is now as low as it has ever been (2.5×10−3 eV2) means that in order to unambiguously measure the oscillation signature, MINOS will need at least the nominal number of protons and more. Neutrino oscillations remain an experimentally driven subject where we are never quite sure what to expect next. We anticipate that the results from MINOS, particularly with higher proton intensity, will be very exciting.

7.2 Personnel

The MINOS personnel are currently mostly Ph.D. physicists. In the last year, Brajesh Choudhary left to take a job in the Main Injector group at Fermilab. We hired two new postdocs to replace Brajesh and the combination of Hwi Kim and Bob Nolty (in this case one postdoc replacing a postdoc and a student due to the budget reduction). We are working at adding graduate students but in the last year have had one student who left Caltech for health reasons and a second who decided to go into a different field of physics. We anticipate adding one or two graduate students next academic year. We generally have one or two undergraduates working part-time on research activities within the group. The specific group members over the last year and involvements are:

• Barry Barish: Professor. U.S. MACRO spokesman. Director of LIGO. Co-Chair of the HEPAP subpanel study on the future of particle physics and Chair of the NAS study on future large-scale neutrino projects. Most of his time is spent on LIGO (and this will continue for the forseeable future).

• Charles Peck: Professor. Provides senior supervision and direction for the MINOS group.

• Harvey Newman: Professor. U.S. CMS Spokesperson. Provides long-term professorial leadership for the group and is thesis advisor for the MINOS students. The rest of his activities are described in the L3/CMS chapter.

• Doug Michael: Senior Research Associate. In summer of 2002 was elected as MINOS Co- Spokesperson. Continues to hold the position of Scintillator System Manager for MINOS and Deputy MINOS Manager. Co-Chaired the joint MINOS/Fermilab committee on proton intensity for MINOS. Now serving on a follow-up committee to recommend a specific plan for Fermilab to invest in proton intensity for all physics objectives. Carries out most management responsibilities for the Caltech MINOS group. Efforts on MINOS include leadership on a broad variety of physics and technical issues. Some specific topics in the last year include work towards analysis on atmospheric neutrinos and anti-neutrinos in the far detector, design and construction of a veto shield for the far detector and work on proton intensity issues. In addition to this work on MINOS, spends some small effort on detector development for possible future neutrino detectors (Fermilab Off-Axis experiment, etc). He also has served on several review committees in the last year, including cost and schedule reviews for U.S. CMS, U.S. ATLAS, U.S. Kamland, VERITAS and CDMS II. 158 7 MINOS

• Hai Zheng: Postdoctoral Scholar. Came to Caltech in July 2002 from Notre Dame where she wrote a thesis on a search for large extra dimensions using data from the D0 experiment. She is working on learning about MINOS data analysis (she will focus on νe appearance) and has contributed to work on proton intensity, including work on steering of the Fermilab LINAC beam and development of a solid-state driver for barrier RF cavities. She has also spent some time working to complete a PRL paper on her thesis analysis.

• Chris Smith: Postdoctoral Scholar. Came to Caltech in October 2002 from University College London where he did a thesis on calibration of MINOS using data from the MINOS Calibration Detector. He is continuing work on the Calibration Detector this year while he starts working towards physics analysis development on atmospheric neutrinos and νµ CC events with the beam neutrinos. He is also starting to work on proton intensity issues for MINOS with a ﬁrst project being involvement in the fabrication of prototype new Booster RF cavities.

• Brajesh Choudhary: Senior Postdoctoral Scholar. Departed Caltech for a job in Fermilab Beams Division in August 2002. MINOS level 3 manager for clear and wavelength-shifting ﬁber. Worked on several diﬀerent beam design issues and proton intensity issues, leading to his current position.

• Jason Trevor: Technician. Crew Supervisor for the Caltech module factory. (Salary funded by Fermilab equipment money through June 2003.)

• Eric Tardiﬀ: Former Caltech undergraduate who worked on MINOS for a year prior to going to graduate school in August 2002. He worked on timing calibrations using cosmic-ray muons in the far detector.

• Paige Randall: Graduate Student. Worked on MINOS from the start of her first year (Sep. 2002) until spring of 2003 when she decided to pursue a different field of physics.

• Philip Harris: Caltech undergraduate who is working together with Doug Michael on detector development. He was awarded a Caltech Summer Undergraduate Research Fellowship for this work for summer 2003.

7.3 MINOS Physics

7.3.1 Atmospheric Neutrinos and Anti-Neutrinos

MINOS is the first large underground experiment which has a magnetic field and hence can momentum analyze muons (up to about 100 GeV/c) and measure their charge. This makes it possible for us to cleanly identify whether neutrino interactions resulted from neutrinos or anti-neutrinos and hence whether these oscillate in the same way. MINOS will be able to directly address some models of CPT violation in neutrino mixing within the next couple of years [2]. The MINOS far detector has now been running since August 2002 for collection of atmospheric neutrino data with the magnetic field turned on in the first supermodule. Neutrino induced events can be classified by whether the neutrino interaction is in the detector (contained or contained-vertex events) or whether the neutrino interaction was outside the detector and only an upgoing muon is observed in the detector. The contained events require the cosmic-ray muon veto shield which has now been built over the first supermodule and is in an ongoing operation/commissioning phase (we keep improving its performance). We believe that this shield is now operating sufficiently well to provide a 7.3 MINOS Physics 159

Figure 7.1: The 1/β values reconstructed for all muons observed in MINOS which traverse at least 20 planes of scintillator and other track quality cuts. Downgoing muons are defined to have positive β while upgoing muons have negative β. A clear peak of 12 upgoing muons is visible. clean sample of contained events but we do not yet have an analysis based on these events ready for release. However, for the upgoing muons the background issues are less severe and essentially all data from August 2002 on is useful for analysis. The MINOS upgoing muon analysis is done in a very similar way to the MACRO upgoing muon analysis. A muon track is reconstructed using the spatial information of the strips and then timing from the scintillator is used to identify whether a muon is moving up or down through the detector. All muons which are moving up through the detector have been generated by neutrino interactions rather than punch-through cosmic-ray muons from above (this has been demonstrated by showing where these events cutoff at Soudan). In MINOS, the time resolution per scintillator plane is about 2.5 ns compared to about 0.5 ns for MACRO. Hence, instead of just two or three planes of scintillator used in MACRO, in MINOS it is necessary to use roughly 20 planes of scintillator to get a reliable determination of the direction of a muon. Of course, MINOS has many more planes of scintillator so this presents no significant problem. Figure 7.1 shows the 1/β values reconstructed for all muons observed in MINOS which traverse at least 20 planes of scintillator (along with a few other quality cuts such as requiring non-showering events and good track reconstruction). Downgoing muons are defined to have positive β while upgoing muons have negative β. In this analysis, there are roughly 1.2 million downgoing muons with a 1/β 160 7 MINOS

Figure 7.2: The number of events versus the log10 of the measured momentum of the upgoing muons. The events are separated into those with negative and positive charge muons. peak centered at 1.0 and with an RMS of 0.056. At 1/β of -1.0, there is a clear peak of upgoing muons. There are a total of 12 upgoing muons with −1.2 < 1/β < −0.8. Figure 7.2 shows histograms of the number of events versus the log10 of the measured momentum and separated into the positive and negative charge muons. We note that this data is based on only about 4 months of total livetime with only 1/2 the ﬁnal full detector. It is clear that within the next year these results from MINOS will start to become quite interesting for the comparison of the neutrinos and anti-neutrinos.

7.3.2 Event Reconstruction Issues for Contained Events

The use of low energy contained events for atmospheric neutrino analyses requires relatively sophisticated ability to reject backgrounds induced by downgoing cosmic-ray muons. The cosmic-ray veto shield is certainly a part of what is required, but in addition improved reconstruction algorithms are also essential. The main backgrounds are induced when highly angled cosmic ray muons enter the detector in the air gaps between the planes producing topologies much like a neutrino events. In order to provide a clean sample of “contained vertex” events, it is particularly important that the reconstruction algorithms are able to distinguish downgoing stopping muons from true neutrino interactions. The Caltech group is planning work on this part of the atmospheric neutrino analysis. One of the goals of any atmospheric neutrino oscillation analysis is to extract as much information as possible from the observed track and from the veto shield hits in order to produce a clean sample. For example, to isolate a partially contained atmospheric neutrino sample, the timing of the veto shield hits may prove useful for determining the muon direction. With these goals in mind, Chris Smith has designed a user-friendly event display to help understand the typical topologies of neutrino events in the MINOS detectors. For Monte Carlo data, the truth digits as well as the reconstructed hits (after demultiplexing) are displayed. Also for real data, veto shield hits are shown. The reconstructed tracks or showers are overlaid and some track information is shown. The display has been used to study the general performance of the event reconstruction, and ultimately will be a useful tool when trying to deﬁne a set of variables for discriminating neutrino events from the cosmic ray background. Figure 7.3 is a screenshot from one of the display windows showing the truth digitizations and some information about the the event. Figure 7.4 shows the reconstruction 7.3 MINOS Physics 161

Figure 7.3: A screenshot of the event display. This window displays the MC truth digits and displays some event information. Note: This is a color ﬁgure that is diﬃcult to interpret in just black and white.

for this event. This event is one of an interesting class of events for which the direction of the muon is diﬃcult to determine purely from the response in the detector. The presence of one or two veto shield hits may provide enough extra time information to allow more events of this type to be included in the analyses. So far, the event display has been limited to the Far Detector. Eﬀorts at Caltech to start running the reconstruction packages on Near Detector Monte Carlo data have meant that the display will require an upgrade to cope with the increased numbers of tracks and showers produced by multiple neutrino events during a spill. This work will take place over the coming few months. The work on the event display was undertaken partly to become familiar with the output from the MINOS “Standard Reconstruction” packages. Looking to the future, the intention is to produce an analysis suite which, in the context of atmospheric neutrinos, can be used to isolate a clean sample of partially contained events. 162 7 MINOS

X vs Z view Y vs X view Y vs Z view

4.5 5 3 4 4 3 2.5 3.5 2 y position (m) y position (m) x position (m) 1 2 3 0 1.5 2.5 -1 -2 2 -3 1 1.5 -4 0.5 -5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 -5-4-3-2-1012345 56789101112 z position (m) x position (m) z position (m)

Time vs Y view Transverse vs Z view - U Planes Time vs Z view -7 -7 x10 4 x10 -0.35 -0.35 3.5

3 time (s) time (s)

-0.4 transverse position (m) 2.5 -0.4

2 5678910 -0.45 z position (m) -0.45

Transverse vs Z view - V Planes -0.5 0.5 -0.5 0

-0.5 -0.55 -1 -0.55 -1.5 transverse position (m)

-2

1 1.2 1.4 1.6 1.8 -2.5 7 7.27.47.67.8 8 8.28.48.68.8 5 6 7 8 9 10 11 12 y position (m) z position (m) z position (m)

Figure 7.4: A screenshot of the event display. This window displays the reconstructed digits and shows those associated with tracks and showers. Note: This is a color ﬁgure that is diﬃcult to interpret in just black and white.

7.3.3 νµ Charged-Current Measurements

Overview

One of the main goals of MINOS is to clearly confirm the oscillation mechanism from the energy distribution of the νµ beam and to measure the associated oscillation parameters. The most sensitive measurements of the oscillation parameters can be obtained using the “T-test” (looking at the ratio of ratios: (NC/CC)near/(NC/CC)far) or by comparing the νµ CC energy spectrum at the Far detector with a prediction based on the spectrum at the Near detector. The T-test removes many of the uncertainties associated with extrapolating measurements from the Near detector to the Far detector, by looking at a ratio of ratios. However, the statistical errors on the measurement are dominated by the number of reconstructed NC events in the sample. By contrast, comparing the νµ CC energy spectra provides a statistically more accurate measurement of the oscillation parameters but is limited by systematic errors in the predicted neutrino energy spectrum. For the low energy beam (best suited to ∆m2 in the expected range), we expect that statistical errors will be dominant. Understanding the systematic and statistical errors associated with a νµ CC measurement 7.3 MINOS Physics 163 is an important topic and is currently being studied within the collaboration. Other issues for this analysis are with the reconstruction efficiencies of NC and CC events. At low energies, NC events present a significant background to the νµ CC and so the probability of misidenti- fying a NC event to be a CC event as a function of neutrino energy needs to be understood. Data from the calibration and near detectors will be useful for better understanding this probability.

Work at Caltech

Chris Smith is focussing on νµ CC analysis. He has converted the Fortran code used to make many of published MINOS limit and measurement plots into C/C++ and some extra functionality is planned. The Fortran code uses a predicted νµ CC flux at the Far detector based on a simulation of the NuMI beamline. Events are obtained by generating y-values to get the muon and hadron energy. The events are reconstructed by smearing the true energies with the MINOS energy resolution. Reconstruction efficiencies are folded in as well as contributions from the NC background. Contour plots in the sin2 2θ -∆m2 plane are made by generating one “data” distribution at a particular value of sin2 2θ and ∆m2, and then generating many “MC” distributions over a range of sin2 2θ and ∆m2.Aχ2 value is calculated between the data and each MC distribution which also takes into account the known systematic errors. Confidence limits are then obtained using the usual prescription for the joint estimation of 2 parameters. One improvement that is underway, is to be able to use the MINOS standard reconstruction output to produce the reconstructed νµ CC energy spectrum. To do this, MC events with a reconstructed track above some low energy threshold (to reduce NC contamination) are used and the muon energy is determined either from range or curvature. The summed charge in the track is subtracted from the total charge in the event to get the total shower response. Then using nominal values from the CalDet measurements, this charge is converted into GeV. At this stage, much of the effort ties in with the work on event reconstruction issues for atmospheric neutrinos. One other important question that needs to be addessed is the number of protons on target (POT) required to make a measurement of the oscillation parameters. Figure 7.5 shows the unoscillated and oscillated spectra with 3.7 × 1020 protons on target (about 1-2 years of running), for maximal mixing and ∆m2 =0.0025eV 2. Using the Fortran algorithms for making contour plots, a study is planned to investigate the number of POT required to see a 5 sigma distortion of the νµ CC energy spectrum for different values of ∆m2.

7.3.4 νe Appearance

Hai Zheng is focussing her analysis work on νe appearance. One of the important goals for MINOS is to extend sensitivity to mixing parameter θ13. The goal is to extend the sensitivity by at least a factor of 3 better than the current limit from the CHOOZ [3] experiment. Specifically, the measurement consists of determining the rate for the process νµ → νe. The experimental challenge with this measurement is the determination and subtraction of the background, due principally to νe’s in the beam and misidentified νµ CC and NC events. With the NuMI low energy beam, the typical neutrino energy is sufficiently below the ντ CC reaction threshold that τ → e decay does not contribute to any significant background for νe appearance. Backgrounds will be measured in the Near Detector (located close enough to the neutrino source so that no oscillation has taken place as yet) and then extrapolated to the Far Detector. In the ideal case of a point source and zero degree neutrino beam, the backgrounds would scale linearly with the ratio of the masses of the two detectors and inversely with their distance squared from the source and the scaling would be independent of the neutrino energy. The departures from this ideal case can generally 164 7 MINOS

20 Figure 7.5: The unoscillated and oscillated νµ CC energy spectrum 3.7 × 10 POT (about 1-2 years of running) for maximal mixing and ∆m2 =0.0025eV 2. be handled by appropriate corrections based on Monte Carlo simulations, whose accuracy at some level depends on how well one understands the production spectra and the focusing system. We believe that these will be sufficiently well understood for MINOS that the systematic uncertainty from this source will be relatively small. However, the above picture is made more complicated by the fact that the scaling is more complex for the case of νµ CC events. The νµ flux changes as it propagates through space due to oscillations; thus its contribution to background will be significantly higher at the Near Detector than at the Far Detector. The accuracy of the estimate of the total background at the Far Detector depends on how well one can estimate the relative contribution of the CC process to the Near Detector background. Yet another complication comes from the fact that the near detector has many events in each spill while the far detector has a very low probability of more than one event in a given spill. In previous studies of MINOS sensitivity, a 10% systematic error on background has been assumed[4]. We are undertaking new work to better quantify this uncertainty. Zheng will perform studies on the near/far detector differences: event pileup, multiplexing and other more subtle effects with real reconstruction codes to determine the impact of these differences. An important issue is to address some of the lack of detail in the current near detector simulation. Issues such as PMT cross-talk, details of the geometry, electronics digitization and other readout details need to be fully simulated. The most important issue to address is that of event overlap in the near detector. With the low energy beam, the near detector will have around 30 events in each 11 µs spill. Given that the time 7.3 MINOS Physics 165

stp.tpos:stp.plane {evthdr.snarl==194391}

-1

-2

-3 0 50 100 150 200 250 300

Figure 7.6: Transverse position of the stip vs plane number in a snarl from the MC sample of the νµ in the near detector in the case of low energy beam.Note: This is a color ﬁgure that is diﬃcult to interpret in just black and white.

extent of each event is about 50 ns, there is a non-negligible probability that events will occur essentially simultaneously. Generally, the events can still be separated due to both time and spatial separation, but speciﬁc algorithms for doing this must be developed and the impact on the near/far comparisons understood. In the near detector, a fast clustering algorithm, based primarily on time, scintillator strips hits within the full length of the spill into time slices based on their clustering in time. Fig. 7.6 shows an example of the situation that we expect the near detector. In one spill, there are 22 time-slices and 16 reconstructed events. The plot shows the strip transverse position vs. plane number, black dots are all the hit strips, blue dots are all the recontructed tracks, and the yellow dots are the reconstructed showers. Not all tracks and showers are reconstructed. For plane number larger than 120 (spectrometer region), there is a 4-fold multiplexing of the readout so algorithms must be developed to properly assign muon tracks in this region to the neutrino vertex. Fig. 7.7 shows the time slicing in this spill, black is the time information for all the strips, and diﬀerent colors shows the time slices for the 16 reconstructed events. Here we can also see that some slices are not reconstructed. This is a work in progress and several improvements need to be made in the reconstruction in order to deal with problems shown here. The goal is to limit the systematic uncertainty for the νe appearance analysis. This is uncharted territory but we anticipate that Zheng will make an important contribution to this important topic. 166 7 MINOS

stp.time1*1e9 {evthdr.snarl==194391} htemp Entries 2732 220 Mean 4488 RMS 2401 200 180 160 140 120 100 80 60 40 20 0 0 2000 4000 6000 8000 10000 12000 14000 16000

Figure 7.7: Time slicing (in ns) of the strips in the same sample as shown in Fig. 7.6. Note: This is a color ﬁgure that is diﬃcult to interpret in just black and white.

7.4 Status of MINOS Detector Construction and Operation

The NuMI project and the MINOS experiment are now approaching the end of the construction phase. Installation of the far detector will be complete in June 2003. The ﬁrst supermodule has already been operating with the magnetic ﬁeld on for most of a year, collecting cosmic-ray data for calibration of the detector and atmospheric neutrino data. Chris Smith and Hai Zheng have contributed to installation work at Soudan over the last year, taking several two week shifts at Soudan. Although it has had no direct Caltech group involvement, the preliminary assembly on the surface of the MINOS near detector was carried out during 2002. This involved attaching the scintillator modules to all of the near detector steel planes and hanging them in racks, ready to transfer underground once the MINOS near hall is ready. This is currently anticipated to be starting in March 2004. The reason to do this assembly on the surface is to make it possible to proceed very rapidly with the underground installation so the near detector can be fully commissioned for the planned beam turn-on in December of 2004. The Caltech group has been responsible for production of 1/2 of the scintillator modules for the far detector. Production of these modules began at Caltech in September of 2000 and was completed in April 2003. The production of these modules was on schedule, within the planned budget and the 7.4 Status of MINOS Detector Construction and Operation 167

RelLOCIT ModuleLOCA_py Entries 79840 Mean 1.006 10000 RMS 0.1245

8000

6000

4000

2000

0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Figure 7.8: Relative light output for all strip ends from Caltech modules as measured with the module mapper at the center of the strips. Only 0.16% of strip ends have relative light output less than 0.5. quality has been excellent. A summary of the module quality is given in section 7.4.1. In 2002, it became clear that a cosmic-ray veto shield would be required to reduce backgrounds for contained neutrino interactions to a sufficiently small level to not confuse the atmospheric neutrino analysis. Doug Michael led an effort to design a veto shield based on MINOS scintillator system components. In addition to the design work, he also led the effort to identify funding for the shield and obtain approval for its construction from the Fermilab PAC and management. The first section (out of four total) was installed in summer of 2002. Measurements from that initial installation suggested several improvements in the details of the design. A second section of shield was installed in October 2002, based on some of these improvements and the first section of the shield was reconfigured based on that information. The remaining scintillator system components have now all been fabricated, using up production spares already available or in purchase contracts for the main scintillator system. The entire first supermodule has now run for several months with a complete veto shield. At this writing, a third section of shield is being installed and the fourth (final) section will be installed shortly after the completion of the detector in June. The installation of the shield has been funded by contributions from MINOS collaborating institutions and from a grant from NSF.

7.4.1 Scintillator System Work at Caltech

During the 2.5 year production run, the Caltech factory built 2043 modules for use at the far detector site. Of these 2043 modules, 1960 modules have been designated for installation in the far detector, while the other 83 are for installation in the far detector veto shield. In addition to the modules 168 7 MINOS

RelLOCIT ModuleLOCA Entries 79840 2 Mean x 1075 Mean y 1.006 1.8 RMS x 566.7 RMS y 0.124 1.6

1.4

1.2

0.8

0.6

0.4

0.2

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Figure 7.9: Relative light output for all strip ends from Caltech modules versus the module serial number. Some systematic variation is apparent. This is mostly due to variation in light output of the scintillator. produced for use at the far detector, 66 modules for installation in the calibration detector were also produced at the Caltech factory. Modules produced in the Caltech factory were generally of excellent quality and exceeded design requirements in a number of areas. For example, the average light output of the modules exceeded the design requirement by roughly a factor of 2. Another important design requirement was the number of allowed dead channels/fibers. The dead channel rate for Caltech modules was .0016, which is six times lower than required. The overall uniformity of the light output from the Caltech modules was also excellent. (Note that the production at the factory at the University of Minnesota was also of excellent quality.) Figure 7.8 shows the relative light output measured at the center of all scintillator strips in Caltech modules. The RMS of this distribution is 12.5% over a total of 80,000 strips. Figure 7.9 shows the same data but plotted in a scatter plot versus the serial number of the module (there is an approximate one-to-one correlation between the serial number and date of production). It is interesting to note that the light output showed some distinct systematic variations through the 2.5 years of production. Most of these trends were due to variation in light output of the scintillator rather than changes in the production process. It is interesting to note that all other variations in the process were relatively small compared to that one. The quality of this work reflects the excellent efforts by both the permanent and temporary Caltech staff who were involved in the production. Once again, John Hanson has demonstrated his considerable skills in construction of detector components and Jason Trevor (unfortunately not a part of our permanent staff) demonstrated both excellent technical and management skills in keeping the production on 7.4 Status of MINOS Detector Construction and Operation 169 track. We have now begun the process of partially decommissioning the Caltech module factory. Nonessen- tial equipment is in the process of being dismantled. Other equipment more integral to module production will be left in functional condition and stored on the first floor of the Lauritsen building in the short term, for possible use in future development of solid scintillator technology. This may be of interest to a new off-axis experiment in the NuMI beamline or other possible applications.

7.4.2 The MINOS Calibration Detector

MINOS has been designed to achieve a 2% relative energy calibration between the Near and Far Detec- tors and a 5% absolute energy calibration. The calibration strategy has three main features. A Light Injection system accounts for short term gain drifts in the phototubes and non-linearities in the read-out chain. Cosmic ray muons are then used to normalise the response of all strip-ends in the detector to account for differences in scintillator light output, light collection efficiency in the wavelength shifting fibre (WLS) and connector efficiency. Finally, measurements from the MINOS Calibration Detector (CalDet), which has been exposed to a number of test-beams at CERN, is used to express energy measurements of electrons and hadrons in terms of GeV. Chris Smith has been present at CERN during all of the CalDet test-beam running and, since joining Caltech, has continued to work on the strip-end calibration of the CalDet using cosmic ray muons. To correctly perform a strip-end calibration, a good understanding of light output variations not related to fundamental properties of the scintillator strip or the light transmission path is required. This entails understanding light output variations with position in the detector (related to signal attenuation in and collection efficiency of the WLS fibre), muon path-length through a strip and scintillator temperature. These variations are being measured at CalDet where empirical corrections are developed to remove the dependencies. The corrected cosmic ray muon strip-end energy spectra are then characterized by taking a truncated mean or by performing a fit to get the position of the peak. These values are used to remove any intrinsic variations in the strip-end responses. Measurements of muons, electrons and hadrons from the test-beam can then be compared irrespective of the position or time of the event in the detector. Future work on the CalDet will include improvements to the current strip-end calibration procedure as well as an effort to study the response of stopping muons from the test-beams. These muons will eventually be used to provide another vital link in the MINOS calibration chain: In order to use the response measurements of electrons and hadrons from the CalDet at the Near and Far Detectors, an energy unit must be chosen that is independent of the detector location. For example, the average response of cosmic ray muons is an invalid choice due to differences in the cosmic ray flux at the three detectors. Instead, the energy loss of a muon at a specific energy can be used, making beam-related stopping muons the ideal candidates for this calibration step at CalDet. Figure 7.10 shows the dE/dx for 1.8 GeV stopping muons in CalDet with and without calibration. The rise in the dE/dx beyond about plane 45 is due to the slowing of the muons. It can be seen that the muon calibration still requires some work to correct the outliers in the distribution. Recent work carried out on this topic has been to make improvements to a tracking package for cosmic ray or beam muons. The package returns a ROOT file containing track and hit information for track-like events and can be used for generating calibration constants or for performing offline analyses. Finally, participation in the test-beam runs during summer 2003 is also planned. In previous years, Far Detector electronics and PMTs have predominantly been used to read-out the CalDet. This year, similar measurements will be made using the Near Detector electronics and PMTs in order to investigate possible systematic differences in the read-out. 170 7 MINOS

dEdx for 1.8 GeV stopping muons in CalDet

2 ADCs (Data 1.8GeV) 1.8 PEs (Data 1.8GeV) 1.6 MIPs (Data 1.8GeV) 1.4 1.2 Normalised Response 1 0.8 0.6 0 1020304050 Plane Number

Figure 7.10: dEdx for 1.8GeV muons in CalDet. The black points show the dEdx without any calibration. The red points are after gain variations have been removed, and the blue points are after the muon calibration. The rise in the dEdx beyond about plane 45 is due to the slowing of the muons. It can be seen that there are still some outliers after the muon calibration; work is underway to correct for these anomalies.

7.5 Proton Intensity through the Main Injector

A major issue which MINOS faces over the next few years is proton intensity to the NuMI target. At present, the Main Injector is used in “single-batch” mode (a single fill of protons from the 8 GeV Booster) for anti-proton production. For MINOS running, we must add five more batches per Main Injector cycle which will be extracted to the NuMI beamline. Furthermore, in order to reach the design intensity for NuMI (4 × 1013 protons every 1.9s) it will be necessary to have more protons in each of the batches which is about 9 times more protons accelerated per Main Injector cycle than the current running. It is not a surprise that to accomplish this there are many technical issues which must be surmounted over the next few years. However, most Fermilab accelerator personnel remain highly focussed on issues and needs for increasing luminosity in the collider. Although this work has some overlap with that needed for high intensity running for NuMI, most of the work for NuMI is in addition to what will be done for the collider. In the last couple of years, Doug Michael has been leading an effort to define actions to take to increase the proton intensity and get new people involved in the necessary work. Last year, he co-chaired a committee to study the issue of proton intensity for NuMI. 7.5 Proton Intensity through the Main Injector 171

NuMI Beamline

Figure 7.11: The accelerator complex at Fermilab. Machines of importance to MINOS are the Linac, Booster and Main Injector. For MINOS, the Main Injector is ﬁlled with at least 6 “batches” of protons from the Booster at 8 GeV and the accelerated to 120 GeV. For MINOS operation, we nominally expect 4 × 1013 protons to be accelerated in each Main Injector cycle of 1.9 seconds. Currently, it is possible to deliver 120 GeV protons at only about 1/3 the nominal rate. Many technical issues must be addressed to reach the full design intensity.

This year, he is leading the eﬀort to get MINOS physicists and institutions involved in this work and also contributing to a broader study on proton intensity for all of the Fermilab physics programs (this committee is chaired by Dave Finley from Fermilab). Figure 7.11 shows the Fermilab accelerator complex. The machines which are relevant to proton acceleration for MINOS are the Linac, Booster and Main Injector. For MINOS running, the nominal operating scenario is the following:

1. Protons are accelerated to 400 MeV in the Linac and injected into the Booster. Injection to the Booster lasts for as long as necessary to “fill” the RF buckets to capacity. Typically this takes 10-20 “turns” of the Booster. (A “turn” is the time it takes for protons to make one revolution in the Booster, ≈ 2µs.) The protons are then accelerated to 8 GeV and extracted to the Main Injector. This process is repeated a total of six times for MINOS running. Each Booster cycle takes 67 ms so the six “batches” takes a total of 400 ms. At present, the Booster has a capacity of 4.5 × 1012 protons. 2. Protons are injected into the Main Injector at 8 GeV. The circumference of the Main Injector is slightly larger than 6 times that of the Booster. This allows six batches of protons from the Booster to be injected. Adding any more than six batches requires new kinds of stacking to be developed which do not yet exist. A recent change to the nominal plan is that two batches of protons will first be “slip-stacked” for the antiproton production for Run II and then the remaining five batches for NuMI will be injected. The Main Injector then accelerates the protons to 120 GeV. Putting six batches of protons into the Main Injector rather than just one batch (which is 172 7 MINOS

the case when just antiproton production is being done) puts extra loading on the RF and induces extra beam stability issues as the total charge is increased. These issues currently limit the total number of protons in the Main Injector in “multi-batch” operation to ≈ 1.5 × 1013, less than half that necessary to meet NuMI design parameters.

In order to meet the NuMI design parameters of 4 × 1013 protons per cycle and corresponding 3.8 × 1020 protons per year delivered to the NuMI target it is necessary to make improvements in both the Booster and Main Injector. We anticipate that several members of the Caltech group will contribute to this work over the next few years and we have started to build some speciﬁc involvements over the last year. The speciﬁc activities which we have undertaken thus far are:

• Linac Beam Steering: This was a get-started project for Hai Zheng to help assure a stable beam position injecting into the Booster from the LINAC and allow her to learn various issues about the accelerator control system.

• New Booster RF Cavities: New Booster cavities should permit an increase in the protons per batch in the Booster due to lower proton losses and lower radiation impact of proton losses. Chris Smith and Doug Michael have been working on helping with get the fabrication of two prototype cavities underway, using both Caltech and other university resources to accelerate the time-scale for carrying this work out.

• Barrier RF Cavity Driver: A promising means of increasing proton intensity in the Main Injector is via a technique called “barrier stacking”. This uses relatively high-power, broad-band RF cavities which use new inductive materials and solid-state switching of power. Hai Zheng is working together with Fermilab physicists and engineers on the design, prototype fabrication and testing of a new driver and cavity.

These activities are described in more detail in the next few sections.

7.5.1 LINAC Beam Steering

Joining effort with the Proton Source Department/Beams Division at Fermilab, Hai Zheng has worked on improvements to the Linac steering program (L32 /PA1342 in Fermilab ACNET [5]) and delivered a product with new features. The old Linac steering program works reasonably well; but contains few sensibility checks and only operates upon manual command. It uses the Beam Position Monitors (BPMs) in the 400 MeV line, between the end of the Linac and the Lambertson, to determine the trajectory of the Linac beam, and it uses the dipole trim magnets to correct this trajectory to an operations-defined “zero”. The program relies on the long-term stability of the BPMs, although these have been suspect. The steering program can recalculate its internal calibration, which is implemented as transfer matrixes from the correction elements to each of the BPMs as a function of the fields in the correction elements. Fig. 7.12 shows an example of BPM reading as a function of the dipole setting, the matrix element is the slope of the fitted line. Recalibrating presently requires a few minutes of dedicated Linac study time, but it is hoped that dedicated study will eventually be un-necessary. No smooth variations in the Linac BPMs have been observed, but there have been step changes in them that have led to problems with the steering program that previously went undetected for days. For example, configuration of the cabling to a BPM changed, which changed the position as seen by the control system. It is also possible that a corrector magnet could change its behavior, i.e., by a turn-to-turn short. Both of these problems are, in principle, detectable. 7.5 Proton Intensity through the Main Injector 173

Figure 7.12: BPM reading in mm as a function of the dipole setting in A.

Besides ﬁxing a couple of problems in the previous version of the code, Zheng implemented a detection algorithm which performs a check of each matrix element, based on a small (± 0.5 A) change in each of the correction elements, and then reports the variation in these matrix elements. The program gives the measured and expected positions after a change in the correction elements has been performed. The new code has been delivered and is now running as the standard version at Fermilab. This development serves as the foundation for future improvement such as automatically keeping track of the matrix elements of these transfer matrixes to verify the system integrity. It beneﬁts the accelerator operation in general.

7.5.2 Barrier RF Cavity Driver

In recent years, a new method of doubling the number of protons in the Main Injector has been in development. This method makes use of rf barriers to transfer 12 booster batches from the Fermilab Booster to the Main Injector in 12 consecutive booster cycles, totaling 800 ms. After that, adiabatic capture of the beam into 53 MHz buckets can be accomplished in about 10 ms. Fig. 7.13 [6] shows an example of the simulation result of the stacking process. This is potentially important to MINOS since it could increase the neutrino ﬂux in NuMI by increasing the proton intensity on the pion production target. The MINOS group at Caltech took on the task of designing and building a driver for the Finemet cores. Hai Zheng has been the primary person involved in this work. The goal is to design and construct a power supply for the moving barrier using solid-state switch circuits to provide a pair of pulses (V = ± 6kV,T=0.3µs) with a zero-voltage gap of variable length. For stacking two batches of protons from the booster, the pulses must repeat every 11.1 µs for 200 ms, repeating every 2 s. To stack 174 7 MINOS

Figure 7.13: 13 booster cycles after the injection of the first batch. All particles are within the designated final momentum spread and adiabatic capture can begin. The moving barrier, illustrated as an unfilled box with an arrow, has a width of T1 =1.0µs and a strength of V = 3.1421 kV.

twelve batches the process must continue for 900 ms, significantly increasing the duty factor for the driving device. Using a SPICE simulation, we came up with the first rough design. The circuit layout is shown in Fig. 7.14. The voltage over the equivalent load from the simulation (with a period of 2 µs for demonstration purpose) is shown in Fig. 7.15. The necessary components have been identified and purchased. We are now in the phase of assembling and testing the driver at Fermilab. In one of our initial tests, we used a switch with different parameters for just one polarity of pulse. The test circuit is shown in Fig. 7.16, it was operated in the burst mode with a repetition rate of 12 Hz on a Finemet cavity. The primary (blue) and secondary (pink) voltage and total current (yellow) waveforms are shown in Fig. 7.17. As we can see, the secondary voltage reached 4 kV, and besides some overshoot and noise, the primary and secondary voltage waveform is close to the desired square wave. The next step will be tuning the circuit to have a better waveform. Once we receive more components, we will build and test the more complete circuit. We expect to test this sometime during the summer of 2003 and we are hoping to make a test in the Main Injector sometime in late 2003.

7.5.3 Booster RF Cavity Upgrade

The maximum repetition rate for the Booster is 15Hz which is fixed by the maximum cycle time of the magnets. Recent running has only been between 3.5-7.5Hz, (a recent upgrade of the septum magnets has made a repetition rate of 7.5Hz possible), however, even if it were possible to run at the maximum rate with the current components, it would lead to unacceptable proton losses in the Booster tunnel. Acceleration in the Booster is accomplished using 18 RF cavities each with an aperture of 2.25 inches. The cavities are run at approximately 53MHz and the cavity frequency is controlled using three ferrite loaded variable inductors (tuners). The high power is provided by 150 kW power amplifiers. Figure 7.18 shows one of these cavities. The power amplifier is mounted on top of the cavity and the 7.5 Proton Intensity through the Main Injector 175

A B C D E F G H 3V1 6000

L2 10u R9 S2 S1 D6 40 D5 VAm1 R2 1619 125.74m 6 15 R3 C4 D1 D2 40 300 70p 13 5

D3 D4 R6

1000 IVm1

11 20 R1 L1 0 C10.0 C3 1408 C2 616u 34p 3300pD11 D10 400p

D12 D8 12 7 14 L3 10u R7 S4 S3 R8 D7 D9 40 40 9 10 8 R10 C5 300 R5 R4 70p 1K 1K 18 17 0 0 V2 V3

0 0

Figure 7.14: Circuit layout for the driver.

three tuners are mounted at either side and underneath (one is clearly visible on the left side of the photo). In order to meet the needs of the upcoming experiments at Fermilab, MINOS included, an upgrade to the present Booster cavities may be necessary. The Booster RF cavities are the smallest apertures in the machine. Consequently, the cavities and power supplies become activated (typically ∼50 mrem/hr at 1 ft) even when running at the modest Booster repetition rates described above. This situation is compounded by the power supplies being one of the highest maintenence components in the machine. Before any maintenance can be performed, time must be allowed for the radiation levels to drop to an acceptable level. This has therefore become one of the major causes of Booster downtime. Even worse (for MINOS needs), until the proton losses can be better controlled it will be impossible to significantly increase the number of protons accelerated through the Booster. In view of these limitations, a project is underway to fabricate and test the performance of an upgraded RF cavity. The new cavity design was originally proposed as part of the new Proton Driver at Fermilab [7]. The main change is that the cavity has a larger beam pipe aperture of ∼5 inches. This is expected to increase the proton intensity in three ways: Downtime due to component activation will be reduced, leading to a few percent increase in proton intensity. Beam losses associated with the smaller aperture will be reduced. Finally, higher intensity running will be possible without incurring unreasonable losses. The project is currently in the process of fabricating two prototype 5 inch aperture cavities. The short term goal of this group is to install the cavities into the Booster during the summer 2003 shutdown and to investigate the effect on the proton losses. In order to meet this deadline and to help establish a working relationship with Fermilab, a collaboration between the Fermilab Booster group and a number of MiniBooNE and MINOS universities has formed. The universities, Caltech included, are fabricating a number of the cavity parts and tooling pieces required for the construction and assembly of the prototype cavities at low cost to Fermilab. Michael and Smith have been involved with orchestrating the MINOS universities participation in the project and Smith is planning to become involved with the commissioning and testing of the prototypes both before and after their installation into the Booster. This collaborative effort is hoped to demonstrate that, for the longer term goal of replacing all the cavities in the Booster, university involvement both in terms of intellectual contribution and component fabrication is a reliable and valuable commodity. Furthermore, due to various financial advantages afforded to research efforts in these institutions, university involvement can substantially reduce the 176 7 MINOS

ktests8-Transient-1-Graph Time (s) 0.01.000u 2.000u 3.000u 4.000u 5.000u 6.000u 7.000u 8.000u 9.000u 10.000u

(V)

4.000k

2.000k

0.0

-2.000k

-4.000k

-6.000k

TIME -1.000 V(IVM 1) -1.000 D(TIME) -1.000 D (V(IVM 1))-1.000

Figure 7.15: The voltage over the equivalent load (R1, L1 and C1) from the simulation. The gaps between the positive pulses and the negative pulses are ﬁxed for simpliﬁcation. The switches are controled by two channels of pulser (V2 and V3)

overall cost of the project.

7.6 Summary

The Caltech group has completed its transition of work on MACRO to work on MINOS. Although ﬁnal MACRO publications are still in preparation, essentially no work is now being done on that at Caltech. We have completed our major construction responsibility for the MINOS scintillator system. We have started working on analysis of far detector data with an initial focus on atmospheric neutrinos, beam CC νµ analysis and beam νe appearance. The need for high proton intensity has been accentuated by an 2 expected low value of ∆m , forcing the use of the low energy beam, increased interest in νe appearance and diﬃculties at Fermilab to provide all of the protons needed for the entire program. This has brought the issue of proton intensity to one of very high priority for MINOS. We are responding by taking direct involvement in various aspects of proton intensity work. We anticipate that this will be a very important activity for our group over the next few years. We anticipate that results from MINOS will be of great interest and importance, adding to our understanding of physics beyond the standard model. We expect that the Caltech group will play a central role in producing those results. BIBLIOGRAPHY 177

Figure 7.16: Circuit layout for the initial test performed on the Finemet cavity.

Bibliography

[1] Accelerator Improvement Options for NuMI Proton Intensity, B. Choudhary etal., Aug. 2002, unpublished. Available at http://hep.caltech.edu/ michael/numipiwg/numipiwg.ps.

[2] CPT Violating Neutrinos in the Light of Kamland, G. Barenboim, L. Borrisov, J. Lykken, hep- ph/0212116.

[3] M. Apollonio et al., Phys. Lett. B466 (1999)415

[4] M. Diwan et al., “A study of νµ → νe sensitivity in MINOS”, MINOS Internal Note NuMI-L-714, 2001.

[5] ACNET Design Note 22.28 http://www-bd.fnal.gov/controls/dataserv.txt

[6] K. Y. Ng Phys. Rev. ST AB 5, 061002,(2002)

[7] Proton Driver Design Study, Edited by W. Chou, C. Ankenbrandt, E. Malamud, FERMILAB-TM- 2136, December 2000 178 7 MINOS

Figure 7.17: The primary (blue) and secondary (pink) voltage and total current (yellow) waveform in our initial test of the circuit on the Finemet cavity.

Figure 7.18: A photograph of a present Booster RF Cavity. Part III

Technical Support

179

8. Experimental Computing

J. Barayoga, F. Porter

The experimental particle physics group at Caltech includes approximately 40 physicists (Ph.D.’s and graduate students) at any given time, plus a few visitors. This group has substantial roles in several efforts, often with a very significant computing component. Relevant to the computing requirements, the Caltech HEP physics program has undergone a significant evolution in its program: The BABAR experiment began colliding beam data acquisition in May 1999, and is well into its physics program with very large datasets; the Caltech CLEO effort is winding down, but some anaylsis projects are still underway; the MACRO experiment is completed, but the group is constructing the new MINOS experiment; and the L3 group has established a leadership role in the CMS collaboration. These new efforts have also brought an evolution in the computing requirements, with increased dataset sizes, and vastly increased simulation needs (much of which is planned to be contributed from the universities), due to the increased statistics, and to the higher energies and/or the increased precision demands. Currently, this is dominated by BABAR’s needs, as the major running activity. The software technology employed in these new efforts has also undergone a major evolution, with object-oriented methodology, dependence on commercial software, e.g., for databases, and emphasis on distributed approaches. These factors caused us to assess our computing facilities, and to adopt plans for accommodating the changing physics program. For FY99 and FY00, we thus submitted a proposal for an upgrade in our computing resources, with cost-sharing from Caltech. We have implemented this upgrade, though it would be wrong to regard it as a one-shot transition in an era of rapid advances in computing technology, as well as in requirements. Thus, every year since then, we have made significant additions in capacity, while gradually eliminating dependence on obselete components. For historical perspective, fiscal years 1989-1991 marked a three-year computer acquistion program for the Experimental High Energy Physics group at Caltech. This was jointly funded by the Department of Energy and Caltech, in a 2/3 to 1/3 ratio. Prior to the upgrade, our principal on-site resource was a VAX 11/780 computer. As a result of the upgrade, we acquired a modern workstation-intensive environment, dominated by Unix, but with some remaining VMS capacity as required by existing experiment and transition demands. Ten years later, there were several transitions underway in the experimental activities of the group as already noted, and the computer world had evolved toward increasing complexity. The new experimental activities require considerably more CPU and storage capacity than formerly available. At the same time, costs per performance has come down dramatically, so it has been reasonable to envision an upgrade which meets the new requirements. Much of the increased complexity is due to the explosive growth of the internet. A major consequence is the unfortunate need to devote considerable attention to maintaining a secure system. An additional complexity factor is the need to maintain two (or more) significant “systems” – a Unix farm for data reduction and simulation, and Microsoft servers and seats in offices for commercial applications software. We thus proposed to embark on a new upgrade effort, with cost sharing again with Caltech. Because we have had some equipment support over the past decade, not all of our facilities were obselete, and the request was approximately half the scale of the previous one a decade ago. On the other hand, this request was necessary because funding pressures in previous years did not enable us to maintain adequate upgrades on a continuous basis. We hope to be able to preserve adequate base funding to avoid falling into this pattern again. In a sense, this is what computer “maintenance” has evolved into – the industry advances so rapidly, as well as requirements, that repairs are generally unnecessary, at the

181 182 8 Experimental Computing same time a program of continued “upgrades” is critical. So far, this has been possible, and has been working pretty well. The principle resource limitation we are currently facing is in personnel, rather than in equipment. We are running a signiﬁcant, complex computing installation with one professional system administrator plus occasional student and scientist assistance.

8.1 Computing Facility

Figure 8.1 shows the network-view of the layout of the workstations, personal computers, and servers with their network connections.

Figure 8.1: Local Area Network Layout. Rectangular boxes represent one or more computer nodes; disk names and other services are listed below computers. 8.1 Computing Facility 183

8.1.1 CPU resources

The CPU resources can be broken up into four general areas:

1. A Linux farm on Intel-based computers. This system is now our primary computing resource, and is used heavily for major production, especially for BABAR simulation. We recently migrated from the DQS batch system to the PBS system, for management of the batch environment. The farm currently consists of 41 dual-CPU rack-mounted computers. Each farm machine has 1 GB of memory. We also have a four dual-CPU compute farm for interactive Linux applications. All of the Linux computers run a RedHat Linux operating system. 2. An IBM RS/6000-based Unix system. Before the purchase of the Linux computers, this was our main computing resource. The process of moving services off of the RS/6000s and onto Linux servers has been completed. The remaining IBM machines are now used only for interactive activities. The main CPU resource for analysis among the IBM machines currently consists of six model 43P (hep211-216) workstations. All of our model 320/320H machines (described in earlier reports) are now decommissioned, and we no longer use any X-terminals. 3. Two Windows NT Servers (heppdc, for “HEP Primary Domain Controller”, and baryon1/hepbdc, for “HEP Backup Domain Controller”). This provides filesystem and print service support for the Windows 98/NT/2000/XP computers used by individuals. We have considered, but have not yet implemented, a server to make Windows applications available to Unix users, using a combination of Windows Terminal Server and Citrix thin clients. A Samba server makes the Unix filesystem visible to NT domain users. We have an NT workstation with DLT-7000 and CD-RW burner services, to satisfy in particular a need for user-backups. We are currently implementing an augmented back-up system which will enahance the Windows services. 4. Intel computers on desktops. These run either Windows NT (with migration to XP on new purchases) or RedHat Linux, depending on the preference of the user.

8.1.2 Peripherals

The peripheral resources are summarized below. The disk space is NFS (Network File System) mounted. In addition, we run a freeware version of the AFS (Andrew File System) distributed ﬁle system (client and server) on Linux.

1. Disk space: Prior to our transition to Linux servers, the principal disk space consisted of collections of SCSI disks, amounting to over 100 100 GByte of formatted disk space connected to the Unix RS/6000 servers. This facility has been superceded by the installation of three Linux based RAID ﬁle servers using ATA/IDE disk drives (proton, with 1.2 TByte; neutron, with 2.2 TByte; xeon, with 1.1 TByte; and neutrino, with 0.98 TByte). We have recently purchased a new ﬁle server as a replacement to the problematic (undependable for critical use) proton server. The new server has a cpacity of 2.2 TByte. The proton server will be recommissioned as a repository for project data that doesn’t require highest availability. 2. Tape drives: We use tape both for experiment data storage and for disk backups. The formats include: (i) Formerly, the principal tape format which we used for data was 8 mm (helical scan), and we still have this capability, though it is now little used. (ii) We have a small capacity for 4 mm DAT, via one drive purchased by CLEO. (iii) The DLT format is now the dominant format for us, having become established as a major player in “inexpensive” tape formats. We have DLT drives available on both our Linux and our NT systems. We also have a DLT tape library system connected to one of our servers (quark). We are in the process of installing a second library. 184 8 Experimental Computing

3. Printers: We continue to rely mostly on laser printers running the PostScript page description language. We have two principal black&white printers (hp2 and hp3), and two color printers (hp2c and hp3c). The hp3c printer is a new purchase, replacing an aging Tektronix color printer. 4. VGA projectors: Meetings frequently include people at multiple sites. These meetings, as well as seminars and local meetings, often rely on electronic copies of presentation material, especially graphical representations of data. To support this, we purchased two VGA LCD projectors, one for our seminar room, and the other for our video-conference meeting room. These have proved to be a great communication aid. We encountered some conﬂicts when people borrowed the projector in the meeting room for use in other rooms. We therefore acquired a third, readily portable projector for migratory use. These projectors were pressed into service for the Heavy Flavors 9 and Calorimetry 2002 conferences held at Caltech. 5. UPS: As we have become more critically dependent on our computing and network connections, it has become more important to protect the equipment and maintain a high level of availability for certain services. We thus have invested in a small UPS (Uninterruptible Power Supply), an APC Smart UPS 1400R. Two RAID ﬁle servers are powered by this UPS, in order to prevent accidental loss of data due to intermittent power outages.

8.1.3 Networks

Networking is a crucial part of our effective computing in HEP. Roughly speaking, there are two aspects here: “Local Area” networks, carrying data between local computers at relatively high speed, and “Wide Area” networks, carrying data between computers at different sites at typically slower speeds. We briefly discuss both of these below.

Local Area Networks

With the advent of “CITNET 2000”, funded by Caltech, our networking underwent a significant upgrade four years ago. All new equipment is now connected to one of two switched 10/100BaseT ethernet subnets. The older equipment has also been moved over to these subnets. The distinction between the two subnets is the following: One is our “managed subnet”, such that all nodes connected to this subnet are under the central administration of our system manager. The other is an “unmanaged subnet”, which is available for people to connect private devices (e.g., laptop computers), and is shared with other users in the Downs-Lauritsen building. Both of these subnets provide DHCP service. The IBM token ring (16 Mbps) network, which we heavily relied on during the 1990’s has been decomissioned. We continue to operate some network equipment of our own: Most significantly, a Cisco 7206 router and a Cisco Catalyst 2948G Gigabit Switch which is used to provide a high bandwidth path between our RAID file servers and the Linux compute farm. By using TCP/IP, the local networks are transparently connected for the user, and disk space and printers are available among them. As mentioned above, the disk drives are mounted on different computers using NFS. Two years ago we added wireless networking capability to the Lauritsen laboratory, with the installation of four IEEE 802.11b Orinoco Access Points. As SLAC and other places have implemented wireless networking, a growing number of our users have acquired wireless cards for their laptop computers. The addition of the capability to communicate with these cards in Lauritsen is a more than a major new convenience, it very rapidly evolved into a necessity! New laptop computers typically have wireless capability built in. 8.2 Caltech Support for HEP Computing 185

Wide Area Networks

It is crucial in HEP to have high-speed connections among HEP sites. Until fairly recently at Caltech, the major HEP connection had been an ESnet link, which we used for TCP/IP connections to other sites. However, once we started to use the network to transfer large quantities of BABAR simulations to SLAC, the ESnet route posed performance problems. Fortunately, we were able to switch the routing to use the CALREN-2 connection at Caltech (to Stanford, and thence to SLAC). Clearly, the advent of CalREN-2, with multiple OC-12 connections to Caltech, is of great importance to us. CalREN-2 is supported by Caltech and other California research universities. Access to these lines is supported by Caltech’s ITS (Information Technology Services). The data transfer demands on network performance are expected to continue to grow, and we spent some eﬀort implementing an upgrade to use the still experimental NTON (National Transparent Optical Network – http://www.ntonc.org/what is ntonc.htm ) connection between Caltech and SLAC. This involved upgrading some of our networking infrastructure in Lauritsen, including a new Cisco Catalyst 3508G Gigabit Switch. Unfortunately, the NTON experiment is currently “on hold”.

8.2 Caltech Support for HEP Computing

Caltech provides supplementary support for HEP computing in several areas:

8.2.1 Information Technology Services

The Information Technology Services (ITS) organization at Caltech is a group of people which supports many aspects of computing at Caltech. HEP beneﬁts from several of the services provided. ITS negotiates and coordinates many site-wide software license agreements, such as Autocad and Mathemat- ica. This now includes many of the Microsoft products, including their “Visual Studio” programming development environment, besides the common Oﬃce software.

8.2.2 Networking

Caltech has historically provided both hardware and expert personnel assistance for our networking needs, and this continues in the CITNET 2000/CalREN-2 era. ITS provides additional related support in such areas as security. ITS provides regular phone lines of dialup modem service, PPP connections, and ISDN service for people working from home, and have negotiated favorable terms for cable-modem access. ITS also provides VPN service, permitting remote users to appear as if on the Caltech network, which enables access to otherwise protected Caltech resources. This has become more and more important as the security concerns have heightened. For the CITNET 2000 upgrade, ITS extensively rewired Lauritsen Laboratory (HEP) with Category 5 structured wiring. Each oﬃce now has at least two switched 100BaseT connections.

8.2.3 Equipment

Besides the formal cost-sharing investments, Caltech has helped meet continuing computing equipment needs. Computers and Xterminals used in research and by secretaries have sometimes been purchased 186 8 Experimental Computing

with non-DOE funds. A fair number of laptop and personal computers have been purchased with Caltech or personal funding, even though they are used mostly for DOE-research activities.

8.2.4 Center for Advanced Computing Research

The CMS/L3 group continues to beneﬁt from the presence of the Caltech Center for Advanced Com- puting Research (CACR), as has been discussed in previous reports. Recent collaborative eﬀorts have focused especially on “Data Grid” development and prototyping activities.

8.3 Operation and Maintenance

The smooth operation of the diverse Caltech HEP computing resources depends primarily on the talents and dedication of the system manager. Because of the significant number of workstations, he must spend a good deal of time upgrading the operating systems to the current revision levels, and develops tools to help him do this effectively. He also makes sure things are maintained properly, and addresses problems when they arise, either fixing them himself, or contacting vendors. When new equipment is acquired he has the task of getting it installed and working. Typically, there is some new equipment under evaluation for purchase. Usually, our system manager is able to obtain these cost-free for the evalutaion. Because of the high-performance requirements on our compute farm, these evaluations have become crucial. A major, but critical, time-sink involves making sure that the networking, both local and remote-access, is running smoothly. Security is a non-trivial aspect of this. We sometimes hire students part-time to help perform a variety of tasks. An indispensable tool is the PBS “batch” system, recently replacing the older dqs system (first installed on the IBM Unix system) on the Linux compute farm. This has permitted the various experiments to make substantial computing runs, and coexist with each other and with interactive users relatively peacefully. In the past, much of the hardware was covered by maintenance contracts, or was covered under warranty. With the phase-out of critical services running on AIX, we are no longer purchasing maintenance contracts. This cost-savings is permitting us to continue to function on a relatively flat budget. We do not puchase maintenance anymore for disks, which usually have long warranty periods, and tape drives, both of which are relatively inexpensive to simply replace. None of the printers are under maintenance, as we can get help from the Information Technology Services on a time and materials basis for the black&white printers, and from the vendors for the color printers. Our “maintenance” strategy has thus evolved into one of replacement rather than repair. This reflects the relatively low cost per unit, and the rapid obsolesence of equipment presently. This strategy is incorporated into our small operating budget, almost entirely going to support our system manager, and our equipment budget, used to purchase replacements and simultaneous upgrades. Finally, the usefulness of the system has been enhanced by efforts of a general nature that various users have been willing to make. This usually is in the form of porting, or assisting the system manager in porting, various tools of value (e.g., CERNLIB, TEX, myriad X-windows programs, CAD, etc.). A limitation is that our budget so far has not permitted us to purchase some of the nicer commercial software tools, such as for network management, database management, and GUI builders. We resort to freeware when it exists (e.g., GUI tools) and do without when it doesn’t (e.g., database, network management). We have the Objectivity object oriented database product under the BABAR license agreement, but under the terms of the agreement, use is restricted solely to BABAR applications. 8.4 Plans 187

8.4 Plans

While our recent major acquisition and shift in resources is “complete”, we have a continuing need for enhanced capability (i.e., “complete” is a misleading term to apply in this dynamic computer era). We are in the midst of further acquisitions with our equipment budget, and must plan for additional purchases in the future:

1. We have been acquiring file servers at the rate of one per year in recent times, and expect the need for additional such acquisitions to continue at roughly this rate for the foreseeable future. As in the past, the need for increased disk space is continuous, and further file servers will continue to be required to keep up. The more-or-less exponential increase in need may be met by the current more-or-less exponential increase in file server capabilities. In addition, we recently performed a cost-effective upgrade of one of our existing servers to disks of greater capacity (replacing 40 GByte drives with 120 GByte drives).

2. We continue to need to purchase additional compute farm nodes, and are presently evaluating possibilities. We have considered the more expensive, but less power-hungry and more space- eﬃcient blade servers. At the moment, it appears that our next purchase will be 1U dual-CPU servers, and we have a candidate machine under evaluation at this time. We will continue to watch the blade market however. BABAR has made substantial use of the existing nodes, and needs to produce ever more events as the real datasets increase in size. The other experiments are expected to place increased demands on the compute farm as well. The current experience is that we should purchase at least 10 dual- or quad-CPU farm nodes per year. The main limiting factor on this is actually personnel.

3. Our interactive Linux server capability was enhanced two years ago, and we will soon be considering the appropriate route to additional capacity.

4. As noted earlier, we recently replaced one of our color printers. We estimate a need to replace one of our “workhorse” printers approximately every two years. 5. At the moment, our networking is static, other than the requirements of supporting new equipment by purchasing additional ports, primarily for the growing farm.

6. Our “desktop” situation is reasonably stable now, with an ongoing need to purchase a few computers per year, estimated to be approximately eight/year in the long term. There is growing dependence on the alternative of portable laptop computers, which are especially convenient for the more peripatetic users. These are somewhat more expensive, and tend to have a shorter useful lifespan. 188 8 Experimental Computing 9. LHCNET: Wide Area Networking and Collaborative Systems for HENP

D. Adamczyk, J. Bunn, D. Collados, G. Denis, J. Fernandes, P. Galvez, D. Nae, H. Newman, S. Ravot, S. Singh, R. Voicu, K. Wei, Y. Xia

9.1 Executive Summary

During the past 12 months, the Caltech group’s work on the development and deployment of networks and collaborative systems on behalf of HENP has progressed rapidly, as reviewed in this chapter. We have also continued and extended our leadership roles in this field, chairing the ICFA Standing Committee on Inter-regional Connectivity1, founding Internet2’s HENP Working Group2 and working on the Internet2 Applications Strategy Council and End-to-end Performance Initiative, making HENP a focal discipline of Internet2. The view of HENP network requirements is changing year-to-year as the present generation of experiments at SLAC (BaBar), Fermilab Run2 and RHIC have begun to take data, and as the distributed “data challenges” of the LHC experiments have progressed. Applications and system concepts are becoming increasingly demanding in terms of network capacity and the need for reliable high performance end-to-end, notably through the adoption of Grids in HENP and other data-intensive fields. In response to these changes we have developed an updated five year plan for the US-CERN link, summarized in this chapter. The plan is based on the work of the Transatlantic Network (TAN) Committee3, tuned to take into account more recent experience with network line costs, and equipment costs, as well as the network requirements that result from the emerging dynamic view of the Grid systems required to support data analysis by large physics collaborations in the LHC era. To meet the technical challenges, starting in 2002, we entered a new regime of network development, working in concert with the DataTAG project4 across the Atlantic, the team at the STARLIGHT5 international peering point in Chicago, Internet26 and the TeraGrid7 to develop and put in production cost-effective networks using optical wavelengths. In order to exploit networks effectively in the Giga- bit/sec (Gbps) range, we began work with DataTAG, the Internet End-to-end Performance Monitoring (IEPM) group8, the Internet2 End-to-end Initiative9 and the HENP Working Group, to develop TCP protocol settings and modifications capable of reaching the required throughput across long range networks. Some of the leading edge protocol developments and tests of high performance are being carried

1See http://cern.ch/icfa-scic for the SCIC reports and presentations to ICFA, February 2003. 2Co-chaired with S. McKee of Michigan, see http://www.internet2.edu/henp. 3Commissioned by DOE and NSF in 2001, co-chaired by H. Newman and L. Price. See http://gate.hep.anl.gov/lprice/TAN 4See http://www.datatag.org/ 5See http://www.startap.net/starlight 6See http://www.internet2.edu 7See http://www.teragrid.org 8L. Cottrell et al., see http://www-iepm.slac.stanford.edu/bw/ 9See http://www.internet2.edu/e2e

189 190 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

out by the network and software engineers, as well as the physicists with expertise in networking, in the Caltech group. In mid-2002, we joined forces with Caltech’s Netlab10, led by computer science professor S. Low. Working with Low, who developed the FAST TCP protocol stack that is capable of providing stable high performance on shared long distance links in the Multi-Gbps range for the first time, the Caltech group led a team (along with SLAC, CERN/DataTAG, the TeraGrid and Los Alamos) that conducted a series of breakthrough data transfer trials between October 2002 and February 2003. In November, using a 10 Gbps link between California and Chicago loaned to us by Level(3) Communications, and routing and switching equipment loaned by Cisco Systems, we achieved 8.6 Gbps sustained with FAST TCP, transferring 22 TBytes of data in 6 hours over a distance of 4000 km, with 10 data streams. In February 2003 S. Ravot used his own “GridDT” TCP stack to transfer a Terabyte of data between California and CERN in a single stream in one hour, using the US-CERN “DataTAG” 2.5 Gbps link at 99% efficiency over a distance of 10,037 km11 In addition to the rapid progress in developing the means to use multi-Gbps efficiently, the view of wide area networks also has been changed by the emergence of the next generation of optical networks. During the last year, CENIC12, UCAID (Internet2) and consortia of universities in the US began the transition to a national optical network infrastructure owned and operated (by an LLC formed) on behalf of the academic community13. The largest part of this initiative is National Lambda Rail (NLR), a dark fiber system covering the central and southern tiers of the US, led by CENIC and UCAID. In the Fall of 2002, Caltech began work with NLR, UCAID and CENIC, to make NLR and advanced networking protocols using the NLR backbone efficiently, available to the HENP community in support of its present and future experiments. Among the first steps now underway is the construction of a demonstration network using NLR and a Starlight-CERN link to be donated by Level(3), in time for the ITU Telecomm 2003 Conference (Geneva) in October, SuperComputing 2003 (Phoenix) in November, and the World Summit on the Information Society in December this year. Building on these developments, in March 2003 we formed and lead the “UltraLight” Collabora- tion14 to propose15 a next-generation “hybrid” packet-switched and circuit-switched optical network, monitored end-to-end and managed using intelligent software agents based on Caltech’s MonALISA16 system. UltraLight is planned to stretch across the US (using a dedicated transcontinental NLR wavelength) and the Atlantic, with extensions to Japan and Taiwan in Asia and Rio de Janeiro in South America. UltraLight uses three sets of flagship applications: HENP, VLBI and Radiation Oncology, to develop next generation networks adapted to the data-intensive needs of our field, as well as other scientific disciplines. The development and deployment of the VRVS system, used for collaboration by the LHC and other major HENP experiments, and throughout the HENP community, also has progressed rapidly. VRVS is now in its seventh year of 24 × 7 operation, and its user community continues to grow exponentially. In February 2003, VRVS underwent a major upgrade, to VRVS Version 3.0, making the system much more scalable, performant and robust, with a range of new features outlined in this chapter. The transition

10See http://netlab.caltech.edu 11This won the Internet2 Land Speed Record award for our team, for the second time. The first award to our team was in November 2002. 12The Corporation for Educational Network Initiatives in California. 13See, for example, http://www.ampath.fiu.edu/Miami03-Presentations/Corbato.pdf 14Led by H. Newman as PI. UltraLight includes Caltech (HEP, CACR and Computer Science), UCAID (Internet2; http://www.internet2.edu), the University of Florida, Florida International University and the AMPATH “Pathway to the Americas” team (http://ampath.fiu.edu), Michigan, MIT/Haystack Lab, Fermilab, SLAC, CERN, National Lambda Rail , Level(3) and Cisco, and the new generation of optically-switched research networks: Translight (US), Netherlight (NL), and UKLight (UK). 15UltraLight: An Ultra-scale Optical Network Laboratory for Next Generation Science. Proposal 0335287 to the NSF ANIR/EIN program, May 2003. 16See http://monalisa.cern.ch/MONALISA 9.1 Executive Summary 191 to Version 3.0 went very smoothly, with no major incidents, although 97% of the VRVS software code was replaced in the new version. At the time of the transition there were more than 14,000 registered hosts using VRVS in 61 countries. This effort is supported and continues to be developed as needed by a small team in our group, jointly funded by DOE and NSF, in collaboration with one team member from CERN.

9.1.1 Transatlantic Network Needs for LHC and HEP

Wide area networking is mission-critical for HENP, and the dependence of our field on high performance networks will increase over the next decade. The effectiveness of US participation in the LHC experimental program, US/European collaboration in other programs, and major HENP programs such as BaBar, RHIC and FNAL Run2, is particularly dependent on the speed and reliability of our national and international networks. As determined by the ICFA Network Task Force in 1997-1999, the ICFA Standing Committee on International Connectivity, and more recently the Transatlantic Network Working Group17,HENPre- quires substantial increases in the speed, quality, reliability, and ease of use of its international networks, continuing over the next several years. Given the trend towards lower prices per unit bandwidth18,the current strategy is to achieve these network upgrades at a gradual increase in cost over the next several years, to reach the multi-Gbps range on the principal links used by HENP within the next few years, and the 10 Gbps range (or higher speeds using multiple 10 Gbps wavelengths) before the LHC starts operation. The requirement for increased network capacity is set by the increasing scale and worldwide distribution of the data analysis for the next two rounds of major DHEP-funded experiments: those that are now data taking at SLAC, FNAL and BNL, and the LHC experiments that will start operation in 2007. The increasing need is reinforced by advances in computing and data handling technologies, and the growth of complex applications that draw upon distributed data storage and processing resources across an ensemble of networks. For the LHC, the use of a highly distributed, integrated set of resources and services in the form of Data Grids19 to satisfy the experiments’ data analysis needs is bringing new demands for network capacity and functionality. It was shown in 2001, in the Transatlantic Network WG, that the network requirements of experiments now taking data at SLAC and Fermilab are much larger than previously foreseen. This indicates that the network requirements for the LHC need to be revisited, as the actual needs are likely to be considerably larger than the baseline estimates developed in 2000-2001 (that were based on studies in 1998-2000). To complement the required bandwidth increases, new router and switch software and firmware will be necessary to manage the increasingly diverse range of applications, from large file transfers to a variety of realtime and multimedia applications. These trends, noted in last year’s Annual Report, have been borne out by the rapid advances in network protocols (at Caltech and elsewhere) and by the appearance of new optical network initiatives such as National Lambda Rail. This points to the need for an ongoing forward-looking evaluation of HENP’s network requirements, taking into account the evolving experience with running experiments, and the rapidly changing capabilities of local and wide area network technologies. As reviewed in the CMS/L3 Chapter of this report, data analysis in the LHC era will involve the use of a networked ensemble of Regional Centres as well as the CERN Centre, scattered throughout the world and serving a community of as many as 1,000 physicists simultaneously engaged in analysis for

17This committee’s work is discussed further in this chapter. 18Accompanied by higher costs for routing and switching equipment as one progresses to higher bandwidth ranges. 19See the PPDG (www.ppdg.net), GriPhyN (www.griphyn.org), iVDGL (www.ivdgl.org) and EU DataGrid (www.eu- datagrid.org) Web sites. Links to other Grid projects including the EU DataGrid project, may be found at these sites. 192 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

each experiment. No distributed system of comparable scope and complexity has ever been constructed, but advances in Grid and network technologies (including some developments by our group) are expected to bring it within reach over the next few years. Networks of suﬃcient bandwidth and sophistication are the fabric on which these systems will be built, and are therefore essential to exploit the physics opportunities at LHC. This is especially true for US physicists working on the LHC who need to take part in the physics discoveries while in the US.

9.1.2 Role of the Caltech Group

The Caltech group first proposed the use of international networks for HENP research, and has had a pivotal role in transatlantic networks for our field since 1982. Our group was funded by DoE to provide transatlantic networking for L3 (“LEP3NET”) starting in 1986, based on earlier experience and incremental funding for packet networks between the US and DESY (1982-1986). From 1989 onward, the group has been charged with providing CERN-US networking for the HEP community, and mission-oriented transatlantic bandwidth for HEP tasks wherever other network services have been inadequate (or nonexistent)20. Since 1996 our group has been charged to provide and support US-CERN network services for the US CMS and US ATLAS programs for the LHC. In 1997-8 we took a leading role in assessing and planning for the networking of future experiments worldwide, through the ICFA Network Task Force. From 1999 Caltech has continued this role on a more permanent basis, through the ICFA Standing Committee on International Connectivity (SCIC), established by ICFA in 1998. In 2001 Caltech (H. Newman) co-chaired the Transatlantic Network Working Group commissioned by the DOE/NSF Joint Oversight Group (JOG) for the LHC, and co-founded the Internet2 HENP Working Group in October 2001. Since December 1995 Caltech has been the major partner in the “USLIC” US LIne Consortium with CERN, Canadian HEP, IN2P321, and the United Nations (WHO) in Geneva funding a dedicated CERN-US line. Operations, management, and the development and application of systems for traffic monitoring and control, are shared between the CERN/IT External Networking Group and our Caltech group. In 1999-2000 our group was instrumental in bringing about CERN’s membership in UCAID, the managing organization of Internet2; the only non-US organization allowed equal member status with the Internet2 member US universities. Since 1996, Caltech also has had a leading role in providing advanced collaborative tools for the HENP community. The VRVS system developed by Caltech is now a de-facto standard within the community and has attracted substantial interest in other fields of science and engineering outside HENP. The Internet2 community is now adopting VRVS as their videoconference service and as a core technology for future collaborative services deployed over Internet2.

9.2 Status and Upgrades of US-CERN Network Links

Over the last year, the LHCnet transatlantic network infrastructure has been upgraded substantially, according to plan. To accommodate the required bandwidth increase, the production 155 Mbps (OC3)

20Apart from our direct responsibility for transatlantic networks for HEP, the Caltech group has had key roles in international network development over the past 20 years. This includes the debugging of the first US-Europe public (X.25) network service, TELENET in 1982; work on the Technical Advisory Group of the NSFNET in 1986-7; hosting the visit of IBM to CERN on behalf of US physicists, that led to the funding by IBM of the first T1 transatlantic link for research (at LEP) in 1988-91; network studies for the LEP and former-SSC eras; leadership in the MONARC, PPDG, iVDGL and related projects based on the Data Grid Hierarchy concept developed by our group. 21Institut National de Physique Nucléaire et de Physiques des Particules, France. See http://www.in2p3.fr 9.2 Status and Upgrades of US-CERN Network Links 193 circuit between CERN and StarLight22 has been upgraded to 622 Mbps (OC12) in June 2003 and a wavelength-based research link at 2.5 Gbps (OC48) has been deployed in the context of the DataTAG23- LHCnet partnership. The current generation of experiments now in operation such as Babar at SLAC, and D0 and CDF at Fermilab, have taken advantage of the newly available bandwidth, and next- generation experiments such as CMS and ATLAS, are already running new data-intensive applications over the transatlantic links. Physicists accessing CERN systems interactively, or using real-time multimedia applications such as VRVS, all rely on the production network’s performance and round-the-clock availability. To handle the increased bandwidth needs, new network equipment has been purchased and installed. Caltech installed a Cisco 7606 Optical Switch Router24 (OSR) and a Juniper M1025 at Starlight on the US side, while CERN purchased similar equipment on the European side. CERN is also renting two Alcatel 1670 Optical multiplexers26 and two Alcatel 7770 Routers using DataTAG funds provided by the European Commission. We have therefore been able to form a unique multi-platform transatlantic testbed. The current configuration, shown in Figure 9.1 provides a variety of services running over an optical wavelength. We are able to provide Ethernet private lines, layer 2 virtual private Networks (VPNs), transparent Ethernet local area networks (LANs) across the Atlantic, and routing using the new Internet Protocol IPv6. In parallel we have increased the CPU and storage capacity of the two sets of servers, at Starlight and at CERN, in order to support on-going protocol and Grid software developments. Our Linux cluster in Chicago now includes 20 Pentium4 CPUs and a storage capacity of 8 Tbytes (in the form of cost-effective ATA-RAID arrays). Alcatel American 1670 European partners 1670 partners Alcatel OC48 (Development and test) Cisco Cisco 7609 Juniper Juniper 7606 Alcatel Alcatel M10 M10 7770 7770 Linux Farm Linux Farm 20 P4 CPU 20 P4 CPU 8 Tbytes 8 Tbytes

OC12 (Production) Cisco Cisco CERN 7606 7609 Int. Network

Caltech (DOE) CERN in Geneva PoP in Chicago

Figure 9.1: LHCnet topology.

Our connectivity to other research networks also has been improved over the last year:

• We have formed an optical triangle by interconnecting Geneva, Amsterdam and Chicago with wavelengths. This is the ﬁrst step towards a ”pure” optical transatlantic network. In future

22The Starlight provides an independent site for co-location of network equipment in Chicago. See http://www.startap.net/starlight 23See Section 9.3 and www.datatag.org 24http://www.cisco.com/en/US/products/hw/routers/ps368/index.html 25http://www.juniper.net/products/ip infrastructure/m series/ 26http://www.alcatel.com/products/ 194 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

we intend to extend this to the UK, forming an optical quadrangle, once the recently approved “UKLight” project begins operations. • Our bandwidth to the Internet2 backbone Abilene has been upgraded from 1 Gigabit Ethernet (GbE) to 10 GbE. This has enabled us to partner more eﬀectively with Internet2 on advanced protocol developments using FAST TCP, and has encouraged them to join us in the UltraLight collaboration. • We have setup two new direct peerings with the Canadian national optical research and education network CA*net427 and the Taiwan research network TANet28. • We will soon have a direct GbE connectivity with FermiLab, who are installing a dark ﬁber to Starlight.

A general view of our peerings and our optical connectivity is shown in Figure 9.2.

SURFnet - Amsterdam TeraGrid Starlight facilities - Chicago 10 Gbps 10 Gbps 2.5 Gbps TeraGrid Hub FermiLab SURFnet Starlight Lambda Chicago 4*1GE PoP 10 Gbps GEANT Caltech triangle SLAC/Caltech (PoP) 10 Gbps** (DoE) PoP Sunnyvale

10 GE 1 GE 2.5 Gbps* 2.5 Gbps Abilene CalREN / CENIC NASA, ESnet, MREN, Canarie CERN - Geneva AMPATH, TAnet, STARTAP

* Will be upgraded at 10 Gbps ** Extension of the LHCnet - Level3 & Cisco Loan Figure 9.2: LHCnet peerings & Lambda Triangle.

In addition to the operation and the maintenance of the transatlantic network, our group is working towards advanced optical networking technologies and on high throughput data transfers, to meet the needs of DOE and NSF’s major experimental programs. Our leading position in those ﬁelds has allowed us to develop excellent relationships with academic and research networks (UCAID, CENIC and NLR in the US in particular), network equipment manufacturers (Cisco, Juniper), and telecom providers (Level(3) in particular). This has led to our ability to sometimes obtain exceptional discounts for routing and switching equipment from vendors, supplemented by loans of (Cisco) equipment and/or links (from Level(3)) for the most demanding tests. From November 2002 to February 2003, in collaboration with SLAC and TeraGrid29 , we extended LHCnet to Sunnyvale CA via a 10 Gbps circuit loaned by Level(3). At Sunnyvale, Cisco Systems loaned us a GSR30 (the most powerful router manufactured by Cisco)

27http://www.canarie.ca/canet4/ 28http://www.tanet2.net.tw 29http://www.teragrid.org 30http://www.cisco.com/en/US/products/hw/routers/ps167/index.html 9.2 Status and Upgrades of US-CERN Network Links 195 and several 10 Gigabit-Ethernet Cisco modules for our routers at Starlight and at CERN. Intel loaned us several 10-Gigabit Ethernet (10GE) PCI-X network adapters (PRO/10GbE LR)31 for some of the servers at either end of the link. This exceptional testbed has allowed us to break several data transfer records in less than 4 months. We have transferred 22 TByte in 6 hours from Baltimore to Sunnyvale via Chicago (over 4,000 km) and we broke the Internet2 Land Speed Records32 for both single- and multiple stream transfers with this trail. We transmitted over a TByte of data in just under an hour from Sunnyvale in California to CERN in Geneva (over 10,000 km) with a single TCP stream between 2 PCs with Intel 10GbE PCI-X network cards (see Section 9.4 for further details). Recently, Juniper loaned us a GbE network adapter for our M10 at Chicago, allowing us to carry out a series of high speed IPv6 experiments, breaking the current records for IPV6 data transfers. Following our plans presented to the TAN WG, and discussed in last year’s Annual Report, we are planning to merge our 622 Mbps production circuit and our 2.5 Gbps R&D circuit this summer in a single OC192 (10 Gbps) circuit. In order to maintain a strict separation between production and test traffic, so that the production service is never interrupted by R&D trials, we will implement separate channels across the Atlantic using MPLS33 technology. We also intend to use MPLS to ensure efficient and reliable use of the link, up to relatively high occupancy levels, for each of a wide variety of network tasks, by dividing the network traffic into different service classes based on their characteristics and requirements. Examples of these classes may include large scale data transfers, realtime collaborative sessions, interactive traffic, Ethernet services, and general Web-access. In order to implement this new level of network service, Caltech and CERN are each purchasing a Juniper T320 Router which is capable of running the 10 Gbps link, and many ports switching the traffic at full speed, while also supporting a complete and dependable MPLS service as well as other proprietary network management facilities. Larger versions of these routers (T640’s) with similar specifications are known to be highly reliable, with a level of processing power that is nearly immune to Denial of Service attacks, and are used in the core networks of most major network providers as well as Abilene and the TeraGrid. The new T320’s will also support further network upgrades in the future, as they are capable of supporting up to four wide area network links of 10 Gbps. The future bandwidth and hardware capacities of the LHCnet backbone are integrated in a new optical network proposal submitted to the NSF. The proposal named UltraLight34 is led by our team. The project will develop a global optical network testbed, and scalable distributed storage and Grid systems, integrating and leveraging the major facilities of LHCNet and DataTAG with transcontinental 10 Gbps wavelengths from National Lambda Rail35, in research partnership with Starlight, UCAID36, Cisco and Level(3). Additional trans- and intercontinental wavelengths in our partner projects Trans- Light37, Netherlight38, UKlight39, AMPATH and CA*Net4 will be used for network experiments on a part-time or scheduled basis. The initial planned UltraLight implementation is shown in Figure 9.3 and the configuration of a typical planned UtraLight site is illustrated in Figure 9.4. This will give us a core experimental network with principal nodes in California (Caltech and SLAC), Chicago (StarLight and Fermilab), Florida (U. Florida and FlU), Michigan (U. Michigan), Massachusetts (MIT/Haystack), CERN, Amsterdam and the United Kingdom (UC London40), with extensions across Abilene, to Taiwan and Japan in Asia, and across AMPATH to South America.

31http://www.intel.com/network/connectivity/products/server adapters.htm 32http://lsr.internet2.edu 33Multi-Protocol Label Switching; see http://www.mplsrc.com/ 34The proposal submitted is available at http://pcbunn.cacr.caltech.edu/UltraLight/ 35http://www.ucar.edu/ncab/Minutes/2003/marla.nlr.umc.20030220.ppt 36http://www.ucaid.edu/ 37http://www.internet2.edu/presentations/20030409-TransLight-DeFanti.ppt 38http://carol.wins.uva.nl/ delaat/optical/ 39http://www.cs.ucl.ac.uk/research/uklight/ 40http://www.cs.ucl.ac.uk 196 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

The foundation of UltraLight will be a very flexible network fabric, high performance network- resident server clusters, ”ultrascale” protocols designed to support stable high performance, MPLS for fair-shared use of networks at multi-Gbps speeds, and intelligent software agents in the MonALISA41 monitoring framework that will manage network operations and provide an interface between the flagship applications and the ultrascale protocol stack to optimize performance. UltraLight will operate in a new ”hybrid” mode. The principal 10 Gbps wavelength interconnecting the core sites will accommodate a diverse traffic mix resulting from the flagship applications, using packet switching. Monitoring agents will handle network flows exceeding the capacity of the core wavelength by dynamically routing the traffic over alternate paths, using the multiply connected topology and the multi-wavelength nature of UltraLight (across the Atlantic and NLR in particular), to transport Terabyte and larger datasets while simultaneously delivering multi-Gigabyte images in seconds. A principal aim of UltraLight will be to apply its results on network operation, dynamic provisioning, optimization and management broadly, and to develop standards that will serve HENP’s major projects, as well as many other fields of science and engineering.

Figure 9.3: Initial Planned UltraLight Implementation.

9.3 DataTAG Project (and Caltech Partnership)

The fundamental objective of the DataTAG42 project is to create a large-scale intercontinental testbed for data-intensive Grid applications. The project focuses upon advanced networking technology and interoperability between the US and European Grid domains. A transatlantic 2.5 Gbps link for high performance network service and data transfer applications has been deployed in order to deﬁne and

41See Section 9.10.8 42DataTAG means Research and Technological Development for a Trans-Atlantic GRID. See http://www.datatag.org 9.3 DataTAG Project (and Caltech Partnership) 197

Figure 9.4: Typical UltraLight site. implement a compatible platform between three U.S. Grid projects (iVDGL43,GriPhyN44 and PPDG45) and the European DataGRID46 and CrossGrid47 projects. The funding for DataTAG is shared between the European Union, US DOE through Caltech, and US NSF through the Eurolink program. Caltech leads the operational and maintenance tasks on the US side of the LHCnet, and we coordinate the connectivity with US research networks such as Abilene, EsNet, CENIC or MREN. We have a major role in the collaboration with US partners (iVDGL, Internet2, PPDG, GridPhyN, SLAC, Fermilab, BNL etc.) and we are actively involved in the DataTAG project’s various work packages. In the area of high-performance data transport, two specific areas are being investigated which together have a major impact upon the level of end-to-end performance. The first is an investigation into end-system behavior, which includes the review of the components that affect performance, and in particular the motherboard and Network Interface Card (NIC) combination and Linux network drivers48. The second area concentrates on the TCP protocol stack. Excellent progress has already been made by our team in translating the theoretical studies of TCP (and its inherent problems) into prototype implementations. We are following two different approaches:

• GridDT49 involves modiﬁcations of the the sender’s TCP stack such that the behavior of multiple TCP connections can be emulated by a single TCP connection.

43http://www.ivdgl.org/index.php 44http://www.griphyn.org/index.php 45http//www.ppdg.net/ 46http://eu-datagrid.web.cern.ch/eu-datagrid/ 47(http://www.crossgrid.org/ 48We are currently using a “Datatag” Linux kernel (2.4.20) which includes an enhanced Syskonnect driver: http://datatag.web.cern.ch/datatag/member area.html 49See http://sravot.home.cern.ch/sravot/GridDT/GridDT.htm 198 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

• FAST 50 TCP is a revolutionary TCP implementation. It has been designed to solve the equilibrium and stability problems of conventional TCP. It achieves its equilibrium properties using end-to-end delay rather than packet loss as the measure of congestion, and is able to achieve any desired degree of fairness, expressed by a utility function.

The two new Caltech implementations are providing test results that indicate that performance can be dramatically improved in production networks. Further details about GridDT and Fast TCP are given in the next section. The project is also very active in demonstrating and implementing advanced end-to-end network services across multiple domains. The Datatag testbed is used to carry out the the following innovative research tasks.

• Evaluation of mechanisms for provisioning of Quality of Service, especially when applied to live traﬃc produced by the Grid (e.g. bulk data, interactive, video, control messages, etc.) and deﬁnition of the most suitable services for the Grid. • Development of an advance reservation mechanism for network resources. This work builds upon the mechanisms developed for the Internet2 Bandwidth Broker, the ITTC51 and the Globus advance reservation and resource allocation service GARA52 that is part of the standard Globus toolkit, and is closely integrated with the work being carried out in the European DataGRID project. • Innovative monitoring tool development, to measure and understand user perceived performance.

In order to study interoperability issues between Grid middleware developed in the US (GriPhyN and PPDG) and in Europe (DataGRID), and to deploy interoperable solutions for specific Grid applications across the Atlantic, a common program called GLUE (Grid Laboratory Uniform Environment) has been defined, together with the iVDGL project. The GLUE program focuses on interoperability between US Grid projects in High Energy Physics (iVDGL, GriPhyN and PPDG) and the European DataGRID and DataTAG projects. The GLUE management and effort is provided by the iVDGL and DataTAG projects specifically. The collaboration between the European and U.S. partners of DataTAG is depicted in Figure 9.5. GLUE includes a range of sub-projects that address various aspects of interoperability:

• Tasks to deﬁne, construct, test and deliver interoperable middleware;

• Tasks to help experiments with their intercontinental Grid deployment and operational issues;

• Establishment of policies and procedures related to interoperability.

Once the GLUE collaboration has established the necessary minimum requirements for interoperability of middleware, any future software designed by the projects under the umbrella of the HICB (High Energy Physics Intergrid Coordination Board) and JTB (Joint Technical Board) will have to maintain the achieved interoperability.

50Fast AQM Scalable TCP. See http://netlab.caltech.edu/FAST/ 51See http://www.ittc.ukans.edu/ kdrao/ai.html 52http://www-fp.mcs.anl.gov/qos/ 9.4 High Throughput Network Developments 199

U.S. Part U.E. Part

iVDGL DataTAG

HICB

GLUE

Griphyn/PPDG DataGRID

HEP Experiments

Figure 9.5: GLUE.

9.4 High Throughput Network Developments

In order to take advantage of new backbone capacities, which are advancing rapidly to 10 Gbps, there is a clear and urgent need for a transport protocol that provides reliable multi-Gbps throughput end-to-end. TCP is the most common solution for reliable data transfers over IP networks, and currently comprises 90% of the traffic on the Internet. Although TCP has demonstrated a remarkable ability to adapt to vastly different networks, recent theoretical and experimental results have shown that TCP becomes inefficient when the bandwidth and the latency increase. The current “additive increase, multiplicative decrease” algorithm it uses to respond to network congestion, or to uncongested packet loss (due to line errors, etc.) limits its ability to use the bandwidth on long, high bandwidth effectively53 HENP experiments, and the LHC experiments in particular, need to overcome these problems if our field is to use its wide area networks as needed. To address these problems, our group has followed two complementary approaches, both of which have proven to be successful over the last 12 months. The first one is a simple change to the traditional TCP congestion control algorithm (RFC2581) which dramatically improves TCP performance in high-speed wide area networks. The new algorithm called GridDT tunes AIMD parameters. Using the GridDT54 linux patch, the user can emulate the behavior of a TCP multi-streams transfer with a single stream. He can also virtually increase the maximum packet size (MTU) and improve the fairness between TCP streams with different round trip times (RTT) that are following different network paths but share a common link along the way. In the context of a collaboration between our team and Politechnica University of Bucharest55, Romanian students are involved in the implementation and optimization of the GridDT algorithm. A new patch will be available at the beginning of July. The second approach is much more innovative. Our group is strongly involved in the development of

53See “The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm”, by M. Matthis et al., Computer Com- munication Review, Number 3, 1997, http://www.cs.ucsd.edu/classes/wi01/cse222/papers/mathis-tcpmodel-ccr97.pdf 54Grid Data Transport. See http://sravot.home.cern.ch/sravot/GridDT/GridDT.htm 55http://www.pub.ro/English/eng.htm 200 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP a new revolutionary TCP stack called FAST56 developed at Caltech by Steven Low’s “Netlab” group in the Computer Science department. In addition to many enhancements, the most important diﬀerence is a new congestion avoidance algorithm that uses queueing delay, as opposed to packet loss as the standard TCP Reno algorithm uses, as a measure of congestion. This has important equilibrium and stability implications. Fast TCP is stable in that it maintains stability for arbitrary delay, capacity, routing and load; rescales itself as these parameters change; does not oscillate in equilibrium; need not be re-tuned as these parameters change; exhibits good performance with negligible queuing delay and loss and responds rapidly to changing events, including moderate packet loss. Figure 9.6 shows the results from simulations of Fast TCP when compared to existing TCP implementations (TCP/RED), illustrating its promise to provide stable high throughput over long distance networks. 1

0.95 FAST

0.9

0.85

0.8

0.75 TCP/RED 0.7 Link utilization

0.65

0.6

0.55

0.5 200 400 600 800 1000 1200 Capacity (pkts/ms) Figure 9.6: A simulation showing the comparison of link utilisation for FAST TCP with TCP/RED for increasing link capacity.

The Caltech FAST kernel was demonstrated publicly for the ﬁrst time in a series of experiments conducted during the SuperComputing Conference in Baltimore, MD, in late November 2002 by a Caltech-SLAC research team working in partnership with CERN, DataTAG, StarLight, Cisco, and Level(3). The demonstrations used a 10 Gbps link donated by Level(3) between Starlight (Chicago) and Sunnyvale (California), as well as the DataTAG 2.5 Gbit/s link between Starlight and CERN (Geneva), and the Abilene backbone of Internet2. The network routers and switches at Starlight and CERN were used together with a GSR 12406 router loaned by Cisco at Sunnyvale, additional Cisco modules loaned at Starlight, and sets of dual Pentium 4 servers each with dual Gigabit Ethernet connections at Starlight, Sunnyvale, CERN and the SC2002 show ﬂoor provided by Caltech, SLAC and CERN. These tests achieved a number of groundbreaking advances during the demonstrations, using the FAST kernel with standard packet size including:

• Demonstration of stable transfer rates of more than 950 Mbit/s between CERN in Geneva and Sunnyvale, over a distance of over 10,000 km on a single Gigabit Ethernet port at each end of the path.

• Sustaining 9 Gbps on an OC192 link, that supports a maximum of 9.6 Gbps, using 12 servers and 12 Gigabit Ethernet ports from Baltimore to Sunnyvale.

56http://netlab.caltech.edu/FAST/ 9.4 High Throughput Network Developments 201

• Sustaining 11.5 Gbps from the SC2002 show ﬂoor to the Level(3) Sunnyvale link and Abilene, using 14 servers and 14 Gigabit Ethernet ports.

A comparison of the FAST TCP and standard (RENO) TCP throughput as a function of time, between Baltimore and Sunnyvale is shown in Figure 9.7. The figure shows the stable high efficiency of the FAST TCP stack, and the lower efficiency and instability of the standard stack. Link Utilization 95% Average 1000 1000

19% Throughput (Mbps) Throughput (Mbps) 200 200

0 Time (s) 3600 0 Time (s) 3600

Linux TCP FAST

Figure 9.7: FAST TCP versus standard (RENO) TCP throughput. The latter is labeled as Linux TCP.

Over the past year, we twice broke the Internet2 Land Speed records57 for both single and multiple data streams. During SC2002 last November, working in collaboration with CERN, SLAC an NIKHEF58, we achieved 923Mbits/s with an end-to-end application-to-application single TCP RENO stream from Amsterdam to Sunnyvale over a link limited by the Gigabit Ethernet ports at either end. In February 2003, we beat our own Internet2 record by a factor of approximately 2.5, by transferring 1 Terabyte of data across 10,037 kilometers in less than one hour from Sunnyvale to CERN. Using the GridDT stack developed by S. Ravot of our group, we sustained a TCP rate of 2.38 Gbps for more than one hour, which means that we managed to use an average of 99% of the available DataTAG link capacity during this time. The throughput results are shown in Figure 9.8. This was the ﬁrst time in the history of the Internet that a transoceanic single TCP stream achieved multi-gigabit per second throughput. The demonstration used the optical networking facilities of LHCnet, TeraGrid and the Chicago-Sunnyvale link loaned by Level(3) Communications. It also relied on the new Intel 10-Gigabit Ethernet (10GE) PCI-X network adapters (PRO/10GbE LR) loaned by Intel and on a Cisco GSR 12406 router loaned by Cisco Systems. It is notable that the throughputs achieved in these recent tests over an intercontinental wide-area network exceed those usually experienced within a single computing cluster. It is clear that the ability

57http://lsr.internet2.edu/ 58/http://www.nikhef.nl/ 202 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

Figure 9.8: Results of the California-CERN trial carried out by the Caltech-led team that won the current Internet2 Land Speed Record. to transport data in this way opens up new options in the way data-intensive science may be done: sharing computational and data handling resources in a more flexible and dynamic way, delivering data and results with better turnaround from a set of resources spread across world regions, and thus enabling data analysis in large experiments to be done more efficiently. It also brings the day closer when the combined resources of a worldwide physics collaboration could be treated as a single logical resource, exploiting Grid software in much more effective ways than are possible today, as a means to future physics discoveries.

9.5 HENP Network Needs and the Transatlantic Network (TAN) Working Group

This section summarizes the bandwidth needs-projections for the major transoceanic links and domestic links to the US HENP labs, as well as links to universities. The starting point for the estimates is the October 2001 report of the TransAtlantic Network (TAN) Working Group59. We update those requirements estimates in this section, taking into account recent trends in the development and use of optical network infrastructure, the continued drop in cost per unit bandwidth for wavelength-based network services, and our recent development, together with Caltech Computer Science, of protocols that are able to use and share long distance networks efficiently, up to the 10 Gbps speed range. Beyond the simple requirement of adequate bandwidth for transferring of large files, physicists in DOE/DHEP’s and NSF/EPP’s major programs require (1) an integrated set of local, regional, national and international networks able to interoperate seamlessly, without bottlenecks, (2) networks that will accommodate and satisfy the latency and jitter requirements of realtime applications coexisting with high-bandwidth data transfers, (3) network and user software that will work together to manage the bandwidth effectively, and (4) a suite of videoconferencing, shared application and other high-level tools for remote collaboration that will make data analysis from the US (and from other remote sites) effective. Our developments in these areas are reviewed in several other sections of this chapter. Two levels of bandwidth are foreseen:

59http://gate.hep.anl.gov/lprice/TAN 9.5 HENP Network Needs and the Transatlantic Network (TAN) Working Group 203

• Baseline requirements based on a relatively conservative view of technology evolution, sufficient for the experiments’ needs for distributed data access, processing, delivery and analysis, in a strictly managed environment. • Requirements to satisfy the needs for access to significant data subsets (0.1 to 1 TB scale) on demand, by individuals and small working groups, and for automated data movements within a unified Grid environment, as well as such new working methods as widespread “persistent” remote collaboration throughout the working day. This corresponds to a best guesstimate (a medium-optimistic view) of the evolution of technology and cost-performance over the next five years.

The baseline bandwidth needs of the major experiments determined in 2001, discussed extensively by the TAN Working Group and given in their report, are summarized in the following table, in Mbps. The table gives the projected bandwidth requirements for each major experiment to the laboratory site where the experiment is located, and where the largest computational and data handling tasks occur. We also note that the table shows that the network requirements for the LHC experiments need to be updated, as they appear to be very conservative compared to the current generation of running experiments. In the case of the LHC experiments, the US ATLAS and US CMS contingents will share network access to the CERN laboratory with ALICE, LHCb and the non-LHC programs at CERN. The US- CERN link also carries most of the network traffic to Europe, for BaBar, Run2 and other major HENP programs, so that the US-CERN bandwidth requirement is expected to reach the 10 Gbps range by 2005; in advance of the start of LHC operations in 2007. The figures in Table 9.1 are the baseline requirements for installed bandwidth, where we assume, based on experience in Internet2 and elsewhere that stable network operation, and support for a set of data-intensive HENP applications, requires a network occupancy of 50% or less. The projected requirements for the actual sustained data rate for an experiment are thus one half of the figures shown60.

Table 9.1: 2001 2002 2003 2004 2005 2006 CMS 100 200 300 600 800 2500 ATLAS 50 100 300 600 800 2500 BaBar 300 600 1100 1600 2300 3000 CDF 100 300 400 2000 3000 6000 D0 400 1600 2400 3200 6400 8000 BteV 20 40 100 200 300 500 DESY 100 180 210 240 270 300

US-CERN BW 155-310 622 2500 5000 10000 20000

The purpose of the conservative baseline ﬁgures (and in the case of the LHC experiments, very

60Discussions with Qwest in 2002 on their next generation network plans have indicated that the maximum sustained occupancy of an IP-based wide area network in the Gbps range should be 40%, and recent reports (e.g. by Roberts, gathering information from the top 20 US service providers) indicate that one should plan for a maximum sustained occupancies in the range of 15-30%. Given the uncertainties in the baseline requirements ﬁgures, and the newness of IP networks at these speeds, the TAN Working Group in 2001 decided to keep the ﬁgures in the table based on 50% occupancy. 204 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

conservative) is to establish a minimum bandwidth level at which experiment will be able to function, within a moderate budget envelope. In case technology evolution and market forces lead to lower unit prices than assumed in the baseline, it is recommended that the experiments be allowed to optimize their network installations and modes of operation to achieve greater efficiency in extracting physics results. The differences in the table among experiments in naively similar situations (e.g. D0 and CDF) are related to the different Computing Models assumed. The CDF model has been relatively highly centralized in 2001-2003, with only modest amounts of data being sent overseas via networks. The D0 model, both for real and simulated data, is substantially more distributed. CDF is also planning to use a more distributed Computing Model for Run 2b, and that is expected to substantially increase the network needs (as shown in the table) starting in 2004.˜ The LHC experiments’ Models are structurally in between the CDF and D0 Models, in that the data flow across networks is assumed to be reduced by providing efficient access to subsets of the data through the use of objects stored in relational databases coupled to Data Grid systems. The Caltech group is developing this mode of data access, but a large software development effort by the experiments is still required over the next few years (in the context of the LHC Computing Grid Project, for example) to make this possible61. Several of the experiments listed would benefit substantially from additional bandwidth. Greater bandwidths would allow: freer access to the data and processed results by individuals and small workgroups; more extensive data distribution among the sites; and greater flexibility in treating the ensemble of site facilities as a coherently managed Grid system. It should be noted that the table entries for US ATLAS and US CMS represent the minimum requirements, in a managed working environment. Even if very efficient modes of access are fully developed for these experiments, the breakdown of the LHC needs into the tasks that are covered by the bandwidth estimates given in the table shown above indicates that these estimates are overly restrictive. Specifically, the entries for the LHC in the table do not include support for the following activities:

• Sufficient bandwidth for BaBar to reprocess and analyze data at a rate substantially larger than the data taking rate, during peak periods. • Data distribution, processing and access by ATLAS at a rate corresponding to the trigger rate foreseen: 270 Hz at 1033 luminosity, and 400 Hz at the design luminosity of 1034. CMS also is considering writing data at a higher rate if that is affordable. Up to the present, the Hoffmann Review on LHC Computing has assumed a reference trigger rate of 100 Hz for both ATLAS and CMS. • The impact of other programs, such as the other LHC experiments, NuMI and the rest of the Fermilab fixed target program, and US participation in the LHC heavy ion program. • The requirements of Grid-enabled analysis, where the analysis would benefit greatly from the ability to access and transport on demand, 0.1 to few-Terabyte collections of analysis objects over wide area networks. As discussed elsewhere in this chapter, we are already able to transport such collections over intercontinental distances in minutes to an hour using links of up to 10 Gbps, and store and handle the collections at moderate cost at Tier2 and Tier3 centers (corresponding to relatively modest costs in 2007).

The ﬁgures above for BaBar roughly correspond to the needs for connectivity into the SLAC site, since large-scale data transfer needs between SLAC and IN2P3, as well as other Regional Center sites,

61If this software effort is less than fully successful, a likely consequence would be increased network needs for the LHC program. 9.5 HENP Network Needs and the Transatlantic Network (TAN) Working Group 205 dominate those needs62. In the case of FNAL, one has to consider the sum of the D0, CDF, and BTeV needs, a major part of the CMS needs, and the needs of MINOS and other FNAL programs. At BNL the needs for the RHIC program as well as ATLAS need to be supported. As a result of these considerations, the installed bandwidth into SLAC that has recently been upgraded to OC12 (622 Mbps) needs to be upgraded to and at least OC48 (2.5 Gbps) by 2005. The connectivity to BNL as a whole taking RHIC into account should reach at least OC48 in 2005, and OC192 (10 Gbps) by 2006. According to the table, FNAL’s connectivity should reach OC192 by approximately 2005, and the needs for installed bandwidth appear likely to exceed OC192 substantially by 200663 Comparing the projected needs of the LHC experiments (estimated in 1999-2000), to the estimated needs for BaBar and Run2, it appears that the estimates for the LHC are overly conservative, and need to be optimized again taken into account the needs of Grid-enabled analysis, and the rapid evolution of network technology and costs. There is further specific evidence for the increased needs of the HENP program. BaBar’s needs for the US-CERN link are as stated in the above table: 600 Mbps installed, corresponding to 300 Mbps throughput maximum. But the actual goal for BaBar in 2002 and early 2003 was 600 Mbps of net throughput. This has been carried mostly over the SLAC link to Internet2 (via Stanford University) and then over the US-CERN link, with the rest being carried over ESnet, the general-purpose GEANT network infrastructure in Europe, and the RENATER French national network. Projecting the SLAC usage (averaged over the last 20 years) forward to 2012 gives an estimated bandwidth requirement of more than 1 Terabit/sec (Tbps) within a decade from now. However, the network usage growth in 2002-3 was roughly a factor of three per year. Projecting the recent growth trend forward gives an estimate of 24 Tbps required in 2012. We note that the requirements figures above are very roughly consistent with earlier network requirements estimates, including those of the ICFA Network Task Force64 (1997-8) and the needs recognized by the ICFA-SCIC (1998 –), although in detail they are larger. This reflects the rapid advance of network technology over the last five years, and the outlook for reductions in the unit price of bandwidth between 2003 and 200765 In response to the increased needs, and the favorable prices for bandwidth achieved after negotiations with KPNQwest, and subsequently T Systems, we have improved our bandwidth projections for the US-CERN link66. For all of the technical and programmatic reasons given above, the new bandwidth profile for the US-CERN link is closer to the actual requirements, and should be able to serve HENP adequately over the next few years. At the same time, and in spite of increasing costs of optical network equipment as one progresses from one bandwidth tier to the next, we have been able to develop cost-effective plan to meet these goals. Specifically, we have been able to slightly reduce our overall cost estimates, relative to the estimated costs we presented last year. This is a result of our joint network developments with

62Note that the bandwidth required to reprocess data in shorter periods than the time it takes to acquire the data in the first place are not included in these “baseline” figures. 63In order to meet these needs, without local network access restrictions, both FNAL and SLAC are proceeding in 2003 with installing dark fiber connections, to Starlight and to Internet2 respectively. 64See the ICFA-NTF Requirements WG Report (May 1998) at http://l3www.cern.ch/ newman/icfareq98.html. 65Estimates of the yearly cost reduction of bandwidth range from 25 to 50%. The recent precipitous drop in prices, in 2001-3 is expected to moderate in the coming years. Most of the major network providers have gone bankrupt as a result of the low prices, and we are entering a period of consolidation. The vendors who are still operating are struggling to recoup their investments (an aggregate of many tens of billions of dollars) in the buildout of national and transoceanic fiber infrastructures. 66In mid-2002 the economic problems throughout the telecom sector affected KPNQwest and other major vendors including Global Crossing. KPNQwest (the European partner of Qwest providing the US-CERN link) filed for Chapter 11 bankruptcy protection in May 2002, and was dissolved in July. CERN and Caltech responded by investigating offerings from alternative vendors to ensure a stable network service, and we were able to obtain a replacement service from T Systems that enabled us to proceed on schedule with the DataTAG research plans, as discussed elsewhere in this chapter. 206 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

leading vendors (recently Cisco and Level(3) Communications and others), leading to our ability to negotiate highly discounted prices for equipment, and relatively low link costs. The updated funding plan is discussed further in Section 9.6. Over the last year we have developed a network roadmap for the next decade for the US-CERN link, as representative of the evolution of the major links used in our field. This roadmap, presented by M. Kasemann at the ICHEP 2002 conference in Amsterdam, and by H. Newman to ICFA on behalf of the ICFA Standing Committee on Inter-regional connectivity in February 2003, is summarized briefly in Section 9.8. The outlook and advances in the corresponding network technologies that are likely to will make this roadmap practical (and affordable) are discussed in Section 9.7.

9.6 LHCNet Funding Plan for the US-CERN Links 2003-2007

In response to one of the main elements of the charge of the TAN Committee, a detailed funding plan for the US-CERN link was developed in 2001 by Caltech, and that plan is updated here for 2003-2007 to meet HENP’s requirements while containing the costs. This follows earlier determinations of the future network needs by the LHC experiments, and the ICFA-NTF and ICFA-SCIC over the last ﬁve years. This plan has been designed to meet the needs for US-CERN transatlantic networking: both for the LHC experiments and for DOE/DHEP’s other major programs67. This would include several 10 Gbps wavelengths (or one 40 Gbps wavelength) on the US-CERN link between the US and CERN at the start of LHC operations, which is expected (according to current estimates) to be adequate to accommodate the needs of the four LHC experiments, along with the needs of BaBar and the Run2 experiments for use of this link. Similar plans are being developed for some of the principal domestic links, including the links to the Tier1 centers at FNAL and BNL and the Tier A center at SLAC by ESnet, as well as links to the Tier2 centers over Internet2 and regional networks. Further updates to optimize this plan will be made yearly, as the Computing Models of the LHC and the other major experiments are reﬁned, and as experience with optical speed network technologies (and pricing) progress. The basic bandwidth requirements and cost parameters in the plan are summarized as follows:

1. Follow the known bandwidth trends on the US-CERN link, as well as other larger scale research and education networks. These trends are remarkably consistent between ESnet, use of the CERN link, DFN in Germany, and the academic portion of “the Internet” in other countries. The rate of bandwidth growth was very close to 100% per year during the late 1990’s, although the rate of growth has accelerated in 2000 to approximately 150-200% per year on Internet2 and in ESnet68 as well national academic and research networks outside the US69 as well as the Internet at large where the annual traﬃc growth rate has averaged 300% over last ten years70. 2. Ramp up in 2003 to 2.5 Gbps, in time to support BaBar, D0, CDF and RHIC running with higher data rates and larger event samples to analyze than in 2002, along with simulated event productions for CMS and ATLAS data challenges, and Data Grid developments for the LHC.

67The discussions in the Review of LHC Computing at CERN have conﬁrmed the recent estimates for network requirements at the start of LHC operation: the aggregate requirement into CERN is estimated to reach several times 10 Gbps by the start of LHC operation in 2007. 68A growth spurt in 2001 temporarily reached a rate of six times per year, as BaBar and other experiments began transferring more data. 69Examples include Japan, Chile, and eastern China. 70The growth rate in the traﬃc carried by the top 20 ISPs in the US peaked at 4.5 times per year in the Summer of 2001, dipped in September 2001, and has since resumed the “traditional” 3 times per year growth. See the presentation by L. Roberts at http://www.caspiannetworks.com/pressroom/press/01.16.02.shtml 9.6 LHCNet Funding Plan for the US-CERN Links 2003-2007 207

The ramp-up to OC3 (155 Mbps) and then to OC12 (2.5 Gbps) has brought substantial economies of scale in 2001-3, as this has kept pace with the typical bandwidth unit in the US (and increasingly across the Atlantic) for medium to large-sized wide area network customers at that time. Similar substantial economies of scale are expected to be obtained this year (by this Fall) with the move to OC48 (2.5 Gbps) together with consolidating the OC48 DataTAG research link into a single bandwidth-managed OC192 (10Gbps) link, as discussed earlier in this chapter71. Budgetary prices from vendors indicate that the cost of OC192 (4 times more bandwidth than OC48) in 2003-4 is going to be only slightly higher than the cost of the two OC12 (production) and OC48 (development) services we are running now on the US-CERN link. 3. Prepare each year for the production network of the following year. We began this procedure in 2003 by installing a research link such as the DataTAG OC48 wavelength in the Sumer of 2002, and by using much of the OC192 wavelength for network development (including the development of “ultrascale” protocols such as FAST TCP, and the use of MPLS for bandwidth management, as discussed in a previous section of this chapter) starting this Fall. This is a very important part of the plan since the equipment to be used (e.g. optical multiplexers, as described above), services (e.g. MPLS, G-MPLS) and software are both new and quite diﬀerent from the traditional IP routers and switches we have used to cover the T1 to OC12 bandwidth range.

Speciﬁcally, develop the optical network infrastructure and methods in 2002, and move to 5 to 10 Gbps on the production link in 2004-5, in partnership with the DataTAG project. 4. Following more conservative, longer term trends, we plan for a bandwidth increase by a factor of close to two (a 100% increase) each year from 2004-2006 (as shown in the tables given above). This proposed rate of growth also is consistent with the longer term trends of HENP network usage over the last decade, as derived by the ICFA-NTF and ESnet, of a factor of 10 in bandwidth every 3 to 4 years, although recent growth rates have been larger72 5. Reach 10 Gbps in production by 2005 and begin to use multiple 10Gbps wavelengths by 2006- 7. This is compatible with the growth pattern in the major academic and research networks (Abilene, Mational Lambda Rail in the US; GEANT across Europe; Netherlight (NL), etc.).This is expected to meet the baseline needs of the LHC experiments, the revised bandwidth estimates from BaBar73 resulting from the exceptional performance of the accelerator and the detector, the estimated use by the Run2 experiments74, and the other transatlantic network use by HENP75. This turns out to be generally consistent with following the bandwidth trends mentioned above. It is also noteworthy that the revised plan for US-CERN bandwidth is more in line with the connectivity requirements estimated into Fermilab. 6. During 2003 to 2004 we will progressively move more of the 10 Gbps link into production service; moving from 2.5 Gbps in 2003 to 5 Gbps in production in 2004 (managed by MPLS) For 2005 and later years, we assume a cost decrease per unit bandwidth of 38% per year, or a factor of 2 every 1.5 years. Although data link costs have recently fallen rapidly, they have often fallen much more slowly in the past. There is also hard evidence that the national buildout of optical

71Note for comparison that the typical bandwidth on the major links in international academic and research networks was 622 Mbps (OC12) in 2000, moved to 2.5 Gbps (OC48) in 2001-2 and is now going to move 10 Gbps (OC192) by 2003. Domestic link bandwidths in Abilene, the G-WIN German academic network, the SuperJANET network in the UK, and the Super-SINET Japanese network all have moved to 10 Gbps in 2002-3. The GEANT pan-European backbone has many links at 10Gbps, and is expected to move to higher speeds in the next 1 to 2 years. 72This more conservative outlook is also consistent with the coming consolidation, and reduced competition that is expected in the telecom industry starting in the near future. 73Reference: Reports and presentations by C. Young (BaBar) to the Transatlantic Network WG in 2001. 74Reference: Reports and presentations by B. Klima (D0) and F. Donno (CDF) to the Transatlantic Network WG in 2001. 75Taking into account that some of the traﬃc for BaBar and Run2 will use other network links, funded by the European partners in these experiments. 208 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

fiber-cable infrastructure is over. The current price drops are the temporary result of vendors acquiring the “distressed assets” of their bankrupt former competitors at low cost. In the near future we expect to enter an era where the remaining network vendors struggle to recover their large investments in building and operating their large infrastructures76. We therefore use this “Moore’s Law” decrease as a long term average, following the cost evolution in other fields of information technology. Note that in reality the cost decreases are likely to be more sporadic, with periods of relatively stable prices followed by precipitous drops. 7. Assume a relatively modest financial support level for network engineering77 at a level evolving to 2 FTEs, at an annual cost of $163k per FTE (including indirect costs), and an annual salary index of 4%. These FTE support levels and costs per FTE are low compared to the cost of network engineers on the open market. Also assume modest support at the level of 1.5 to 2 FTEs for VRVS, which is the minimum level required for the current level of usage according to recent experience78. 8. Reach a constant DOE funding level during LHC operation from 2008 onwards. This assumes that non-DOE sources contribute a substantial portion of the overall transatlantic link cost, as is the case now. In 2007 and beyond, it is assumed that the bandwidth will continue to increase, but at a reduced rate corresponding to continued funding at the 2007 level, in 2007 constant dollars.

The proposed funding profile, which represents the best current estimate of what is required to meet HENP’s network needs, is summarized in the following plot. Apart from the charges for leasing the transatlantic link, there are significant charges for “Infrastructure” which includes the required network hardware (routers, switches, optical fibers, and interfaces), rental costs for placing and maintaining a rack at the point of presence, connections to the general purpose Internet, a modest amount for salaries of network support engineers, and maintenance (24 hour/7 day per week/4 hour response time) at the termination point of the link. Note that in contrast to the bandwidth costs, network routing and switching equipment costs have not come down as quickly as foreseen in 2000-2, and so these equipment costs shown are represent a tight budget for managing the 10 Gbps link. As mentioned above, however, the overall costs for this year are slightly lower than last year’s estimates for 2004. The budgets below include some necessary funding for “research” links at higher bandwidth, which are based on the use of one wavelength (or a subchannel within one wavelength) on a fiber carrying many wavelengths79. In contrast to the traditional services which, until now, are based fully on redundant SONET or SDH80 links, the wavelength services are typically “unprotected”. This means that wavelength-based services today do not come equipped with a “hot backup” line that cuts in immediately in case of a failure of the primary link. For this reason, the wavelength-based services are typically 50-75% less expensive than fully redundant SONET or SDH services. The projected annual costs for the US-CERN line are summarized in Figure 9.9. Note that in 2002,

76The onset of this era seems to have been delayed by the sell-off of assets described in the text. See “Busted by Broadband”, Time Magazine March 26, 2001 at http://www.time.com/time/magazine/article/0,9171,1101010326-1-2923,00.html. 77For tasks including network configuration, monitoring, troubleshooting, installations and upgrades at vendor points of presence; some help with hardware installations and/or configuration and troubleshooting at various HEP sites. Work in coordination with the CERN External Networking Group, the Internet2 and Regional Network engineering teams, and the groups developing and operating Data Grids. 78Significantly expanded use by HENP would require additional funding for support. We assume here that any large scale use of VRVS outside of HENP will be funded by sources other than HENP. 79Using Wavelength Division Multiplexing (WDM). For an explanation see for example: ftp://ftp.netlab.ohio-state.edu/pub/jain/courses/cis788-99/h 5opt2.pdf 80SONET and SDH are a set of related standards for synchronous data transmission over fiber optic networks. SONET is short for Synchronous Optical NETwork and SDH is an acronym for Synchronous Digital Hierarchy. SONET is the United States version of the standard published by the American National Standards Institute (ANSI). SDH is the international version of the standard published by the International Telecommunications Union (ITU). See http://www.techfest.com/networking/wan/sonet.htm 9.6 LHCNet Funding Plan for the US-CERN Links 2003-2007 209 we requested $ 2.24 M and received $ 2.00 M. After delaying purchases and the upgrade of the link for a few months, the projected network costs for 2003 are $ 2.07 M. In order to make up for the shortfall, the request for FY2004 is $ 2.41 + 0.07 = 2.48 M. The costs for the “Infrastructure” listed above are broken down by category and year in the following Figure 9.10, and Table 9.2.

Figure 9.9: Projected bandwidth and annual costs for the US-CERN line.

LHCNet Costs and US - CERN Line Bandwidth 40 Gbps 4.00 20 Gbps 10 Gbps 3.00 5 Gbps 0.6 Gbps 2.5 Gbps M$ 2.000.15-.3Gbps

1.00

0.00 2001 2002 2003 2004 2005 2006 2007 Infrastructure (M$) 0.67 0.91 0.97 1.21 1.47 1.51 1.69 TA Link and Internet 0.85 1.09 1.10 1.20 1.49 1.85 2.29 Access Costs (M$)

Table 9.2: LHCNET Infrastructure Budget Breakdown by category for 2001-2006 (M$) 2001 2002 2003 2004 2005 2006 2007 Routing and Switching Equipment 0.150 0.270 0.270 0.375 0.400 0.400 0.425 Salaries for Network + Support Engineers 0.370 0.465 0.530 0.593 0.793 0.825 0.953 VRVS Reﬂectors, Workstations, Interfaces 0.035 0.035 0.040 0.045 0.050 0.050 0.052 Network Perf & Monit & Test Systems 0.030 0.040 0.040 0.060 0.075 0.075 0.075 HW and Software Maint.; Collocation 0.065 0.075 0.075 0.095 0.100 0.110 0.130 Travel 0.020 0.025 0.025 0.045 0.050 0.050 0.050 TOTAL INFRASTRUCTURE COST 0.670 0.910 0.980 1.213 1.468 1.510 1.685 210 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

Figure 9.10: Breakdown of annual infrastructure costs for the US-CERN line. 1.800 2001 2002 1.600 2003 2004 1.400 2005 2006 1.200 2007

1.000 M$ 0.800

0.600

0.400

0.200

0.000 Routing and Salaries for Network + VRVS Reflectors, Network Perf. HW and Software Travel TOTAL Switching Equipment Support Engineers Workstations, Monitoring and Test Maint.; Colocation INFRASTRUCTURE COST Interfaces Systems 9.7 Upcoming Advances in Network Technologies; National Lambda Rail 211

9.7 Upcoming Advances in Network Technologies; National Lambda Rail

There are now strong prospects for breakthrough advances in a wide range of network technologies within the next one to ﬁve years, including:

• Optical fiber infrastructure: Switches, routers and optical multiplexers supporting multiple 10 Gigabit/sec (Gbps), 40 Gbps wavelengths and possibly higher speeds; a greater number of wavelengths on a fiber; possible dynamic path building • New versions of the basic Transport Control Protocol (TCP), and/or other protocols that provide stable and efficient data transport at speeds at and above 10 Gbps (as discussed above) • Ethernet at 10 Gbps speeds end-to-end, across both wide and local area networks; 10 Gbps Ethernet network interfaces on servers and eventually PCs; Ethernet at 40 Gbps or 100 Gbps • Mobile handheld devices with I/O and wireless network speed, and computing capability to support persistent, ubiquitous data access and collaborative work

Two of the major projects currently under development that make use of the DWDM technology mentioned above in the US are the TeraGrid81 “National Lambda Rail”, shown in the following Figure. TeraGrid, National Light Rail, and the earlier development of dark ﬁber links supporting wavelengths among customer-owned networks in CA*net4 in Canada, could mark an important change: from an era of managed bandwidth services, to one where the research and education community itself owns and shares the cost of operating the network infrastructure. In the new scheme, the costs of adding additional wavelengths to the infrastructure, while still signiﬁcant, are much lower than was previously thought to be possible. These and other developments, discussed further in the report of the Advanced Technologies Working Group of the SCIC82 are beginning to trigger a rapid shift in the way the HENP community uses networks in support of its data-intensive science, while at the same time triggering further advances in Grid technologies and distributed information systems.

9.8 Future Vision: HENP Networks in 2005-2013; Petabyte- Scale Grids with Terabyte Transactions

Given the continued rapid decline of network prices per unit bandwidth, and the technology developments summarized above, a shift to a more dynamic view of the role of networks began to emerge during the last year, triggered in part by planning for and initial work on the “Grid-enabled Analysis Environment”83. This has led in turn to a more dynamic view of the Grid systems being prepared for the LHC experiments, as managed systems where physicists at remote locations could, beginning a few years from now, extract Terabyte-sized subsets of the data from the multi-petabyte data stores on demand, and if needed rapidly deliver this data to local or regional facilities for further processing and analysis. The recent developments in networks and protocols discussed in this chapter make it clear that it will soon be practical to deliver these Terabyte scale collections of data in a short “transaction” lasting

81See http://www.teragrid.org 82See http://cenr.ch/icfa-scic 83As discussed in the CMS/L3 Chapter of this report. 212 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

SEA

POR

SAC NYC BOS OGD CHI SVL DEN CLE FRE PIT WDC KAN RAL STR NAS LAX PHO WAL ATL SDG OLG

DAL JAC NLR Buildout Started 15808 Terminal, Regen or OADM site November 2002 Initially 4 10 Gb Fiber route Wavelengths Transition beginning now to optical, multi-wavelength R&E networks. To 40 10Gb Also Note: IEEAF/GEO plan for dark fiber in Europe Waves in Future

Figure 9.11: Planned layout of the optical fiber route (from Level(3)) and the Cisco optical multiplexers, of National Lambda Rail. Construction in the western part of the US began in November 2002. minutes, rather than hours. This would enable the distributed Grid resources to be used more effectively by major HENP experiments, while also making physics groups remote from the experiment better able to carry out competitive data analyses. Completing these data-intensive transactions in just minutes would increase the likelihood of the transaction being completed successfully, and it would substantially increase the physics groups working efficiency, by relieving them of the need to monitor and follow the transfers, and to redo them later in the day (or week) if they did not complete. Such short transactions also are necessary to avoid the bottlenecks and fragility of the Grid system that would result if hundreds to thousands of such requests were left pending for long periods, or if a large backlog of requests was permitted to build up over time. It is important to note that transactions on this scale, while still representing very small fractions of the data, correspond to throughputs across networks of 10 Gbps and up. A 1000 second-long transaction shipping 1 TByte of data corresponds to 8 Gbps of net throughput. Larger transactions, such as shipping 100 TBytes between Tier1 centers in 1000 seconds (which may become practical a few years after LHC startup), would require 0.8 Terabits/sec (comparable to the capacity of a fully instrumented fiber today). These considerations, and the current trend of both network vendors and academic and research organizations (such as National Lambda Rail and CA*net4) to move to multi-wavelength networks with much higher aggregate link capacities, led to a roadmap for HENP networks in the coming decade, shown in Figure 9.12. Using the US-CERN production and research network links as an example of the possible evolution of major network links in our field, the roadmap shows progressive upgrades every 2 to 3 years, going from the present 2.510 Gbps range to the Tbps range within approximately the next 9.9 ICFA SCIC 213

10 years84. The column on the right shows the progression from static bandwidth provisioning (up to today), to the future use of multiple wavelengths on an optical ﬁber, and the increasingly dynamic provisioning of end-to-end network paths through optical circuit switching.

Year Production Experimental Remarks

2001 0.155 0.622-2.5 SONET/SDH

2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.

2003 2.5 10 DWDM; 1 + 10 GigE Integration

2005 10 2- 4 X 10 λ Switch; λ Provisioning

2007 2-4 X 10 ~10 X 10; 1st Gen. λ Grids 40 Gbps

2009 ~10 X 10 ~5 X 40 or 40 Gbps λ or 1-2 X 40 ~20-50 X 10 Switching

2011 ~5 X 40 or ~25 X 40 or 2nd Gen. λ Grids ~20 X 10 ~100 X 10 Terabit Networks

2013 ~Terabit ~MultiTbps ~Fill One Fiber

Figure 9.12: A Roadmap for major links in HENP networks over the next ten years. Future projections follow the trend of bandwidth pricing improvements of the last decade: by a factor of several hundred in performance at equal cost every 10 years.

9.9 ICFA SCIC

ICFA’s involvement in wide area networking for HEP began with its visionary “Statement on Commu- nications in International High Energy Physics Collaborations85” that concluded

ICFA urges that all countries and institutions wishing to participate even more eﬀectively and fully in international high energy physics collaborations should: • review their operating methods to ensure that they are fully adapted to remote participation • strive to provide the necessary communication facilities and adequate international bandwidth.

84This represents a factor of 1000 improvement in capacity over ten years. For comparison, the speed of the US- CERN network connectivity increased by approximately a factor of 5,000 over the last 10 years, and only 200 (prior to deregulation in the US and Europe) in the decade before that. 85The complete text of this statement may be found at http://www.fnal.gov/directorate/icfa/icfa communicaes.html 214 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

With the rising dependence on and demand for high performance international networks, ICFA commissioned a Network Task Force86 in 1997, where H. Newman chaired the Requirements Working Group. In 1998 ICFA established the Standing Committee on Inter-regional Connectivity87 charged to:

Make recommendations to ICFA concerning the connectivity between America, Asia and Europe. As part of the process of developing these recommendations, the committee should monitor traﬃc, keep track of technology developments, periodically review forecasts of future bandwidth needs, and provide early warning of potential problems.

H. Newman served on the SCIC from its inception, representing the US HENP community on behalf of the APS DPF. He was appointed to ICFA and took over as Chair of the SCIC in February 2002. The past year has been a period of intensive activity for the SCIC. The Committee met 9 times, and presented its progress and ﬁndings during the ICFA meetings of the Laboratory Directors (February 2002), at ICHEP02 in Amsterdam in July and the ICFA Seminar at CERN in October, as well at the LISHEP (Rio; 2/02) and APAN (Tokyo 1/03) workshops, at the Pan-European Ministerial Meeting for the World Summit on the Information Society (11/02). In the March 2002 SCIC meeting, working groups were formed in order to prepare a major series of reports, that was submitted and presented to ICFA in February 2003. It was decided that the SCIC focus for 2002-3 would be the “Digital Divide” that separates scientists in the less economically-favored world regions from those in the most favored regions (namely the US, western Europe and Japan), since this is critical issue for our global physics collaborations88. The Working Groups and their chairs are currently:

• Monitoring: Les Cottrell (SLAC), with R. Hughes-Jones, S.Ilyin (Moscow), S. Ravot(Caltech), F. Yuasa(KEK), D. Davids (CERN), S. Berezhnev(RUHEP) • Advanced Technologies: R. Hughes-Jones (Manchester), with H. Newman, S. Ravot, O. Martin, V. Korenkov (Dubna) • The Digital Divide: A. Santoro (UERJ, Rio), with S. Ilyin, Y. Karita (KEK), V. White (FNAL), J. Ibarra and H. Alvarez (AMPATH), D.O. Williams (CERN), S. Novaes (Sao Paolo), D. Son (Ko- rea), H. Hoorani and S. Zaidi (Pakistan), S. Banerjee (India), H. Newman • Key Requirements: H. Newman (Caltech), with C. Young (SLAC)

The working groups prepared ﬁve reports, that were presented by H. Newman to ICFA at its February 2003 Directors’ meeting89:

• Networking for High Energy and Nuclear Physics • Digital Divide Executive Report • Report on the Digital Divide in Russia • Advanced Technologies Interim Report90 • Network Monitoring Report

The general conclusions, recommendations and perspective on the Digital Divide in the SCIC reports of February 2003 are summarized in the following subsections of this report.

86Chaired by D. O. Williams of CERN, see http://davidw.home.cern.ch/davidw/icfa/icfa-ntf.htm 87See http://www.hep.net/ICFA/index.html 88It was also decided that in general, the SCIC will choose a diﬀerent topical subject each 1-2 years to present as its major focus, and to make clear in its reports where ICFA might help (by approaching a government, funding agency, ISP, etc.) 89These reports and the presentation to ICFA, as well as minutes, presentations and associated documents, may be found at the SCIC Web site: http://cern.ch/icfa-scic. 90A further report is due in the Summer of 2003 9.9 ICFA SCIC 215

The SCIC is currently continuing its work on the Digital Divide, on monitoring and sharing information on the world networks, and on sharing information and developing advanced network technologies. We are planning the ﬁrst Workshop on the Digital Divide in Rio in February 2004. Organizational meetings for this event will start in July 2003.

9.9.1 General Conclusions of the 2003 SCIC Reports

The February 2003 SCIC Reports concluded:

• The bandwidth of the major national and international networks used by the HENP community, as well as the transoceanic links is progressing rapidly and has reached the 2.5 10 Gbps range. This is encouraged by the continued rapid fall of prices per unit bandwidth for wide area networks, as well as the widespread and increasing affordability of Gigabit Ethernet. • A key issue for our field is to close the Digital Divide in HENP, so that scientists from all regions of the world have access to high performance networks and associated technologies that will allow them to collaborate as full partners: in experiment, theory and accelerator development. • The rate of progress in the major networks has been faster than foreseen (even 1 to 2 years ago). The current generation of network backbones, representing a typical upgrade by a factor of four in speed, arrived in the last 15 months in the US, Europe and Japan. This rate of improvement is 2 to 3 times Moores Law. This rapid rate of progress, confined mostly to the US, Europe and Japan, threatens to open the Digital Divide further, unless we take action. • Reliable high End-to-end Performance of networked applications such as large file transfers and Data Grids is required. Achieving this requires: – End-to-end monitoring extending to all regions serving our community; a coherent approach to monitoring that allows physicists throughout our community to extract clear, unambiguous and inclusive information is a prerequisite for this. – Developing and deploying high performance (TCP) toolkits in a form that is suitable for widespread use by users. Training the community to use these tools well, and wisely, also is required. – Removing local, last mile, and national and international bottlenecks end-to-end, whether the bottlenecks are technical or political in origin.

9.9.2 Recommendations of the February 2003 SCIC Reports

• ICFA should work vigorously locally, nationally and internationally, to ensure that networks with suﬃcient raw capacity and end-to-end capability are available throughout the world. This is now a vital requirement for the success of our ﬁeld and the health of our global collaborations.

• The SCIC, and where appropriate other members of ICFA, should work in concert with other cognizant organizations as well as funding agencies on problems of global networking for HENP as well as other ﬁelds of research and education. The organizations include in particular Internet2, Terena, AMPATH; DataTAG, the Grid projects and the Global Grid Forum.

It was recognized that HENP and its worldwide collaborations could be a model for other scientiﬁc disciplines, and for new modes of information sharing and communication in society at large. The provision of adequate networks and the success of our Collaborations thus could have broader implications, that extend beyond the bounds of scientiﬁc research. 216 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

It was further recognized, however, that the world community will only reap the benefits of global collaborations in research and education, and of the development of advanced network and Grid systems, if we are able to close the Digital Divide. ICFA members thus have been requested to work with and advise the SCIC on closing this Divide country by country, and region by region91. The Digital Divide problem is illustrated in Figure 9.13. The figure92 shows that the maximum achievable throughput in each world regions is progressing rather rapidly, by a factor of 100 approximately every 8 years. But the lines for the different regions are parallel, showing that the less-favored regions are not catching up, with the time-lag between the best and worst-off regions in the graph remaining in the range of 6 to 8 years93.

Figure 9.13: The maximum achievable TCP throughput from SLAC to groups of sites in diﬀerent world regions, versus time. The fact that the lines are parallel indicates that the less favored regions are remaining far behind in their network capability.

As reviewed elsewhere in this chapter, the recent rapid advances by our team and others in the US and Europe are much faster than this rate, and this will tend to open the Digital Divide further. Some speciﬁc approaches to closing the Digital Divide are:

• Identify and work on speciﬁc network problems, which may be related performance limitations on existing national, metropolitan and local network infrastructures, such as last mile problems, political problems, or a lack of coordination among diﬀerent organizations.

• Create and encourage inter-regional programs to solve speciﬁc regional problems. Leading examples include the Virtual Silk Highway project (http://www.nato.int/science/e/silk.htm) led by DESY, the support for links in Asia by the KEK High Energy Accelerator Research Organization in Japan (http://www.kek.jp), and the support of network connections for research and education

91Key examples where work is going on include China, India, Russia, Pakistan, Brazil, Korea and Romania, as well as other areas of South America, Central and Southeast Asia, Southeast Europe, and Africa. 92From SLAC’s Internet End-to-end Performance Monitoring (IEPM) initiative. 93Note that the graph for Africa, which is the farthest behind, is not shown. The IEPM project, working with ICTP Trieste, is just beginning to acquire and systematize that data. 9.9 ICFA SCIC 217

in South America by the AMPATH “Pathway to the Americas” (http://www.ampath.ﬁu.edu ) at Florida International University.

• Make direct contacts, and help educate government oﬃcials on the needs and beneﬁts to society of the development and deployment of advanced network infrastructure and applications: for research, education, industry, commerce, and society as a whole.

• Use (lightweight; non-disruptive) network monitoring to identify and track problems, and keep the research community (and the world community) informed on the evolving state of the Digital Divide. One leading example in the HEP community is the Internet End-to-end Performance Monitoring (IEPM) initiative (http://www-iepm.slac.stanford.edu) at SLAC94

• Share and systematize information on the Digital Divide. The SCIC is gathering information on these problems and developing a Web site on the Digital Divide problems of research groups, universities and laboratories throughout its worldwide community. This will be coupled to general information on link bandwidths, quality, utilization and pricing. Monitoring results from the IEPM will be used to track and highlight ongoing and emerging problems. Speciﬁc aspects of information that will help develop a general approach to solving the problem include:

– Examples of how the Divide can be bridged, or has been bridged successfully in a city, country or region. One class of solutions is the installation of short-distance optical fibers leased or owned by a university or laboratory, to reach the “point of presence” of a network provider. Another is the activation of existing national or metropolitan fiber-optic infrastructures (typically owned by electric or gas utilities, or railroads) that have remained unused. A third class is the resolution of technical problems involving antiquated network equipment, or equipment configuration, software settings, etc. – Making comparative pricing information available. Since international network prices are falling rapidly along the major Transatlantic and Transpacific routes, sharing this information should help us set lower pricing targets in the economically poorer regions, by pressuring multinational network vendors to lower their prices in the region, to bring them in line with their prices in larger markets.

• Create a new “culture of collaboration”, where the major experiments and the HENP laboratories are geared towards full participation by physicists located around the world95

• Work with the Internet Educational Equal Access Foundation (IEEAF)96, and other organizations that aim to arrange for favorable network prices or outright bandwidth donations, where possible.

• Prepare for and take part in the World Summit of the Information Society97. The WSIS will take place in Geneva in December 2003 and Tunis in 2005. HENP (and Caltech in particular) has been involved in preparatory and regional meetings in Bucharest in November 2002, the National Academies in December 2002, and Tokyo in January 200398.

94It is vital that support for the IEPM activity in particular, which covers 79 countries with 80% of the world population be continued and strengthened, so that we can monitor and track progress in network performance in more countries and more sites within countries, around the globe. 95Starting with collaboration and laboratory policies, as discussed in the SCIC reports. 96http://www.ieeaf.org 97WSIS; http://www.itu.int/wsis/ 98The WSIS process aims to develop a society where “highly-developed networks, equitable and ubiquitous access to information, appropriate content in accessible formats and effective communication can help people achieve their potential ”. These aims are clearly synergistic with the aims of our field, and its need to provide worldwide access to information and effective communications. 218 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

As a result of these efforts on closing the Digital Divide, HENP has been recognized widely in the past year, as having relevant experience in effective methods of initiating and promoting international collaborations, and in harnessing or developing new technologies and applications to achieve these aims. Caltech was invited, on behalf of the HENP community and the US and European Grid projects, to run a session on The Role of New Technologies in the Development of an Information Society last November, and to take part in the planning process for the Summit itself. CERN is planning a scientific event (the Research Summit on the Information Society) shortly before the Geneva Summit in December. Caltech is working with CERN on this event, and the US State Department’s Telecommunications Advisory Council on preparations for the WSIS.

9.10 VRVS: Collaborative Tools for LHC and the HENP Com- munity

9.10.1 Introduction; Overview

The Caltech “Virtual Rooms Videoconferencing System” (VRVS)99 has become a standard part of the toolset used daily by a large sector of HEP, and it is used increasingly for other DoE-supported programs. It has also attracted substantial interest in diverse fields of science and engineering outside the HEP. A new version of VRVS (V3.0) has been successfully deployed in mid-February 2003. Already more than 3,000 users from more than 61 countries have registered and used the new VRVS system. There are currently 57 “reflectors” that create the interconnections and manage the traffic flow, in the Americas, Europe and Asia. New reflectors recently have been installed in Brazil, Romania, Portugal, and 5 new reflectors have been deployed in several US universities using the Internet2 Backbone. In addition to improving dramatically the scalability and usability, the new release brings new features and capabilities (e.g. authentication via encrypted username and password, a powerful database for handling thousands of VRVS users, and more). The additional features are described in the following paragraphs. The VRVS system is managed by the Caltech CMS group. Since last year, the number of multipoint collaborative sessions (national and international) using VRVS increased by 180% mainly pushed by the successful transition to the new VRVS version, where a nearly unlimited set of virtual rooms is available. This high adoption rate confirmed the need within the Research and Academic community for easy- to-use collaborative tools with high performance. Integration with the AccessGrid technology100 has been successful, and the VRVS/AG gateway (VAG, or “Virtual Access Grid”), where one can join any Access Grid Virtual Venue from a desktop or laptop, is heavily used by the community. The development of the next generation of the VRVS system is currently underway; details are described in latter subsections of this report. The relationship with Internet2 and other international organizations has been consolidated. As a software-based collaborative infrastructure, VRVS is now recognized as the only application capable of scaling to provide future collaboration services throughout these organizations.

99See http://www.vrvs.org. 100See http://www.accessgrid.org 9.10 VRVS: Collaborative Tools for LHC and the HENP Community 219

9.10.2 VRVS Deployment Status

Since the release of VRVS 3.0 we have had an average of more than 30 users registering each day. As a tribute to the ease of registering and the usefulness of VRVS, in less than 3 months we have had more than 3000 users register and use VRVS. By the end of the year we expect to have 7500 users register. Over the past year we have added 14 new “reflectors” (Linux PCs that manage and direct the traffic streams) to bring the total to 57. The reflectors are located in the following countries: United States (19), Brazil (4), Canada (2), Czech Republic, Finland, Germany, Greece (2), Hungary, Ireland, Israel (2), Italy, Japan (2), Poland, Portugal (2), Romania, Russia, Slovakia, Spain (5), Switzerland (3), Taiwan (2), United Kingdom (3), and Venezuela. A map of the distribution of the reflectors is shown in figure 9.14.

Figure 9.14: Worldwide Distribution of the 57 VRVS Reﬂectors.

With the previous version of VRVS (2.5) we often reached the maximum number of possible bookings. As can be seen in Figure 9.15 from October 2001, we had reached a practical limit of approximately 1300 hours of meetings booked per month. Since the release of VRVS version 3.0 in mid-February 2003, the number of hours booked monthly is once again rising steeply.

Figure 9.15: VRVS Hours Booked per month, in 2000-2003.

Version 3.0 has full support for Mac OS X, as has been requested for a long time by the VRVS community. (Figure 9.16) shows that the Macintosh is already used as the client platform by a signiﬁcant 220 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP fraction of the VRVS user community, and this fraction is rising (since there was very limited support for Mac under VRVS 2.5).

Figure 9.16: VRVS Client Platform Usage.

9.10.3 The Transition to Version 3.0

The transition from version 2.5 to 3.0 went very smoothly. In the course of a weekend we successfully transferred all the bookings from 2.5 to 3 and moved the new server into production, so that on the Monday hundreds of users easily registered with and used the new system. For a few months now VRVS 3.0 has been the production system. Its key points are robustness and scalability: up to thousands of users connected to hundreds of meetings at the same time. The new design of the system enables a comfortable integration of new tools, clients and emerging standards. In addition, in a few months a dynamic network topology of reflectors (already tested, and reported on in a following subsection) will route the conferences through the best path, in realtime and in a transparent way to the end user. The VRVS 3.0 system has been built practically from scratch, only reusing ∼ 5% of the VRVS 2.5 code. In this new version, the whole system turns around a relational database, that we can consider the kernel of the system. The use of a real database has sped up the system enormously, enabling the support of thousands of users in parallel. Together with this, the Administration Interface has been completely rebuilt using Java technologies and implementing new services. At the same time, this part of the system has been secured and SSL encrypted. Access to this site is restricted to only a few administrators that, with enhanced tools can provide user support, check and export statistics, monitor, control or even extend the system from anywhere, easily and quickly. As an example, they can add, remove, edit or modify information related to users, reflectors, meetings that are being performed, communities of users, etc. We can now also export statistics to other formats (like Excel) with one single click. Current developments in these interfaces include the integration of new tools that will allow the monitoring of reflectors parameters in real-time. Data from reflectors like their bandwidth consumption, CPU load, CPU used by users or processes, packet transmission delays, IO traffic, among others will be soon monitored through the Administration site. As scalability is one of the key features of the system, we have reorganized VRVS in different levels. Depending on their use, the reflectors are arranged in different and disconnected topologies so that high bandwidth traffic used by some organizations (like AccessGrid) does not interfere with the other VRVS topologies. At the same time, users are distributed in communities, and these communities each hold a different number of Virtual Rooms, so space for booking a room is no longer a problem. All these changes come joined to a new design and better look and feel of the website. This makes use of the system easier, and allows new users to get familiar with the system more quickly. 9.10 VRVS: Collaborative Tools for LHC and the HENP Community 221

The installation of the software packages have also been improved on most of the platforms. End users benefit now from installations mechanisms such as InstallShield for Windows, RPMs for Linux and tar files under other platforms. Moreover, VRVS fully supports Mac OS X now, through the common MBone tools (VIC and VAT). The installation procedures, like any other aspect of the system, are fully documented on the web. The VRVS Booking System, where users reserve their meetings, has also been enhanced. Thanks to the new booking wizard, the user no longer needs to specify which Virtual Room he wants to book, but only the date and time of the meeting. Another improvement is the ability to extend the duration of an ongoing meeting in real time. Also, the users can easily copy a particular booking to the next or future days with a few mouse clicks. In our previous version the time zone handling was not complete, and was able only to compute local times if the time zones involved differed by an integral number of hours. The new VRVS version uses a table of all time zones in the world, and is precise enough to compute local times where the time zone differences involve a fraction of an hour: India, for example, uses GMT +5:30. An automatic process can detect “Summer” (or Daylight) time for any part of the world. The user sets his time zone in his VRVS profile but he can, at any time, dynamically select another time zone to see what time the meeting will be for his colleagues in another part of the world. All the times and dates involved in scheduling a meeting are computed automatically, and shown as local times to a user in any time zone who checks the schedule, and during a meeting. Another advance in Version 3.0 concerns the user profiles. The system is now user-oriented rather than machine-oriented, which helps support users who move between time zones. From now on, users have their own login username and password, so they can log in to VRVS from any machine, while the system always keeps their settings and preferences. This also means that users do not have to re- register each time they move to a different machine. When the user joins a Virtual Room to participate in a conference, the system calculates automatically, and in a transparent way, the best reflector to be attached to. This association is more accurate now, thanks to a set of rules that are defined in the database that allows the system to attach the users machine to the best/closest available reflector at any time. In the reflectors software there are many improvements as well. The connection between peer reflectors is now done using tunneling though one single port to simplify the configuration of firewalls and improve the security of the sites. At the same time, VRVS now supports the use of machines that are behind a Network Address Translation (NAT). This is done through the installation of one of these new and enhanced reflectors in the sites that contain NATs. Furthermore, the sharing service presented in VRVS through the VNC application is more efficient now. A dedicated reflector has been created to deal with the distribution of the sharing data generated by the VNC application. In a further step, all this sharing data will flow through the global topology of reflectors using multiple paths, that are monitored and optimized in real time. All these aspects of the system have been integrated gradually and smoothly. For a few months the VRVS 2.5 and 3.0 versions were running in parallel. During that time, the VRVS users were able to test the new system, get used to it and to its new features. At the same time, we were able to fix the small problems that were detected and reported by the users. At the end of this transition period, the switch was done and the result was a near-perfect migration of thousands of users from two completely different systems, without any major problem. 222 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

9.10.4 Next Generation Systems: Version 3.n and 4: Milestones and Sched- ule

The VRVS system (and its different releases) has now been running in production during the last five and a half years, offering the HENP community a working and reliable tool for collaboration within groups and among physicists dispersed world-wide. The next generation of collaborative systems (based on current VRVS development) will evolve to be more scalable and reliable, will integrate new emerging standards, will support new products to provide very high-end quality, and will provide state-of-the- art efficiency in terms of quality per unit bandwidth. More specifically, the VRVS development and enhancement includes the integration of IPv6, Wireless Technologies, SIP101, multiple forms of Security (Firewall, NAT, Authentication, Encryption), High quality interactive sessions (MPEG4, HDTV), real time monitoring and system optimization, and alarm notification tools (using Java software agents, Web services, JINI and RMI technologies). At a later date we will also consider migrating some of the services to the Open Grid Services Architecture (OGSA102). The development effort for the coming year will focus on four main areas, to:

(1) Design and architect a powerful monitoring, system management and alarm notiﬁcation toolset based on the MonALISA system103 developed at Caltech, to be able to oﬀer an extremely reliable VRVS service to the community. Due to the high scalability of the new VRVS infrastructure and the current exponential usage growth, such monitoring tools are essential.

(2) Integration of the SIP protocol into VRVS. This is pushed by the strong adoption of SIP by Microsoft and the use of IP telephony (3) Improved Security mechanism (ﬁrewall, encryption, authentication) (4) Continued investigation of new and emerging technologies, including: MPEG4 and/ot HDTV codecs, various forms of Wireless (3G or 4G; 802.11g, 802.16 when available), new graphics cards, etc.

This effort will be in conjunction with the ongoing VRVS deployment and support, and the participation in several international demonstrations and conferences. In the meantime, some short term features/enhancement of the current production system are planned that will be considered as minor releases, of Versions 3.n. We are currently working on, and soon plan to provide (1) the possibility to have a chairman of the session with the privilege to mute/unmute the video/audio of any participant(s), decide who is the speaker and which video should be received by the remote participants; (2) a troubleshooting wizard program that could be run on the local machine of the user and determine if the machine is correctly configured to run VRVS, or help diagnose problems; (3) the possibility for the user sharing his desktop to grant or ungrant any remote participant(s) control of his desktop on the fly (4) the ability to remotely control the position (zoom, pan, tilt) of appropriately equipped cameras, etc. Our major near-term milestones are summarized below:

• May 2003: Integrate and continue to develop monitoring tools to deploy in the reflector nodes. • June 2003: Deploy reflector software NAT/Firewall beta version in the production network. Demonstrate prototype of multi-screen affordable video wall set-up based on PC and multi-monitor graphic cards.

101The Session Initiation Protocol; see http://www1.cs.columbia.edu/sip/ 102See http://www.globus.org/ogsa/ 103see http://monalisa.cern.ch/MONALISA/ 9.10 VRVS: Collaborative Tools for LHC and the HENP Community 223

• July 2003: Demonstrate SIP alpha version interoperability. • August 2003: Integrated automatic downloading/upgrade of VRVS software to the reflector using encryption/authentication for security. • September 2003: Start to develop advanced Monitoring and Tracking tools to manage several hundred simultaneous sessions and users; Start to port all code to be IPv6 compliant. • October 2003: Demonstration at Internet2 fall member meeting conference; Start integration of encryption communication capability between reflectors. • November 2003: Demonstration of full monitoring and tracking console in conjunction with new peer-to-peer reflector network. • December 2003: Develop and start to implement a model to incorporate web services technology in VRVS. Part of the web server functionality should move to the reflector nodes. Demonstrate interoperability with handheld devices. • January 2004: Provide a powerful statistics/tracking toolset for monitoring VRVS usage (per user/ per reflector, per hour,etc.) • February 2004: Start to port all code to be IPv6 compliant. Integrate automatic best routing algorithm between reflectors. • March 2004: Demonstrate Notification and Alarm Trigger events over the global VRVS network • April 2004: Demonstrate new high end videoconferencing services developed by VRVS based on MPEG and/or HDTV technologies. • May 2004: Use the dynamic Virtual Room technology to provide H.323 point-to-point connections. Provide the possibility to extend an initial bi-directional communication session to 3 or more participants, on the fly.

9.10.5 Handheld VRVS

As with all computer technologies today, handheld computers are becoming more and more powerful. They are now able to run the software codecs needed to process the real-time audio and video streams needed for a ﬂuid videoconference. Wireless connections are also becoming faster. With 802.11a (and the upcoming 802.11g) able to send 54 Mbps (up to 24 Mbps in practice) as opposed to 802.11b which can send only 11 Mbs (to 4-5 Mbps in practice) sending full audio and video streams over a wireless connection is now practicable. More cameras for handheld computers are coming on the market every day. These cameras work at faster frame rates and higher resolutions than before, and more videoconferencing clients for handheld computers are becoming available. There are now H.323 and SIP clients for handheld computers which could be integrated into VRVS. This would make VRVS more of a ubiquitous tool for physics collaborations at work. A prototype of a VRVS audio/video client that runs on a Pocket Figure 9.17: VRVS Client. PC has been implemented (Figure 9.17). It supports the standard H.261 CIF video and G.711 µ-Law audio. We still need to construct interfaces to run on the smaller screen space but include all the controls we have for all the clients (meeting scheduler, chat, etc). Especially vital for handheld videoconferencing are the video controls. It is only practical to display 224 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP one video at a time because of the limited screen space of a handheld computer (usually just “CIF” size, namely 288 by 352 pixels). Being able to choose easily which video of the participants in the meeting to view (using shortcuts) could be important in the future, especially when using a handheld Virtual Access Grid (VAG; see the next section) in which each venue can hold 30 or more videos. To help enable seamless collaboration we can now allow mobile individuals to connect as a group in a meeting. Using wireless handheld computers with audio and video capabilities anyone in a group will be able to participate in a meeting at any time anywhere.

9.10.6 VRVS AG Gateway - Full Connectivity to Access Grid

With VRVS 3.0, the VAG (VRVS AG Gateway; or Virtual Access Grid) has been improved based on users’ experience and feedback. The new VAG has full connectivity to Access Grid and full functionality. VAG give users a minimal learning curve, and supports combines multicast-and-unicast collaborative sessions. The VAG has been shown to support a full Access Grid session on a laptop, consuming a few Mbps of bandwidth (or less, under user control), and can run over 802.11a or 802.11g wireless networks without packet loss. VAG reflectors have been installed in Internet2, and at Argonne National Laboratory, and will be deployed on institutional AG nodes as needed, based on users’ requests. A VAG reflector is functionally identical to other VRVS reflectors, is very easy to configure. To connect to Access Grid Virtual Venues or any multicast videoconferencing, VRVS users only need to login to VRVS 3.0, and within 5 intuitive clicks, the user is ready for a collaborative session that includes both AG and other VRVS participants. VRVS users have the maximum flexibility to choose from UCL Mbone, OpenMash Mbone, H.323, SIP, Quicktime, JMF (Java Media Framework) on various platform including Linux, Windows and Mac OS X. The audio transcoder has been improved to transcode AG linear L16-16-Mono to the ITU H.323 standard G.711µ-Law. An audio mixer feature is implemented to support H.323 audio mixing and avoid blocked video because of a noisy site injecting noise into the session. The new VAG also supports a useful range of video modes, particularly to accommodate VRVS users with limited local network and/or CPU power capacity. Specifically, VRVS provides four video modes: (1) Voice switched - the default mode for H.323 clients, receiving one video stream at a time; (2) Timer switched - one browses through all the video based on preset timer, receiving one video stream at a time; (3) Selected Streams - the default mode for Mbone clients. Click among the video participants to view selected video streams (one or several streams are available), which is useful for limited bandwidth network connections and/or legacy low-power local computing systems; and (4) All Streams - Mbone will receive all the video streams subscribed to the virtual venue multicast address. This is the best mode for full interactivity, if the network will support the data flow. Development of a personal or small group VAG setup prototype is underway at Caltech. VRVS users can get 6400 X 1200 pixels of display space by driving 4 screens with a dual Xeon 2.8 Ghz PC server, for under $8,000. This will support a next-generation analysis environment that includes the usual analysis applications, Grid Views, multiple VRVS windows or a VAG session. 9.10 VRVS: Collaborative Tools for LHC and the HENP Community 225

Figure 9.18: View of a low cost, multipurpose Personal VAG Prototype, under development at Caltech.

9.10.7 VRVS and Internet2

As part of the Internet2 Commons initiative104, Internet2 and the VRVS Caltech team continue to deploy a series of VRVS servers (currently 11), known as reﬂectors, over the Abilene backbone105 and Internet2 Universities to both provide better performance for existing VRVS users, and to facilitate access by new users. There are currently some discussions among the diﬀerent partners of this initiative to move to the next phase, and provide a real VRVS videoconferencing service to the Internet2 community. Some fees using a cost recovery model would be requested in order to support the service, and to ensure that the same or better level of service to the HENP community would continue at no additional cost to DOE. Because of the nature of VRVS, its capability for supporting real-time applications within large communities, and its ability to support both multicast and unicast technologies, other Internet2 Initia- tive and Working Groups expressed a high level of interest in collaborating with VRVS. Such groups are for example, the Internet2 End-to-End Performance Initiative106 as well as the Internet2 QoS and Internet2 Multicast Working Groups. We are also involved in the Global Grid Forum initiative, where new services are starting to be developed, based on the Open Grid Services Infrastructure (OGSI). Among these are services planned to be capable of monitoring real-time activity, and oversee and manage the whole VRVS distributed system. This will make VRVS collaboration a persistent part of the overall standards-based Grid enabled working environment, that is currently being developed by the HENP community.

9.10.8 Real Time System Monitoring, Management and Optimization

Following the release of VRVS V3.0, we started to integrate the MonALISA monitoring service into the VRVS system. MonALISA (Monitoring Agents in a Large Integrated Services Architecture) [20] was adapted and deployed on the VRVS reflectors. Dedicated modules to interact with the VRVS reflectors were developed: to collect information about the topology of the system; to monitor and track the traffic among the reflectors and report communication errors with the peers; and to track the number of clients and active virtual rooms. In addition, overall system information is monitored and reported in real time for each reflector: such as the load, CPU usage, and total traffic in and out. For each VRVS reflector, a MonALISA service is running as a registered JINI service [21]. For the VRVS version the MonALISA service is used with an embedded Database, for storing the results locally, and runs in a mode that aims to minimize the reflector resources it uses (typically less than 16MB of memory and less than 1% system load.)

104more information can be found at: http://commons.internet2.edu 105Abilene is an advanced backbone network that supports the development and deployment of new applications developed within the Internet2 community 106See http://www.internet2.edu/e2e 226 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

A dedicated GUI for the VRVS version was developed as a java web-start client [22]. This GUI provides real time information dynamically for all the reﬂectors which are monitored, as illustrated in Figure 9.19.

Figure 9.19: MonALISA Monitoring Service, showing the real time graphical panels of the web-start client for the jini services.

If a new reflector is started it will automatically appear in the GUI and its connections to its peers will be shown. Filter agents to compute an exponentially mediated quality factor of each connection are dynamically deployed to every MonALISA service, and they report this information to all active clients who are subscribed to receive this information. We implemented a maximum flow algorithm, to optimize the way the reflectors are connected. In the graphical view the maximum flow path is shown with blue lines. The subscription mechanism allows one to monitor in real time any measured parameter in the system. Examples of some of the services and information available, visualizing the number of clients and the active virtual rooms, the traffic in and out of all the reflectors, as well as problems such as lost packets between reflectors are shown in figure. In addition to dedicated monitoring modules and filters for the VRVS system, we developed agents able to supervise the running of the VRVS reflectors automatically. This will be particularly important when scaling up the VRVS system further. In case a VRVS reflector stops or does not answer correctly to the monitoring requests, the agent tries to automatically restart it. If this operation fails twice the Agent will send an email to a list of administrators. These agents are the first generation of modules BIBLIOGRAPHY 227 capable of reacting and taking well defined actions when errors occur in the system. We are developing agents able to provide an optimized dynamic routing of the videoconferencing data streams. These agents require information about the quality of alternative connections in the system and they solve a minimum spanning tree problem to optimize the data flow at the global level. These agents, capable of taking action in the system, may be dynamically loaded but for security reasons must be digitally signed by developers with trusted certificates, declared for each running service. These developments are transforming the VRVS system into the first of a new class of large scale distributed systems with real time constraints. The MonALISA framework is an means of carrying out the development of this system, both in terms of its operational characteristics (heuristic, self-discovering, autonomous) and the relatively short development time required for implementing a distributed monitoring and management system of this scale and complexity. At the same time, the agent-based architecture in MonALISA is able to process and analyze the monitoring information dynamically, and to improve and optimize the way such complex applications operate across wide area networks, while themselves consuming very little processing or network resources. These developments have a broader range of applications, to the global distributed Grid-based systems required for major HENP experiments, and other data-intensive project. Similar MonALISA- based services are going to be used in Abilene, as a mainstream development of the Internet2 End- to-end Performance Initiative, and also an integral part of the proposed UltraLight next generation optical network. This real time system also includes much of the functionality required of the OGSA standardized services planned by the Global Grid Forum in the future.

Bibliography

[1] P. Galvez and H. Newman Standard Conﬁgurations for PC-based Packet and ISDN Videoconferencing Systems, California Institute of Technology, Pasadena, http://vrvs.cern.ch/Proj/pcconf/pc conﬁguration.html, December 1997 [2] Videoconferencing for LHC Experiments. Project Execution Plan URL: http://vrvs.cern.ch/Proj/pep/pep.html, Postscript: http://vrvs.cern.ch/Proj/pep/pep.ps, July 19 97 [3] Ch. Isnard and P. Galvez Videoconferencing Project Interim Status Report, CERN and California Institute of Tec hnology, February 1998 [4] V. Jacobson and S. McCanne, Vic: A Flexible Framework for Packet Video, Lawrence Berkeley Laboratory and University of California, Berkeley, http://www-nrg.ee.lbl.gov/vic, 1996 [5] V. Jacobson and S. McCanne, Vic:Visual Audio Tool, Lawrence Berkeley Laboratory and University of California, Berkeley, http://www-nrg.ee.lbl.gov/vat, 1996 [6] Virtual Rooms Videoconferencing System. http://www.vrvs.org [7] “Networking, Videoconferencing and Collaborative Environments”, P.Galvez, H.Newman, California In- stitute of Technology, Computing in High Energy Physics, Chicago 1998, CHEP98 [8] “A Next Generation Integrated Environment for Collaborative Work Across Internets Networking”, P.Galvez, H.Newman, G.Denis California Institute of Technology, Computing in High Energy Physics, Padova 2000, CHEP2000 [9] “Next Generation of Collaborative Tools over Next Generation Networks”, P.Galvez, H.Newman, G.Denis California Institute of Technology, Computing in High Energy Physics, Beijing 2001, CHEP2001 [10] The RFC2543 for SIP. http://www.ietf.org/rfc/rfc2543.txt 228 9 LHCNET: Wide Area Networking and Collaborative Systems for HENP

[11] Guide to Cisco Systems’ VoIP Infrastructure Solution for SIP. http://www.cisco.com/univercd/cc/td/doc/product/voice/sipsols/biggulp/index.htm [12] Peter Granstrom, Sean Olson and Mark Peck. The Future of communication using SIP. http://www.ericsson.com/about/publications/review/2002 01/ﬁles/2002013.pdf [13] Overview of the MPEG-4 Standard, Rob Koenen, http://mpeg.telecomitalialab.com/standards/mpeg-4/mpeg-4.htm [14] MPEG-4 Codecs Compared, Ben Waggoner, DV June 2002 P16-27 [15] MPEG-4: Why, What, How and When? Fernando Pereira, http://leonardo.telecomitalialab.com/icjﬁles/mpeg-4 si/2-overview paper/2-overview paper.htm [16] MPEG-4, why use it? Leonardo Chiariglione, http://leonardo.telecomitalialab.com/paper/mpeg-4/ [17] MPEG-2 FAQ, http://bmrc.berkeley.edu/frame/research/mpeg/mpeg2faq.html [18] From MPEG-1 to MPEG-21: Creating an Interoperable Multimedia Infrastructure, Rob Koenen, http://mpeg.telecomitalialab.com/documents/from mpeg-1 to mpeg-21.htm [19] Scaling Video Conferencing through Spatial Tiling, Ladan Gharai, Colin Perkins & Allison Mankin, http://www.east.isi.edu/projects/NMAA/ [20] MonaLisa web page http://monalisa.cern.ch/MONALISA/ [21] Jini web page http://www.jini.org [22] MonaLisa Web Start Client http://monalisa.cern.ch/vrvsjClient/ Part IV

Curriculum Vitae

229

231 Barry C. Barish Linde Professor of Physics

Born: January 27, 1936 Omaha Nebraska

Education: B.A. University of California, Berkeley 1957 Ph.D. University of California, Berkeley 1962

Academic & Research Positions: Linde Professor of Physics California Institute of Technology 1991–present Professor of Physics California Institute of Technology 1972–1991 Associate Professor California Institute of Technology 1969–1972 Assistant Professor California Institute of Technology 1966–1969 Research Fellow California Institute of Technology 1963–1966 Research Fellow University of California, Berkeley 1962–1963

Awards and Honors Fellow American Physical Society (APS) Member National Academy of Sciences (NAS) Member National Science Board (NSB) Klopsteg Award American Association of Physics Teachers (AAPT) Selected Recent Publications: With M. Ambrosia, et al., MACRO Collaboration. Neutrino astronomy with the MACRO detector. Astrophys. J. 546, 1038 (2001); astro-ph/0002492. With M. Ambrosia, et al., MACRO Collaboration. Search for diffuse neutrino flux from astrophysical courses with MACRO. Astroparticle Phys. 19, 1 (2003); astro-ph/0203181. With M. Ambrosia, et al., MACRO Collaboration. Search for cosmic ray sources using muons detected by the MACRO experiment. Astroparticle Phys. 18, 615 (2003); hep-ex/0204188. With M. Ambrosia, et al., MACRO Collaboration. Final results of magnetic monopole searches with the MACRO experiment. Eur. Phys J C25, 512 (2002); hep-ex/0207020. With M. Ambrosia, et al., MACRO Collaboration. Atmospheric neutrino oscillations from upward throughgoing muon multiple scatterings in MACRO. Submitted to Eur. Phys. J.C.; hep-ex/0304037. With M. Ambrosia, et al., MACRO Collaboration. Search for the sidereal and solar diurnal modulations in the total MACRO muon data set. Phys. Rev. D67, 042002 (2003); astro-ph/0211119. With M. Ambrosio, et al., MACRO Collaboration. Matter effects in upward-going muons and sterile neutrino oscillations. Phys. Lett. B517, 59 (2001); hep-ex/0106049. With M. Ambrosio, et al., MACRO Collaboration. Nuclearite search with the MACRO detector at Gran Sasso. Eur. Phys. J. C13, 453 (2000); hep-ex/9904031. With M. Ambrosio , et al., MACRO Collaboration. Low-energy atmospheric muon neutrinos in MACRO, Phys. Lett. B478, 5 (2000); hep-ex/0001044. Report of the DOE/NSF HEPAP Subpanel on Long Range Planning for U.S. High-Energy Physics, January 2002. B.C. Barish and R. Weiss, LIGO and the Detection of Gravitational Waves Phys. Today October, 44-50 (1999). 232 David G. Hitlin Professor of Physics

Born: April 15, 1942 Brooklyn, NY

Education: BA Columbia University 1963 MA Columbia University 1965 PhD Columbia University 1968

Academic & Research Positions: Professor of Physics California Institute of Technology 1985–present Associate Professor California Institute of Technology 1979–1985 Assistant Professor Stanford Linear Accelerator Center 1975–1979 Assistant Professor Stanford University 1972–1975 Research Associate Stanford Linear Accelerator Center 1969–1972 Instructor Columbia University 1967–1969

Selected Recent Publications: + ◦ Observation of a narrow meson decaying to Ds π at a mass of 2.32 GeV. BABAR Collaboration (B. Aubert et al.). Phys. Rev. Lett., 90, 242001 (2003). Measurement of the branching fraction and CP -violating asymmetries in neutral B decays to D∗±D∓. BABAR Collaboration (B. Aubert et al.). Phys. Rev. Lett. 90, 221801 (2003). Physics Motivation And Detector Design For A 1036 B Factory. N ucl. Instrum. Meth. A 494,29 (2002). − − Study of CP -violating asymmetries in B0 → π+π ,K+π Decays. BABAR Collaboration (B. Aubert et al.). Phys. Rev. D65, R5150 (2002). Measurement of B → K γ Branching Fractions and Charge Asymmetries BABAR Collaboration (B. Aubert et al.). Phys. Rev. Lett. 88, 101805 (2002). The BABAR Detector. BABAR Collaboration (B. Aubert et al.. N ucl. Instrum. Meth. A479, 1 (2002). Observation of CP Violation in the B0 Meson System. BABAR Collaboration (B. Aubert et al.). Phys. Rev. Lett. 87, 91801 (2001). 233 Marc Kamionkowski Professor of Theoretical Physics and Astrophysics

Born: July 27, 1965 Cleveland, OH

Education B.A. Washington University (St. Louis) 1987 Ph.D. University of Chicago 1991

Academic & Research Positions Professor California Institute of Technology September 1999–present Assistant and Associate Professor of Physics Columbia University 1994–1999 Member and Long-Term Member Institute for Advanced Study 1994 Astrophysics Editor Physics Reports 1998–present Receiving Editor J. High Energy Physics 1998–present

Awards and Honors Outstanding Junior Investigator Department of Energy 1998–1999 Helen B. Warner Prize American Astronomical Society 1998 Alfred P. Sloan Foundation Fellow 1996–1999 SSC National Fellow 1991–1993 Selected Recent Publications: Separation of Gravitational-Wave and Cosmic-Shear Contributions to Cosmic Microwave Background Polarization, (with M. Kesden and A. Cooray) Phys. Rev. Lett. 89, 011304 (2002). Spin-Dependent WIMPs in DAMA? (with P. Ullio and P. Vogel), JHEP 0107, 044 (2001). Spintessence! New Models for Dark Matter and Dark Energy, (with L. A. Boyle and R. R. Caldwell), Phys. Lett. B 545, 17 (2002). Intrinsic and Extrinsic Galaxy Alignment (with P. Catelan and R. D. Blandford) Mon. Not. R. Astron. Soc. 320, L7 (2001). The Dearth of Dwarf Galaxies: Is There Power on Small Scales? (with A. R. Liddle) Phys. Rev. Lett. 84, 4525 (2000). The Cosmic Microwave Background and Particle Physics (with A. Kosowsky), Ann. Rev. Nucl. Part. Sci. 49, 77 (1999). A Probe of Primordial Gravity Waves and Vorticity (with A. Kosowsky and A. Stebbins), Phys. Rev. Lett. 78, 2058 (1997). Cosmological-Parameter Determination with Microwave Background Maps (with G. Jungman, A. Kosowsky, D. Spergel), Phys. Rev. D 54, 1332 (1996). Small-Scale Cosmic Microwave Background Anisotropies as a Probe of the Geometry of the Universe (with D. Spergel, N. Sugiyama), Astrophys. J. Lett. 426, L57 (1994). Supersymmetric Dark Matter (with G. Jungman and K. Griest), Phys. Rep. 267, 195 (1996). 234 Anton Kapustin Assistant Professor of Physics

Born: November 10, 1971 Moscow, Russia

Education:

M.Sc. Moscow State University 1993 Ph.D. California Institute of Technology 1997

Academic & Research Positions: Assistant Professor of Physics California Institute of Technology 2001–present Long-term member Institute for Advanced Study, Princeton, NJ 2000-2001 Member Institute for Advanced Study, Princeton, NJ 1997-1999

Selected Recent Publications: Worldsheet descriptions of wrapped NS five-branes (with K. Hori), hep-th/0203147. Hyperkaehler metrics from periodic monopoles (with S.A. Cherkis), Phys. Rev. D65 (2002) 084015, hep-th/0109141. Remarks on A-branes, mirror symmetry, and the Fukaya category (with D. Orlov), hep-th/0109098. Duality of the fermionic 2d black hole and N = 2 Liouville theory as mirror symmetry (with K. Hori), JHEP 0108 (2001) 045, hep-th/0104202. Vertex algebras, mirror symmetry, and D-branes: the case of complex tori (with D. Orlov), hep- th/0010293. Nahm transform for periodic monopoles and N = 2 super Yang-Mills theory (with S.A. Cherkis), Commun. Math. Phys. 218 (2001) 333, hep-th/0006050. Noncommutative instantons and twistor transform (with A. Kuznetsov and D. Orlov), Commun. Math. Phys. 221 (2001) 385, hep-th/0002193. D-branes in a topologically nontrivial B-field, Adv. Theor. Math. Phys. 4 (2000) 127, hep-th/9909089. On mirror symmetry in three dimensional Abelian gauge theories (with M.J. Strassler), JHEP 9904 (1999) 021, hep-th/9902033. 235 Harvey B. Newman Professor of Physics Born: April 28, 1948 Brooklyn, NY Education: B.S. Massachusetts Institute of Technology 1968 Sc.D. Massachusetts Institute of Technology 1973 Academic & Research Positions: US CMS Collaboration Board Chair 1996–present Professor of Physics California Institute of Technology 1990–present Associate Professor California Institute of Technology 1982–1990 Physicist DESY, Hamburg 1978–1982 Consultant MIT Laboratory for Nuclear Science 1975–1982 Research Fellow Harvard University 1975–1977 Scientific Associate CERN - NP Division, Geneva 1974–1975 Awards: General Electric Award for Time Sharing Technology 1968 European Physical Society Special Award for the Discovery of Gluons at PETRA 1995 Internet2 Land Speed Records 2002, 2003 Selected National and International Committees: Internet Task Force on Scientific Computing 1986 - 1989 Technologies and the Conduct of Research (Consultant) 1986 - 1987 ICFA Network Task Force 1997 - 1998 ICFA SCIC Chair 1999 – Internet2 Applications Strategy Council 2001 – Internet2 HENP Working Group Chair 2001 – Selected Publications: Search for a Higgs Boson Decaying into Two-Photons at LEP, L3 Collaboration, Phys. Lett. B534 (2002) 28-38. Production of Single W Bosons at LEP and Measurement of WWγ Gauge Coupling Parameters, L3 Collaboration, Phys. Lett. B547 (2002) 151. + − Search√ for Neutral Higgs Bosons of the Minimal Supersymmetric Standard Model in e e Interactions at s up to 209 GeV, L3 Collaboration, Phys. Lett. B545 (2002) 30. Standard Model Higgs Boson with the L3 Experiment at LEP, L3 Collaboration, Phys. Lett. B517 (2001) 319-331. Search for Heavy Isosinglet Neutrino in e+e− Annihilation at LEP, L3 Collab., Phys. Lett. B517 (2001) 75-85. Data-intensive Grids for High Energy Physics, with J. Bunn, in Grid Computing, Making the Global Infrastructure a Reality, Berman et al. (Eds.), ISBN 0-470-85319-0, Wiley, UK (2003). History of Original Ideas and Basic Discoveries in Particle Physics, H. B. Newman and T. Ypsilantis (Eds.), Plenum, NY (1996); NATO ASI Physics Vol. 352 B. Quantum Chromodynamics and the Discovery of Gluon Jets at PETRA, Proceedings of the 1995 International School of Subnuclear Physics, Erice, Italy. Crystal Calorimeters in Particle Physics, G. Gratta, H. Newman, and R. Y. Zhu, Ann. Rev. Nucl. Part. Sci. 44 (1994) 453. 236 Hirosi Ooguri Professor of Physics

Born: March 13, 1962 Gifu, Japan

Education:

B.Sc. Kyoto University 1984 M.Sc. Kyoto University 1986 Ph.D. University of Tokyo 1989

Academic & Research Positions: Professor of Physics California Institute of Technology 2000–present Faculty Senior Scientist Lawrence Berkeley National Laboratory 1994 - 2000 Professor of Physics University of California at Berkeley 1994–2000 Associate Professor of Physics Kyoto University 1990–1994 Assistant Professor of Physics University of Chicago 1989–1990 Research Associate Institute for Advanced Study, Princeton 1988–1989 Assistant Professor of Physics University of Tokyo 1986–89

Selected Recent Publications: An Exact Solution To Seiberg-Witten Equation Of Noncommutative Gauge Theory, (with Y. Okawa) e-Print Archive: hep-th/0104036. Energy Momentum Tensors in Matrix Theory and in Noncommutative Gauge Theories, (with Y. Okawa) e-Print Archive: hep-th/0103124. How Noncommutative Gauge Theories Couple to Gravity, (with Y. Okawa) Nucl.Phys.B599 (2001) 55. Nonrelativistic Closed String Theory, (with J. Gomis) e-Print Archive: hep-th/0009181, Holography in Superspace, (with J. Rahmfeld, H. Robins, J. Tannenhauser), JHEP 0007 (2000) 045. Srings in AdS and the SL(2,R) WZW Model, PART 2: Euclidean Black Hole, (with J. Maldacena) e-Print Archive: hep-th/0005183. Knot Invariants And Topological Strings, (with C. Vafa), Nucl. Phys. B577 (2000) 419. Large N Field Theories, String Theory And Gravity, (with O. Aharony, S. Gubser, J. Maldacena, Y. Oz) Phys. Rep. 323 (2000) 183. Wilson Loops And Minimal Surfaces, (with N. Drukker, D. Gross) Phys. Rev. D60 (1999) 1250006. 237 Charles W. Peck Professor of Physics

Born: November 29, 1934 Freer, Texas

Education: B.S. New Mexico College of Agriculture and Mechanical Arts 1956 Ph.D. California Institute of Technology 1964

Academic & Research Positions: Chair, PMA Division California Institute of Technology 1993–1998 Executive Officer of Physics California Institute of Technology 1983–1986 Professor California Institute of Technology 1977–present Associate Professor California Institute of Technology 1969–1977 Assistant Professor California Institute of Technology 1965–1969 Research Fellow California Institute of Technology 1964–1965 Selected Recent Publications: With M. Ambrosia, et al., MACRO Collaboration. The MACRO detector at Gran Sasso. Nucl.Instr. Methods, Phys. Res. A486, 663 (2002). With M. Ambrosia, et al., MACRO Collaboration. A combined analysis technique for the search for fast magnetic monopoles with the MACRO detector. Astroparticle Phys. 18, 27 (2002), hep-ex/0110083. With M. Ambrosia, et al., MACRO Collaboration. Neutrino astronomy with the MACRO detector. Astrophys. J. 546, 1038 (2001); astro-ph/0002492. With M. Ambrosia, et al., MACRO Collaboration. Search for diffuse neutrino flux from astrophysical courses with MACRO. Astroparticle Phys. 19, 1 (2003); astro-ph/0203181. With M. Ambrosia, et al., MACRO Collaboration. Search for cosmic ray sources using muons detected by the MACRO experiment. Astroparticle Phys. 18, 615 (2003); hep-ex/0204188. With M. Ambrosia, et al., MACRO Collaboration. Muon energy estimate through multiple scattering with the MACRO detector. Nucl. Instr. Methods, Phys. Res. A492, 376 (2002); physics/0203018. With M. Ambrosia, et al., MACRO Collaboration. Final results of magnetic monopole searches with the MACRO experiment. Eur. Phys J C25, 512 (2002); hep-ex/0207020. With M. Ambrosia, et al., MACRO Collaboration. Atmospheric neutrino oscillations from upward throughgoing muon multiple scatterings in MACRO. Submitted to Eur. Phys. J.C.; hep-ex/0304037. With M. Ambrosia, et al., MACRO Collaboration. Search for the sidereal and solar diurnal modulations in the total MACRO muon data set. Phys. Rev. D67, 042002 (2003); astro-ph/0211119. With M. Ambrosia, et al., MACRO Collaboration. Search for nucleon decays induced by GUT magnetic monopoles with the MACRO experiment. Eur. Phys. J. C26, 163 (2002). With M. Ambrosio, et al., MACRO Collaboration. Matter effects in upward-going muons and sterile neutrino oscillations. Phys. Lett. B517, 59 (2001); hep-ex/0106049. With M. Ambrosio, et al., MACRO Collaboration. Nuclearite search with the MACRO detector at Gran Sasso. Eur. Phys. J. C13, 453 (2000); hep-ex/9904031. With M. Ambrosio , et al., MACRO Collaboration. Low-energy atmospheric muon neutrinos in MACRO, Phys. Lett. B478, 5 (2000); hep-ex/0001044. 238 Frank Porter Professor of Physics

Born: November 6, 1950 Schenectady, New York

Education: B.S. Physics, California Insistute of Technology 1972 Ph.D. – Physics University of California, Berkeley 1977

Academic & Research Positions Professor of Physics California Institute of Technology 1994–present Associate Professor of Physics California Institute of Technology 1988–1994 Assistant Professor of Physics California Institute of Technology 1982–1988 Senior Research Fellow California Institute of Technology 1980–1982 Weizmann Research Fellow California Institute of Technology 1978–1980 Research Fellow California Institute of Technology 1977–1978 Selected Recent Publications:

B. Aubert et al. [BABAR Collaboration], “Measurement of the CKM matrix element |Vub| with B → ρeν decays,” Phys. Rev. Lett. 90, 181801 (2003). B. Aubert et al. [BABAR Collaboration], “Simultaneous measurement of the B0 meson lifetime and 0 ∗− + mixing frequency with B → D ν decays,” Phys. Rev. D 67, 072002 (2003). B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fraction for inclusive semileptonic B meson decays,” Phys. Rev. D 67, 031101 (2003). B. Aubert et al. [BABAR Collaboration], “Measurements of branching fractions and CP-violating asymmetries in B0 → π+π−, K+π−, K+K− decays,” Phys. Rev. Lett. 89, 281802 (2002). B. Aubert et al. [BABAR Collaboration], “Measurement of the CP-violating asymmetry amplitude sin 2β,” Phys. Rev. Lett. 89, 201802 (2002). B. Aubert et al. [BABAR Collaboration], “A study of time dependent CP-violating asymmetries and ﬂavor oscillations in neutral B decays at the Υ(4S),” Phys. Rev. D 66, 032003 (2002). B. Aubert et al. [BABAR Collaboration], “Search for the rare decays B → K + − and B → K∗ + −,” Phys. Rev. Lett. 88, 241801 (2002). J. Z. Bai et al. [BES Collaboration], “A measurement of ψ(2S) resonance parameters,” Phys. Lett. B 550, 24 (2002). B. Aubert et al. [BABAR Collaboration], “Observation of CP violation in the B0 meson system,” Phys. Rev. Lett. 87, 091801 (2001). B. Aubert et al. [BABAR Collaboration], “The BaBar detector,” Nucl. Instrum. Meth. A 479, 1 (2002). J. Z. Bai et al., “The BES upgrade,” Nucl. Instrum. Meth. A 458, 627 (2001). J. Z. Bai et al. [BES Collaboration], “Measurement of ψ(2S) decays to baryon pairs,” Phys. Rev. D 63, 032002 (2001). J. Z. Bai, et al. [BES Collaboration], “Measurement of the Mass of the Tau Lepton,” Phys. Rev. D 53 20-34 (1996). F. C. Porter. “Interval Estimation Using the Likelihood Function,” CALT-68-1979, (1995). Nucl. Instrum. Meth. A 368 793-803, (1996). 239 John P. Preskill Professor of Theoretical Physics

Born: January 19, 1953 Highland Park, Illinois

Education: A.B. (Physics) Princeton University 1975 A.M. (Physics) Harvard University 1976 Ph.D. (Physics) Harvard University 1980

Academic & Research Positions: Professor of Theoretical Physics California Institute of Technology 1990–present Associate Professor California Institute of Technology 1983–90 Associate Professor Harvard University 1982–83 Assistant Professor Harvard University 1981–82 Junior Fellow Harvard Society of Fellows 1980–81 Selected Recent Publications: Secure Quantum Key Distribution Using Squeezed States (with D. Gottesman), Phys. Rev. A63 (2001) 022309. Simple Proof of Security of the BB84 Quantum Key Distribution Protocol (with P. Shor), Phys. Rev. Lett. 85 (2000) 441. Quantum Infromation and Precision Measurement (with A.M. Childs and J. Renes), J. Mod. Opt. 47 (2000) 155. Battling Decoherence: the Fault-Tolerant Quantum Computer, Phys. Tod. 52 (6) (1999) 24. Fault Tolerant Quantum Computation, in Introduction to Quantum Computation, ed. H.-K. Lo, S. Popescu and T.P. Spiller (World Scientiﬁc, 1998) p. 213-269. Reliable Quantum Computers, Proc. Roy. Soc. Lond. A454 (1998) 385-410. Quantum Computing: Pro and Con, Proc. Roy. Soc. Lond. A454 (1998) 469-486. Eﬃcient Networks for Quantum Factoring (with D. Beckman, A. Chari, and S. Devabhaktuni), Phys. Rev. A54 (1996) 1034-1063. Black Hole Thermodynamics and Information Loss in Two Dimensions (with T. M. Fiola, A. Strominger, and S. P. Trivedi), Phys. Rev. D50 (1994) 3987-4014. Nonabelian Vortices and Nonablian Statistics (with H.-K. Lo), Phys. Rev. D48 (1993) 4821-4834. Quantum Hair on Black Holes (with S. Coleman and F. Wilczek), Nucl. Phys. B378 (1992) 175-246. 240 John H. Schwarz Harold Brown Professor of Theoretical Physics

Born: November 22, 1941 North Adams, Massachusetts

Education:

A.B. Harvard College 1962 Ph.D. University of California, Berkeley 1966

Academic & Research Positions:: Harold Brown Professor California Institute of Technology 1989–present Professor of Theoretical Physics California Institute of Technology 1985–89 Senior Research Associate California Institute of Technology 1981–85 Research Associate California Institute of Technology 1972–81 Assistant Professor Princeton University 1969–72 Instructor and Lecturer Princeton University 1966–69

Selected Recent Publications: An SL(2,Z) Multiplet of Type IIB Superstrings, Phys. Lett. B360 (1995) 13 (hep-th/9508143); Erra- tum: ibid. B364 (1995) 252. The Power of M Theory, Phys. Lett. B367 (1996) 97 (hep-th/9510086). Anomaly-Free Supersymmetric Models in Six Dimensions, Phys. Lett. B371 (1996) 223 (hep-th/9512053). Anomalies, Dualities, and Topology of D=6 N=1 Superstring Vacua, (with M. Berkooz, R.G. Leigh, J. Polchinski, N. Seiberg, and E. Witten), Nucl. Phys. B475 (1996) 115–148 (hep-th/9605184). Lectures on Superstring and M Theory Dualities, presented at the ICTP spring school (March 1996) and the TASI summer school (June 1996), Nucl. Phys. (Proc. Suppl.) 55B (1997) 1 (hep-th/9607201) Interacting Chiral Gauge Fields in Six Dimensions and Born–Infeld Theory, (with M. Perry) Nucl. Phys. B489 (1997) 47 (hep-th/9611065). Gauge-Invariant and Gauge-Fixed D-Brane Actions, (with M. Aganagic and C. Popescu) Nucl. Phys. B495 (1997) 99 (hep-th/9612080). Coupling a Self-Dual Tensor to Gravity in Six Dimensions, Phys. Lett. B395 (1997) 191 (hep- th/9701008). World-Volume Action of the M Theory Five-Brane, (with M. Aganagic, J. Park, and C. Popescu), Nucl. Phys. B496 (1997) 191 (hep-th/9701166). Wrapping the M Theory Five-Brane on K3, (with Sergey Cherkis), Phys. Lett. B403 (1997) 225 (hep-th/9703062). 241 Alan J. Weinstein Professor of Physics

Born: December 22, 1957 Brooklyn, New York

Education: A.B. (Physics) Harvard University, Cambridge MA 1978 Ph.D. (Physics) Harvard University, Cambridge MA 1983

Academic & Research Positions: Professor California Institute of Technology 1999–present Associate Professor California Institute of Technology 1995–1999 Assistant Professor California Institute of Technology 1988–1995 Research Fellow University of California, Santa Cruz 1983–1988 Research Fellow Harvard University, Cambridge MA 1983 Selected Recent Publications: With W. Sun and the CLEO Collaboration (B.I. Eisenstein et al.), Measurement of the Charge Asym- metry in B → K∗(892)±π∓. CLNS 03/1823, 2003 (accepted by Phys.Rev.D); hep-ex/0304036. 0 + − With W. Sun and the CLEO Collaboration (E. Eckhart et al.), Observation of B → KSπ π and B → K∗±π∓. Phys. Rev. Lett. 89, 251801 (2002). With C. Boulahouache and the CLEO Collaboration (A.H. Mahmood et al.), Measurement of Lepton Momentum Moments in the Decay B¯ → X ν¯ and Determination of Heavy Quark Expansion Parameters and |Vcb|. Phys. Rev. D67, 072001 (2002). CLEO contributions to tau physics. Published in the Proceedings of the Seventh International Workshop on Tau Lepton Physics (TAU02), Santa Cruz, CA, Sept 2002 (Nuclear Physics B). With Y. Maravin and the CLEO Collaboration (G. Bonvicini et al.), Search for CP Violation in τ → Kπντ Decays. Phys. Rev. Lett. 88, 111803 (2002). 0 With Y. Maravin and the CLEO Collaboration (P. Avery et al.), Search for CP Violation in τ → ππ ντ Decay. Phys. Rev. D64, 092005 (2001). With D. Cronin-Hennessy and the CLEO Collaboration (D. Cronin-Hennessy et al.), Hadronic Mass Moments in Inclusive Semileptonic B Meson Decays. Phys. Rev. Lett. 87, 251808 (2001). With A. Lyon and the CLEO Collaboration (S. Chen et al.), Branchng Fraction and Photon Energy Spectrum for b → sγ. Phys. Rev. Lett. 87, 251807 (2001). With A. Lyon and the CLEO Collaboration (T.E. Coan et al.), Bounds on the CP Asymmetry in b → sγ Decays. Phys. Rev. Lett. 86, 5661 (2001). With D. Cinabro and the CLEO Collaboration (S. Ahmed et al.), First Measurement of Γ(D∗+)and Precision Measurement of mD∗+ − mD0 . Phys. Rev. Lett. 87, 251801 (2001). With W. Sun and the CLEO Collaboration (V. Savinov et al.), Search for a Scalar Bottom Quark with Mass 3.5–4.5 GeV/c2 Phys. Rev. D63, 051101 (2001). With J. Urheim and the CLEO Collaboration (S. Anderson et al.), Hadronic Structure in the Decay − − 0 τ → π π ντ . Phys. Rev. D61, 112002 (2000). With M. Schmidtler and the CLEO Collaboration (D. Asner et al.), Hadronic Structure in the Decay − − 0 0 τ → ντ π π π and the Sign of the Tau Neutrino Helicity. Phys. Rev. D61, 012002 (2000). 242 Mark B. Wise John A. McCone Professor of High Energy Physics

Born: November 9, 1953 Montreal, Quebec, Canada

Education:

B.Sc. University of Toronto 1976 M.Sc. University of Toronto 1977 Ph.D. Stanford University 1980

Academic & Research Positions: McCone Professor of Physics California Institute of Technology 1992–present Professor of Physics California Institute of Technology 1985– present Assistant Professor California Institute of Technology 1983–85 Junior Fellow Harvard University 1980–83

Selected Recent Publications: Two Nucleon Systems From Eﬀective Field Theory (with D. Kaplan and M. Savage) Nucl. Phys. B534 (1998) 329. Modulus Stabilization with Bulk Fields (with W. Goldberger) Phys. Rev. Lett. 83 (1999) 4922. Phenomenology of a Stabilized Modulus (with W. Goldberger) Phys. Lett. 475 (2000) 275. Generalized *-Products, Wilson Lines and the Solution of the Seiberg Witten Equations (with T. Mehen) JHEP 0012:008 (2000). Renormalization Group Flows For Brane Couplings (with W. Goldberger) Phys. Rev. D65 (2002) 0205011. 243 Ren-Yuan Zhu Member of Professional Staﬀ Born: October 17, 1945 Zhejiang, China

Education:

Ph.D. Massachusetts Institute of Technology 1983 B.S. Tsinghua University 1968

Academic & Research Positions:

Member of Professional Staff California Institute of Technology 1996–present Senior Research Associate California Institute of Technology 1992–1996 Senior Research Fellow California Institute of Technology 1987–1992 Research Fellow California Institute of Technology 1984–1987 Research Associate Massachusetts Institute of Technology 1983–1984 Physicist DESY, Hamburg 1978–1983 Selected Recent Publications: L3 Collaboration, Search√ for Neutral Higgs Bosons of the Minimal Supersymmetric Standard Model in e+e− Interactions at s = 209GeV at LEP, Phys. Lett. B545 (2002) 30 BaBar Collaboration, Measurement of the Branching Fraction and CP Content for the Decay B0 → D∗+D∗−, Phys. Rev.Lett. 89 (2002) 061801. T. Hu et al., Absolute Calibration of Electromagnetic Calorimeter at LHC with Physics Processes, in “Proceedings of the 10th International Conference on Calorimetry in Particle Physics”, ed. R.Y. Zhu, World Scientific (2002) 459. Q. Deng et al., Development of Yttrium Doped Lead Tungstate Crystal for Physics Applications, in “Proceedings of the 10th International Conference on Calorimetry in Partcle Physics”, ed. R.Y. Zhu, World Scientific (2002) 190. R.H. Mao et al., New Types of Lead Tungstate Crystals with High Light Yield, Nucl. Instr. and Meth. A486 (2002) 196. X.D. Qu et al., A study on Yttrium Doping in Lead Tungstate Crystals, Nucl. Instr. and Meth. A480 (2002) 468. X.D. Qu et al., A study on Sb Doping in Lead Tungstate Crystals, Nucl. Instr. and Meth. A469 (2001) 193. L.Y. Zhang et al., Monitoring Light Source for CMS Lead Tungstate Crystal Calorimeter at LHC, IEEE Trans. Nucl. Sci. NS-48 (2001) 372. U. Chaturvedi et al., Result of L3 BGO Calorimeter Calibration Using an RFQ Accelerator, IEEE Trans. Nucl. Sci. 47 (2000) 2102. X.D. Qu, L.Y. Zhang and R.Y. Zhu, Radiation Induced Color Centers and Light Monitoring for Lead Tungstate Crystals, IEEE Trans. Nucl. Sci. 47 (2000) 1741. R.Y. Zhu, Crystal Technologies for Future Linear Colliders, in “Proceedings of the 8th International Conference on Calorimetry in High Energy Physics”, ed. G. Barreira, World Scientific (2000) 226. R.Y. Zhu, Precision Crystal Calorimetry in High Energy Physics, Nucl. Phys. B78 (1999) 203. R.Y. Zhu, Radiation Damage in Scintillating Crystals, Nucl. Instr. and Meth. A413 (1998) 297. 244 Part V

Bibliography

245

A. Published Papers — Theory (2002 - present)

1. “Associated Production of h+− and W −+ in High Energy e+e− Collisions in the Minimal Super- symmetric Standard Model.” By H. Logan, S. Su. CALT-68-2375, Mar 2002, 21pp. Phys. Rev. D66:035001,2002. 2. “Belinfante Tensors Induced by Matter Gravity Couplings.” By V. Borokhov. CALT-68-2369, Jan 2002. 10pp. Published in Phys. Rev. D65:125022, 2002.

3. “Boundary States for AdS2 Branes in AdS3.” By P. Lee, H. Ooguri, J. Park. CALT-68-2363, Dec 2001. 212pp. Published in Nucl. Phys. B632:283-302, 2002. 4. “Comments on Superstring Interactions in a Plane-Wave Background.” By J.H. Schwarz. CALT- 68-2402, Aug. 2002, 11pp. Published in JHEP 0209:058, 2002. 5. “Confinement-Higgs Transition in a Disordered Gauge Theory and the Accuracy Threshold for Quantum Memory.” By C. Wang, J. Preskill. CALT-68-2374, Jul 2002. 15pp. Published in Annals. Phys. 303:31-58, 2003. 6. “Cubic Interactions in PP-Wave Light Cone String Field Theory.” By P. Lee, S. Moriyama, J. Park. CALT-68-2390, Jun 2002. 15pp. Published in Phys. Rev. D66:085021, 2002. 7. “The Equality of Solutions in Vacuum String Field Theory.” By T. Okuda. CALT-68-2371, Jan 2002. 8pp. Published in Nucl. Phys. B641:393-401, 2002. 8. “Enhanced Subleading Structure Functions in Semileptonic B Decay. ” By. A. Leibovich, Z. Ligeti, M.B. Wise. CALT-68-2384, May 2002. 12pp. Published in Phys. Lett. B539:242-248, 2002. 9. “Explicit Formulas for Neumann Coefficients in the Plane-Wave Geometry.” By Y.-H. He (Penn. U.), M. Spradlin (Princeton U.), A. Volovich (Santa Barbara, KITP), J.H. Schwarz, et al. CALT- 68-2413, Nov. 2002, 26pp. Published in Phys. Rev. D67:086005, 2003. 10. “Hadronic Lighty by Light Contribution to Muon g-2 in chiral perturbation theory. ” By M. Ramsey-Musolf. CALT-68-2372, Jan. 2002. 7pp. Published in Phys. Rev. Lett. 89:041601,2002. 11. “Holography and Defect Conformal Field Theories.” By O. DeWolfe (Santa Barbara, KITP), D.Z. Freedman (Santa Barbara, KITP & MIT & MIT, LNS), H. Ooguri. CALT-68-2359, Nov 2001. 46pp. Published in Phys. Rev. D66:025009, 2002. 12. “Implications of the Higgs Boson Searches on Different Soft Susy Breaking Scenarios.” By S. Ambrosanio, A. Dedes, S. Heinemeyer, S. Su, G. Weiglein, CALT-68-2315, Jun 2001, Jun 2001. 41pp. Published in Nucl. Phys. B624:3-44,2002.

13. “Matter from G2 Manifolds.” By P. Berglund (CIT-USC), A. Brandhuber. CALT-68-2385, May 2002. 30pp. Published in Nucl. Phys. B641:351-375, 2002. 14. “Minimal Superspace Vector Fields for 5D Minimal Supersymmetry.” By L. Rana (Swarthmore Coll.), S.J. Gates. CALT-68-2389, May 2002. 10pp. Published in Russ. Phys. J. 45:682-689, 2002, and Izv. Vuz. Fiz. 2002N7:35-41, 2002.

247 248 A Published Papers — Theory (2002 - present)

15. “Monopole Operators and Mirror Symmetry in Three-Dimensions.” By V. Borokhov, A. Ka- pustin, X. Wu. CALT-68-2397, Jul 2002. 29pp. Published in JHEP 0212:044, 2002.

16. “New Supersymmetric String Compactiﬁcations.” By S. Kachru (Stanford U., Phys. Dept. & SLAC), P.K. Tripathy, S.P. Trivedi (Tata Inst.), M.B. Schulz. CALT-68-2408, Nov 2002. 44pp. Published in JHEP 0303:061, 2003.

17. “A Note on Cubic Interactions in PP-Wave Light Cone String Field Theory.” By P. Lee, S. Moriyama, J. Park. CALT-68-2404, Sep 2002. 11pp. Published in Phys. Rev. D67:086001, 2003.

18. “A Note on Noncommutative D-Brane Actions.” By C. Ciocarlie, P. Lee, J. Park. CALT-68-2364, Dec 2001. 11pp. Published in Nucl. Phys. B632:303-310, 2002.

19. “Open Strings in PP-Wave Background From Defect Conformal Field Theory.” By P. Lee, J. Park. CALT-68-2377, Mar 2002. 14pp. Published in Phys. Rev. D67:026002, 2003.

20. “Open String States and D-Brane Tension from Vacuum String Field Theory.” By Y. Okawa. CALT-68-2378, Apr 2002. 53pp. Published in JHEP 0207:003, 2002.

21. “Penrose Limit of N = 1 Gauge Theories.” By J. Gomis, H. Ooguri. CALT-68-2373, Feb 2002. 24pp. Published in Nucl. Phys. B635:106-126, 2002.

22. “Permeable Conformal Walls and Holography.” By C. Bachas (Ecole Normale Superieure & Santa Barbara, KITP), J. de Boer (Amsterdam U. & Santa Barbara, KITP), R. Dijkgraaf (Amsterdam U. & Amsterdam U., Inst. Math. & Santa Barbara, KITP), H. Ooguri. CALT-68-2361, Nov 2001. 34pp. Published in JHEP 0206:027, 2002.

23. “PP-Wave/CFT2 Duality.” By L. Motl, A. Strominger (Harvard U.), J. Gomis. CALT-68-2394, Jun 2002. 20pp. Published in JHEP 0211:016, 2002.

24. “PP-Waves from Rotating and Continuously Distributed D3-Branes.” By K. Sfetsos (Patras U.) A. Brandhuber. CALT-68-2418, Dec 2002. 24pp. Published in JHEP 0212:050, 2002.

25. “Renormalization Group Flows For Brane Couplings. ” By W. Goldberger and M. Wise. CALT- 68-2327, April 2001. Published in Phys. Rev. D65, 025011, 2002.

26. “Secure Quantum Key Distribution With an Uncharacterized Source.” J. Preskill, M. Koashi. CALT-68-2401, Aug. 2002. Published in Phys. Rev. Lett. 90:057902, 2003.

27. “Strings in AdS3 and the SL(2,R) WZW Model. Part 3. Correlation Functions.” By J.M. Maldacena (Harvard U. & Princeton, Inst. Advanced Study), Hirosi Ooguri. CALT-68-2360, Nov 2001. 87pp. Published in Phys. Rev. D65:106006, 2002.

28. “The Stringy Quantum Hall Fluid.” By J.H. Brodie (SLAC), O. Bergman, Y. Okawa. CALT-68- 2338, Jul 2001. 24pp. Published in JHEP 0111:019, 2001.

29. “Topological Disorder Operators in Three-Dimensional Conformal Field Theory.” By V. Borokhov, A. Kapustin, X. Wu. CALT-68-2391, Jun 2002. 24pp. Published in JHEP 0211:049, 2002.

30. “Topological Quantum Memory.” By E. Dennis (Princeton U.), A. Kitaev, A. Landahl, John Preskill. CALT-68-2346, Oct 2001. 39pp. Published in J. Math. Phys. 43:4452-4505, 2002.

31. “Top-Squark Study at a Future e+e− Linear Collider.” By R. Kitano, T. Moroi (Tohoku U.), S. Su. CALT-68-2399, Aug 2002. 19pp. Published in JHEP 0212:011, 2002. 249

32. “Variation of the Cross Section for e+e− → W +h− in the Minimal Supersymmetric Standard Model. ” By H. Logan, S. Su. CALT-68-2392, Jun 2002. 6pp. Published in Phys. Rev. D67:017703,3003.

33. “World Sheet Descriptions of Wrapped NS Five-Branes.” By K. Hori (Princeton, Inst. Advanced Study), A. Kapustin. CALT-68-2376, Mar 2002. 41pp. Published in JHEP 0211:038, 2002. 250 A Published Papers — Theory (2002 - present) B. Published Papers — Experiment (2002 – present)

1. CLEO Collaboration (R. Godang et al.), “Search for Charmless B → VV Decay”, Phys. Rev. Lett. 88, 021802 (2002).

2. CLEO Collaboration (D.M. Asner et al.), “Search for B0 → π0π0 Decay”, Phys. Rev. D 65, 031103 (2002).

3. CLEO Collaboration (K.W. Edwards et al.), “First Observation of B¯0 → D∗0π+π+π−π− Decays”, Phys. Rev. D 65, 012002 (2002).

4. CLEO Collaboration (T.E. Coan et al.), “Observation of B¯0 → D0π0 and B¯0 → D∗0π0”, Phys. Rev. Lett. 88, 062001 (2002).

++ 5. CLEO Collaboration (M. Artuso et al.), “Measurement of the Masses and Widths of the Σc and 0 Σc Charmed Baryons”, Phys. Rev. D 65, 071101-1 (R)(2002). 6. CLEO Collaboration (G. Masek et al.), “Further Experimental Studies of Two-Body Radiative Υ Decays”, Phys. Rev. D 65, 072002-1 (2002).

7. CLEO Collaboration (A. Bornheim et al.), “Improved Measurement of |Vub| with Inclusive Semilep- tonic B Decays”, Phys. Rev. Lett. 88, 231803-1 (2002).

8. CLEO Collaboration (S.B. Athar et al.), “Measurement of the Ratio of Branching Fractions of the Υ(4S) to Charged and Neutral B Mesons”, Phys. Rev. D 66, 052003-1 (2002).

+ 9. CLEO Collaboration (A.H. Mahmood et al.), “Measurement of the Ξc Lifetime”, Phys. Rev. D 65, 031102 (R)(2002).

10. CLEO Collaboration (A. Anastassov et al.), “First Measurement of Γ(D∗+) and Precision Mea- surement of mD∗+ − mD0 ”, Phys. Rev. D 65, 032003 (2002).

11. CLEO Collaboration (S.E. Csorna et al.), “Lifetime Diﬀerences, Direct CP Violation and Partial Widths in D0 Meson Decays to K+K− and π+π−”, Phys. Rev. D 65, 092001 (2002).

12. CLEO Collaboration (K.W. Edwards et al.), “Search for Lepton-Flavor-Violating Decays of B Mesons”, Phys. Rev. D 65, 111102 (2002).

13. CLEO Collaboration (G. Bonvicini et al.), “Search for CP Violation in τ → Kπντ Decays”, Phys. Rev. Lett. 88, 111803 (2002).

14. CLEO Collaboration (R. Mahapatra et al.), “Observation of Exclusive B¯ → D(∗)K∗− Decays”, Phys. Rev. Lett. 88, 101803 (2002).

∗ 15. CLEO Collaboration (R.A. Briere et al.), “Improved Measurment of |Vcb| using B¯ → D ν De- cays”, Phys. Rev. Lett. 89, 081803 (2002).

+ ∗0 + 16. CLEO Collaboration (G. Brandenburg et al.), “Measurement of the D → K¯ ν Branching Fraction”, Phys. Rev. Lett. 89, 222001 (2002).

251 252 B Published Papers — Experiment (2002 – present)

17. CLEO Collaboration (K. Benslama et al.), “Anti-Search for the Glueball Candidate fJ (2220) in Two-Photon Interactions”, Phys. Rev. D 66, 077101 (2002).

0 + − 18. CLEO Collaboration (E. Eckhart et al.), “Observation of B → KSπ π and Evidence for B → K∗±π∓”, Phys. Rev. Lett. 89, 251801 (2002).

19. CLEO Collaboration (S. Anderson et al.), “Measurements of Inclusive B → ψ Production”, Phys. Rev. Lett. 89, 282001 (2002).

0 − + 20. CLEO Collaboration (R. Ammar et al.), “Observation of the Decay Ωc → Ω e νe”, Phys. Rev. Lett. 89, 171803 (2002). ¯ 21. CLEO Collaboration√ (Z. Metreveli et al.), “Correlated Inculsive ΛΛ Production in e + e− Anni- hilations at s ∼10.GeV”, Phys. Rev. D 66, 052002 (2002).

22. CLEO Collaboration (S. Ahmed et al.), “Measurement of B(B− → D0π−)andB(B¯0 → D+π−) and Isospin Analysis of B → Dπ Decays”, Phys. Rev. D 66, 031101 (2002).

0 0 + − 23. CLEO Collaboration (H. Muramatsu et al.), “Dalitz Analysis of D → KSπ π ”, Phys. Rev. Lett. 89, 251802 (2002).

24. CLEO Collaboration (S.A. Dytman et al.), “Measurement of Exclusive B Decays to Final States Containing a Charmed Baryon”, Phys. Rev. D 66, 091101 (R) (2002) .

0 25. CLEO Collaboration (S.Chen et al.), “Search for Neutrinoless τ Decays Involving the KS Meson”, Phys. Rev. D 66, 071101 (R) (2002).

26. CLEO Collaboration (N.E. Adam et al.), “Determination of the B¯ → D ∗ ν¯ Decay Width and |Vcb|”, Phys. Rev. D 67, 032001 (2003).

27. CLEO Collaboration (M. Artuso et al.), “Inclusive η Production from the Υ(1S)”, Phys. Rev. D 67, 052003 (2003).

28. CLEO Collaboration (D. Cronin-Hennessy et al.), “First Observation of the Exclusive Decays + + Λc → Λπ + π + π − π0 and Λc → Λωπ+”, Phys. Rev. D 67, 012001 (2003). 29. “Search for a Higgs Boson Decaying into Two-Photons at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B534/1-4 (2002) 28-38.

30. “Inclusive D∗± Production in Two-Photon Collisions at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B535/1-4 (2002) 59-69.

31. “Λ and Σ0 Pair Production in Two-Photon Collisions at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B536/1-2 (2002) 24-33. √ + − 32. “Determination of αs from Hadronic Event Shapes in e e Annihilation at 192 ≤ s ≤ 208 GeV”, L3 Collab., P. Achard et al., Phys. Lett. B536/3-4 (2002) 217-228.

33. “The e+e− → Zγγ → qqγγ¯ Reaction at LEP and Constraints on Anomalous Quartic Gauge Boson Couplings”, L3 Collab., P. Achard et al., Phys. Lett. B540/1-2 (2002) 43-51.

34. “Measurement of Genuine Three-particle Bose-Einstein Correlations in Hadronic Z Decay”, L3 Collab., P. Achard et al., Phys. Lett. B540 (2002) 185.

35. “Measurement of Bose-Einstein Correlations in e+e− → W+W− Events at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B547 (2002) 139. 253

36. “Production of Single W Bosons at LEP and Measurement of WWγ Gauge Coupling Parameters”, L3 Collab., P. Achard et al., Phys. Lett. B547 (2002) 151.

+ − 37. ’‘Search for Neutral√ Higgs Bosons of the Minimal Supersymmetric Standard Model in e e In- teractions at s up to 209 GeV”, L3 Collab., P. Achard et al., Phys. Lett. B545 (2002) 30.

38. “Inclusive Charged Hadron Production in Two-photon Collisions at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B554 (2003) 105.

39. “Measurement of W Polarisation at LEP”, L3 Collab., P. Achard et al., Phys. Lett. B557 (2003) 147.

40. “Study of the e+e− → Ze+e− Process at LEP ”, L3 Collab., P. Achard et al., CERN-EP/2002-103, Accepted by Phys. Lett. B.

41. “Search for Colour Reconnection Eﬀects in the e+e− → W+W− → hadrons through Particle-Flow Studies at LEP ”, L3 Collab., P. Achard et al., CERN-EP/2003-012, Accepted by Phys. Lett. B.

42. ”Proton-Antiproton Pair Production in Two-Photon Collisions at LEP”, L3 Collab., P. Achard et al., CERN-EP/2003-014, Accepted by Phys. Lett. B.

43. ”Search for Excited Leptons at LEP”, L3 Collab., P. Achard et al., CERN-EP/2003-013, Accepted by Phys. Lett. B.

44. ”Measurement of Exclusive ρ0ρ0 Production in Two-Photon Collisions at High Q2 at LEP”, L3 Collab., P. Achard et al., CERN-EP/2003-020, Accepted by Phys. Lett. B.

45. B.C. Barish et al., “Development of Large Size Sapphire Crystals for Laser Interferometer Grav- itational Wave Observatory”, IEEE Trans. Nucl. Sci. NS-49 (2002) 1233.

46. X.D. Qu et al., “ Yttrium Doped Lead Tungstate Crystals”, Nucl. Instr. and Meth. A486 (2002) 102.

47. R.H. Mao et al., “ New Types of Lead Tungstate Crystals with High Light Yield”, Nucl. Instr. and Meth. A486 (2002) 196.

48. X.D. Qu et al., “ A study on Sb Doping in Lead Tungstate Crystals”, Nucl. Instr. and Meth. A486 (2002) 89.

49. X.D. Qu et al., “ A study on Yttrium Doping in Lead Tungstate Crystals”, Nucl. Instr. and Meth. A480 (2002) 468.

50. B. Aubert et al. [BABAR Collaboration], “Measurement of the CKM matrix element |Vub| with B → rho e nu decays,” Phys. Rev. Lett. 90, 181801 (2003).

51. B. Aubert et al. [BABAR Collaboration], “Simultaneous measurement of the B0 meson lifetime and mixing frequency with B0 → D*- l+ nu/l decays,” Phys. Rev. D 67, 072002 (2003).

52. B. Aubert et al. [BABAR Collaboration], “A study of the rare decays B0 → D/s(*)+ pi- and B0 → D/s(*)- K+,” Phys. Rev. Lett. 90, 181803 (2003).

53. B. Aubert et al. [BABAR Collaboration], “A measurement of the B0 → J/psi pi+ pi- branching fraction,” Phys. Rev. Lett. 90, 091801 (2003).

54. B. Aubert et al. [BABAR Collaboration], “Study of inclusive production of charmonium mesons in B decay,” Phys. Rev. D 67, 032002 (2003). 254 B Published Papers — Experiment (2002 – present)

55. B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fraction for inclusive semileptonic B meson decays,” Phys. Rev. D 67, 031101 (2003).

56. B. Aubert et al. [BABAR Collaboration], “Measurements of branching fractions and CP-violating asymmetries in B0 → pi+ pi-, K+ pi-, K+ K- decays,” Phys. Rev. Lett. 89, 281802 (2002).

57. B. Aubert et al. [BABAR Collaboration], “Measurement of the CP-violating asymmetry amplitude sin 2beta,” Phys. Rev. Lett. 89, 201802 (2002).

58. B. Aubert et al. [BaBar Collaboration], “Measurement of the branching fraction and CP content for the decay B0 → D*+ D*-,” Phys. Rev. Lett. 89, 061801 (2002).

59. B. Aubert et al. [BABAR Collaboration], “Search for T and CP violation in B0 - anti-B0 mixing with inclusive dilepton events,” Phys. Rev. Lett. 88, 231801 (2002).

60. B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 lifetime with partially reconstructed B0 → D*- l+ nu/l decays,” Phys. Rev. Lett. 89, 011802 (2002) [Erratum-ibid. 89, 169903 (2002)].

61. B. Aubert et al. [BABAR Collaboration], “Measurement of D/s+ and D/s*+ production in B meson decays and from continuum e+ e- annihilation at s**(1/2) = 10.6-GeV,” Phys. Rev. D 65, 091104 (2002).

62. B. Aubert et al. [BABAR Collaboration], “A study of time dependent CP-violating asymmetries and ﬂavor oscillations in neutral B decays at the Upsilon(4S),” Phys. Rev. D 66, 032003 (2002).

63. B. Aubert et al. [BABAR Collaboration], “Search for the rare decays B → Kl+l-andB→ K* l+ l-,” Phys. Rev. Lett. 88, 241801 (2002).

64. J. Z. Bai et al. [BES Collaboration], “A measurement of psi(2S) resonance parameters,” Phys. Lett. B 550, 24 (2002).

65. J. Z. Bai et al. [BES Collaboration], “psi(2S) two- and three-body hadronic decays,” Phys. Rev. D 67, 052002 (2003).

66. MACRO Collaboration, M. Ambrosio et al., “Muon energy estimate through multiple scattering with the MACRO Detector”, Nucl.Instrum.Meth. A492 (2002), 376.

67. MACRO Collaboration, M. Ambrosio et al., “The MACRO Detector at Gran Sasso”, Nucl.Instrum.Meth. A486 (2002), 663.

68. MACRO Collaboration, M. Ambrosio et al., “Measurement of the residual energy of muons in the Gran Sasso Underground Laboratories”, Astropart.Phys. 19 (2003), 313, hep-ex/0207043.

69. MACRO Collaboration, M. Ambrosio et al., “Search for nucleon decays induced by GUT magnetic monopoles with the MACRO Experiment”, Eur.Phys.J. C26 (2002), 163, hep-ex/0207024.

70. MACRO Collaboration, M. Ambrosio et al., “Final results of magnetic monopole searches with the MACRO Experiment”, Eur.Phys.J. C25 (2002), 511, hep-ex/0207020.

71. MACRO Collaboration, M. Ambrosio et al., “Search for comic ray sources using muons detected by the MACRO Experiment”, Astropart.Phys. 18 (2003), 615, hep-ph/0204188.

72. MACRO Collaboration, M. Ambrosio et al., “Search for diﬀuse neutrino ﬂux from astrophysical sources with MACRO”, Astropart.Phys. 19 (2003), 1, astro-ph/0203181. 255

73. MINOS Collaboration, P. Adamson et al., “The MINOS Scintillator Calorimeter System”, IEEE Trans.Nucl.Sci. 49 (2002) 861.

74. P. Adamson, et. al., “The MINOS Light Injection Calibration System”, Nucl.Instrum.Meth. A492 (2002), 325, hep-ex/0204021.

75. D0 Collaboration, H. Zheng et al., “Search for large extra dimensions in the monojet +MET channel with the D0 Detector, Feb 2003, accepted for publication in Phys.Rev.Lett., hep-ex/030214. 256 B Published Papers — Experiment (2002 – present) C. Unpublished Papers — Theory (2002 - present)

1. “Anyons from Non-Solvable Discrete Groups Are Suﬃcient for Universal Quantum Computation.” By C. Mochon. CALT-68-2393, June 2002.

2. “The C-Deformation of Gluino and Non-planar Diagrams.” By H. Ooguri, C. Vafa. CALT-68- 2428, Feb 2003. 32pp.

3. “Chiral Rings, Superpotentials and the Vacuum Structure of N = 1 Supersymmetric Gauge Theories.” By H. Ita, H. Nieder, Y. Oz (Tel Aviv U.), Christian Romelsberger (Southern California U.), A. Brandhuber. CALT-68-2431, Feb 2003. 40pp.

4. “Comment on Quark Masses in SCET.” By A.K. Leibovich (Pittsburgh U. & Fermilab), Z. Ligeti (LBL, Berkeley), M.B. Wise. CALT-68-2432, Mar 2003. 7pp.

5. “Coupling of Rolling Tachyon to Closed Strings.” By T. Okuda, S. Sugimoto (Bohr Inst.). CALT- 68-2403, Aug 2002. 18pp.

6. “D Branes in Landau-Ginzburg Models and Algebraic Geometry.” By A. Kapustin, Y. Li. CALT- 68-2412, Aug 2002. 47pp.

7. “e+e− → ννA¯ 0 in the Two-Higgs-Doublet Model.” By T. Farris, J.F. Gunion (UC, Davis), H.E. Logan (Wisconsin U., Madison), S.-F. Su. CALT-68-2429, Feb 2003. 25pp.

8. “Eﬀective Field Theories on Non-Commutative Spacetime.” By X. Calmet, M. Wohlgenannt. CALT-68-2436, April 2003.

9. “Enhanced Nonperturbative Eﬀects in Jet Distributions.” By C.W. Bauer, A.V. Manohar (UC, San Diego), M.B. Wise. Dec 2002. 4pp.

10. “Exact N = 2 Supergravity Solutions with Polarized Branes.” By I. Bena (UCLA), C. Ciocarlie. CALT-68-2417, Dec 2002. 22pp.

11. “Families of N = 2 Strings.” By Y.-K. E. Cheung, Y. Oz (CERN & Tel Aviv U.), Z. Yin (CERN). CALT-68-2410, Nov 2002. 67pp.

12. “The Glueball Superpotential.” By M. Aganagic, K. Intriligator, C. Vafa, N.P. Warner. CALT- 68-2437, Apr 2003. 55pp.

13. “Grand Uniﬁcation and Time Variation of the Gauge Couplings.” By X. Calmet, H. Fritzsch. CALT-68-2416, Nov. 2002.

14. “Gravity Induced C Deformation.” By H. Ooguri, C. Vafa (Caltech & Harvard U., Phys. Dept.). CALT-68-2433, Mar 2003. 13pp.

15. “Inside the Horizon with AdS/CFT.” By P. Kraus (UCLA), H. Ooguri, S Shenker (Stanford U., Phys. Dept.). CALT-68-2421, Dec 2002. 31pp.

257 258 C Unpublished Papers — Theory (2002 - present)

16. “The Lightest Higgs Boson of mSUGRA, mGMSB AND mAMSB at Present and Future Colliders: Observability and Precision Analyses.” By A. Dedes (Munich, Tech. U.), S. Heinemeyer (Munich U.), S. Su (Caltech), G. Weiglein (Durham U., IPPP). CALT-68-2407, Feb 2003. 33pp. 17. “N = 1 Supersymmetry, Deconstruction, and Bosonic Gauge Theories.” By R. Dijkgraaf (Ams- terdam U. & Amsterdam U., Inst. Math.), C. Vafa. CALT-68-2425, Feb 2003. 33pp. 18. “A Path Integral Approach to Noncommutative Superspace.” By I. Chepelev, C. Ciocarlie. CALT-68-2434, Apr 2003. 13pp. 19. “Probing Supersymmetry with Parity Violating Electron Scattering.” By A. Kurylov, M.J. Ramsey-Musolf, S. Su. CALT-68-2430, Mar 2003. 30pp. 20. “Quantum Aspects of Seiberg-Witten Map in Noncommutative Chern-Simons Theory.” By K. Kaminsky, Y. Okawa, H. Ooguri. CALT-68-2420, Jan 2003. 27pp. 21. “Reexamining Classical Solution and Tachyon Mode in Vacuum String Field Theory.” By H. Hata, S. Moriyama. CALT-68-2395, June 2002. 22. “Security of Quantum Key Distribution With Imperfect Devices.” J. Preskill, D. Gottesman, H.-K. Lo, N. Ltkenhaus. CALT-68-2406, Sept. 2002. 23. “Single Heavy MSSM Higgs Production at e+e− Linear Collider.” By S.-F. Su. Oct 2002. 6pp. Talk given at 10th International Conference on Supersymmety and Uniﬁcation of Fundamental Interactions (SUSY02), Hamburg, Germany, 17-23 Jun 2002. 24. “Softening the Naturalness Problem.” By X.P. Calmet. CALT-68-2427, Feb. 2003. 25. “Supersymmetric Eﬀects in Deep Inelastic Neutrino Nucleus Scattering.” By A. Kurylov, M.J. Ramsey-Musolf, S. Su. CALT-68-2422, Jan 2003. 30pp. 26. “Supersymmetry, Axions and Cosmology.” By T. Banks, M. Dine (UC, Santa Cruz), M. Graesser. CALT-68-2409, Oct 2002. 25pp. 27. “SYM Description of PP-Wave String Interactions: Singlet Sector and Arbitrary Impurities.” By J. Gomis, S. Moriyama, J. Park. CALT-68-2424, Jan 2003. 51pp. 28. “SYM Description of SFT Hamiltonian in a PP-Wave Background.” By J. Gomis, S. Moriyama, J. Park. CALT-68-2411, Oct 2002. 17pp. 29. “Topological Correlators in Landau-Ginzburg Models with Boundaries.” By A. Kapustin, Y. Li. 21pp 30. “The Topological Vertex.” By M. Aganagic, A. Klemm, M. Marino, C. Vafa. CALT-68-2439, May 2003. 70pp. 31. “Two Graviton Interaction in PP-Wave Background in Matrix Theory and Supergravity.” By H.K. Lee, X.-K Wu. CALT-68-2423, Jan 2003. 29pp. 32. “Universality Classes for Horizon Instabilities.” By S.S. Gubser (Princeton U.), A. Ozakin. PUPT-2059, Dec 2002. 22pp. 33. “Update on String Theory.” By J.H. Schwarz. CALT-68-2417, Apr 2003. 14pp. 34. “When Superspace is Not Enough.” By W.D. Linch, III, J. Phillips (Maryland U.), S.J. Gates. CALT-68-2387, Nov 2002. 44pp. 35. “World Sheet Derivation of a Large N Duality.” By H. Ooguri, C. Vafa. CALT-68-2386, May 2002. 39pp. D. Preprints and Conference Proceedings — Experiment (2002 – Present)

1. CLEO Collaboration (A.H. Mahmood et al.), “Measurement of Lepton Momentum Moments in the Decay B¯ → X ν¯ and Determination of Heavy Quark Expansion Parameters and |Vcb|”, CLNS 102/1810, CLEO 102–16 (submitted to Phys. Rev. D). 2. CLEO Collaboration (T.E. Coan et al.), “First Search for the Flavor Changing Neutral Current Decay D0 → γγ”, CLNS 102/1806, CLEO 102–15 (submitted to Phys. Rev. Lett.). 3. CLEO Collaboration (S.E. Csorna et al.), “Measurements of the Branching Fractions and Helicity Amplitudes in B → D∗ρ Decays”, CLNS 103/1813, CLEO 103–01 (submitted to Phys. Rev. D). 16/1/2003 4. CLEO Collaboration (A. Bornheim et al.), “Measurements of Charmless Hadronic Two-Body B Meson Decays and the Ratio B(B → DK)/B(B → Dπ)”, CLNS 103/1816, CLEO 103–03 (submitted to Phys. Rev. D). 19/2/2003 5. CLEO Collaboration (R.A. Briere et al.), “Branching Fractions of τ Leptons Decays to Three Charged Hadrons”, CLNS 103/1818, CLEO 103–04 (submitted to Phys. Rev. Lett.). 20/2/2003 6. CLEO Collaboration (G. Bonvicini, et al.), “Study of Charmless Inclusive B → ηX Deacy”, CLNS 103/1815, CLEO 103–02 (submitted to Phys. Rev. D). 10/3/2003 7. CLEO Collaboration (S.B. Athar, et al.), “Study of the q2-Dependence of the B → π ν and B → ρ(ω) ν Decay and Extraction of |Vub|”, CLNS 103/1819, CLEO 103–05 (submitted to Phys. Rev. D). 11/3/2003 − 8. CLEO Collaboration (N.E. Adam, et al.), “Search for B → pe¯ ν¯eX Decay using a Partial Recon- struction Method”, CLNS 103/1820, CLEO 103–06 (submitted to Phys. Rev. D). 9. CLEO Collaboration (B.I. Eisenstein, et al.), “Measurement of the Charge Asymmetry in B → K∗(892)±π∓”, CLNS 103/1823, CLEO 103–07 (submitted to Phys. Rev. D). 24/4/2003 10. CLEO Collaboration (K.W. Edwards, et al.), “Search for Baryons in the Radiative Penguin Decay b → sγ”, CLNS 103/1824, CLEO 103–08 (submitted to Phys. Rev. D). 5/5/2003 11. A. Weinstein (for the CLEO Collaboration), “Recent results on B meson decays from CLEO”, in the Proceedings of the XVI Rencontres de Physique de La Vallee d’Aoste, La Thuile, Italy, March 3-9, 2002. 12. A. Weinstein (representing the CLEO Collaboration), “CLEO contributions to tau physics”, in the Proceedings of the Seventh International Workshop on Tau Lepton Physics (TAU02), Santa Cruz, Ca, USA, Sept 2002, to be published in: Nuclear Physics B. 13. A. Bornheim (representing the CLEO Collaboration), “CLEO - Recent Results and Future Prospects”, in Proceedings of The Fifth KEK Topical Conference, KEK, Tsukuba, Japan, November 2001, ed. S. Hashimoto and T.K. Komatsubara, Nuclear Physics B - Proceedings Supplement (2002).

259 260 D Preprints and Conference Proceedings — Experiment (2002 – Present)

14. “Search for a Higgs Boson decaying to Weak Boson pairs at LEP”, L3 Collab., P. Achard et al., CERN-EP/2002-080, To Be Submitted to Phys. Lett. B. 15. “A Combination of Preliminary Electroweak Measurements and Constraints on the Standard Model”, L3 Collab., P. Achard et al., CERN-EP/2002-091. 16. ”Measurement of Branching Fractions of τ Hadronic Decays”, L3 Collab., P. Achard et al.,CERN- EP/2003-019, Submitted to Phys. Lett. B. 17. ”Search for the Standard Model Higgs Boson at LEP”, L3 Collab., P. Achard et al.,CERN- EP/2003-xxx, Submitted to Phys. Lett. B. 18. “Interpretation of Neutralino and Scalar Lepton Searches in Minimal GMSB Model”, M. Gataullin, S. Rosier-Lees, L. Xia, H. Yang, L3 Note 2777 (2002). 19. CMS Internal Note 2003/023, T. Lee, V. Litvin, H. Newman, S. Shevchenko, “Search for intermediate mass Higgs boson, decaying into two photons.” 20. CMS Note 2002/030, V. Litvin, H. Newman, S. Shevchenko, N. Wisniewski, “The rejection of background to the H → γγ process using isolation criteria based on information from electromagnetic calorimeter and tracker.” 21. CMS IN 2002/034, V. Litvin, H. Newman, S. Shevchenko, N. Wisniewski, “Isolation studies for photons from Higgs decay based on the Tracker.” 22. CMS IN 2002/023, V. Litvin, H. Newman, S. Shevchenko, “Isolation studies for electrons and photons based on ECAL.” 23. Q. Deng et al., “ Development of Yttrium Doped Lead Tungstate Crystal for Physics Applica- tions”, in Proceedings of the 10th International Conference on Calorimetry in Partcle Physics, ed. R.Y. Zhu, World Scientific (2002) 190. 24. T. Hu et al., “Absoluate Calibration of Electromagnetic Calorimeter at LHC with Physics Pro- cesses”, in Proceedings of the 10th International Conference on Calorimetry in Partcle Physics, ed. R.Y. Zhu, World Scientific (2002) 459. 25. L. Zhang et al., “Monitoring Light Source for CMS Lead Tungstate Crystal Calorimeter at LHC”, in Proceedings of the 10th International Conference on Calorimetry in Partcle Physics, ed. R.Y. Zhu, World Scientific (2002) 469. 26. “Data Intensive Grids and Networks for High Energy and Nuclear Physics”, H. Newman, Nucl. Phys. B 120 (2003) 109. 27. “The Clarens Web Services Architecture”, C.D. Steenberg, E. Aslakson, J.J. Bunn, H.B. New- man, M. Thomas, F. van Lingen, Proceedings of CHEP 2003, paper MONT008, Submitted for publication, (2003). 28. “Clarens Client and Server Applications”, C.D. Steenberg, E. Aslakson, J.J. Bunn, H.B. New- man, M. Thomas, F. van Lingen, Proceedings of CHEP 2003, paper TUCT005, Submitted for publication, (2003). 29. “MonALISA : A Distributed Monitoring Service Architecture’, H.B. Newman et al., Proceedings of CHEP 2003, paper MOET001, Submitted for publication, (2003). 30. “Distributed Heterogeneous Relational Data Warehouse In A Grid Environment”, S. Iqbal, J.J. Bunn, H.B. Newman, Proceedings of CHEP 2003, Submitted for publication, (2003). 261

31. ”Global Platform for Rich Media Conferencing and Collaboration”, D. Adamczyk et al., Proceed- ings of CHEP 2003, Submitted for publication, (2003).

32. B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fractions for the exclusive decays of B0 and B+ to anti-D(*) D(*) K,” arXiv:hep-ex/0305003.

33. B. Aubert et al. [BABAR Collaboration], “A search for B- → tau- anti-nu recoiling against a fully reconstructed B,” arXiv:hep-ex/0304030.

34. M. Battaglia et al., “The CKM matrix and the unitarity triangle,” arXiv:hep-ph/0304132.

35. B. Aubert et al. [BABAR Collaboration], “Observation of a narrow meson decaying to D/s+ pi0 at a mass of 2.32-GeV/c**2,” arXiv:hep-ex/0304021.

36. B. Aubert et al. [BABAR Collaboration], “A Search for the decay B- → K- nu anti-nu,” arXiv:hep- ex/0304020.

37. B. Aubert et al. [BABAR Collaboration], “Rare B decays into states containing a J/psi meson and a meson with s anti-s quark content,” arXiv:hep-ex/0304014.

38. B. Aubert et al. [BABAR Collaboration], “Search for D0 - anti-D0 mixing and a measurement of the doubly Cabibbo-suppressed decay rate in D0 → K pi decays,” arXiv:hep-ex/0304007.

39. B. Aubert et al. [BABAR Collaboration], “Measurements of the branching fractions and charge asymmetries of charmless three-body charged B decays,” arXiv:hep-ex/0304006.

40. B. Aubert et al. [BABAR Collaboration], “Measurements of CP-violating asymmetries and branching fractions in B meson decays to eta’ K,” arXiv:hep-ex/0303046.

41. B. Aubert et al. [BABAR Collaboration], “Limits on the lifetime diﬀerence of neutral B mesons and on CP, T, and CPT violation in B0 anti-B0 mixing,” arXiv:hep-ex/0303043.

42. B. Aubert et al. [BABAR Collaboration], “Observation of B meson decays to omega pi+, omega K+, and omega K0,” arXiv:hep-ex/0303040.

43. B. Aubert et al. [BABAR Collaboration], “Observation of B meson decays to eta pi and eta K,” arXiv:hep-ex/0303039.

44. B. Aubert et al. [BABAR Collaboration], “Evidence for B+ → J/psi p anti-Lambda and search for B0 → J/psi p anti-p,” arXiv:hep-ex/0303036.

45. B. Aubert et al. [BABAR Collaboration], “A search for B+ → tau+ nu(tau) recoiling against B- → D0 l- nu/l X,” arXiv:hep-ex/0303034.

46. B. Aubert et al. [BABAR Collaboration], “Branching fractions in B → Phi h and search for direct CP violation in B+- → Phi K+-,” arXiv:hep-ex/0303029.

47. B. Aubert et al. [BABAR Collaboration], “Observation of the decay B+- → pi+- pi0, study of B+- → K+- pi0, and search for B0 → pi0 pi0,” arXiv:hep-ex/0303028.

48. B. Aubert et al. [BABAR Collaboration], “Measurements of the branching fractions of charged B decays to K+ pi- pi+ ﬁnal states,” arXiv:hep-ex/0303022.

49. B. Aubert et al. [BABAR Collaboration], “Rates, polarizations, and asymmetries in charmless vector-vector B decays,” arXiv:hep-ex/0303020. 262 D Preprints and Conference Proceedings — Experiment (2002 – Present)

50. B. Aubert et al. [BABAR Collaboration], “Study of time-dependent CP asymmetry in neutral B decays to J/psi pi0,” arXiv:hep-ex/0303018.

51. B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fraction and CP- violating asymmetries in neutral B decays to D*+- D-+,” arXiv:hep-ex/0303004.

52. B. Aubert et al. [BABAR Collaboration], “Measurement of B0 → D/s(*)+ D*- branching fractions and B0 → D/s(*)+ D*- polarization with a partial reconstruction technique,” arXiv:hep- ex/0302015.

53. B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 meson lifetime with partial reconstruction of B0 → D*- pi+ and B0 → D*- rho+ decays,” arXiv:hep-ex/0212012.

54. B. Aubert et al. [BABAR Collaboration], “A measurement of the B0 → J/psi pi+ pi- branching fraction,” arXiv:hep-ex/0203034.

55. B. Aubert et al. [BABAR Collaboration], “Measurement of branching fractions of color suppressed decays of the anti-B0 meson to D0 pi0, D0 eta, and D0 omega,” arXiv:hep-ex/0207092.

56. B. Aubert et al. [BABAR Collaboration], “Dalitz plot analysis of D0 hadronic decays D0 → K0 K- pi+, D0 → anti-K0 K+ pi- and D0 → anti-K0 K+ K-,” arXiv:hep-ex/0207089.

57. B. Aubert et al. [BABAR Collaboration], “Measurement of branching ratios and CP asymmetries in B- → D0(CP) K- decays,” arXiv:hep-ex/0207087.

58. B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fractions for the exclusive decays of B0 and B+ to anti-D(*) D(*) K,” arXiv:hep-ex/0207086.

59. B. Aubert et al. [BABAR Collaboration], “Measurement of the B0 → D*- a1+ branching fraction with partially reconstructed D*,” arXiv:hep-ex/0207085.

60. B. Aubert et al. [BABAR Collaboration], “Measurement of the ﬁrst hadronic spectral moment from semileptonic B decays,” arXiv:hep-ex/0207084.

61. B. Aubert et al. [BABAR Collaboration], “Search for decays of B0 mesons into pairs of leptons,” arXiv:hep-ex/0207083.

62. B. Aubert et al. [BABAR Collaboration], “Evidence for the ﬂavor changing neutral current decays B → Kl+l-andB→ K* l+ l-,” arXiv:hep-ex/0207082.

63. B. Aubert et al. [BABAR Collaboration], “Measurement of the inclusive electron spectrum in charmless semileptonic B decays near the kinematic endpoint,” arXiv:hep-ex/0207081.

64. B. Aubert et al. [BABAR Collaboration], “Measurement of the CKM matrix element |V(ub)| with charmless exclusive semileptonic B meson decays at BABAR,” arXiv:hep-ex/0207080.

65. B. Aubert et al. [BABAR Collaboration], “Measurement of B0 → D/s(*)+ D*- branching fractions and polarization in the decay B0 → D/s*+ D*- with a partial reconstruction technique,” arXiv:hep-ex/0207079.

66. B. Aubert et al. [BaBar Collaboration], “Determination of the branching fraction for inclusive decays B → X/s gamma,” arXiv:hep-ex/0207076.

67. B. Aubert et al. [BABAR Collaboration], “b → s gamma using a sum of exclusive modes,” arXiv:hep-ex/0207074. 263

68. B. Aubert et al. [BABAR Collaboration], “Search for the exclusive radiative decays B → rho gamma and B0 → omega gamma,” arXiv:hep-ex/0207073.

69. B. Aubert et al. [BABAR Collaboration], “Measurement of time-dependent CP asymmetries and the CP-odd fraction in the decay B0 → D*+ D*-,” arXiv:hep-ex/0207072.

70. B. Aubert et al. [BABAR Collaboration], “Simultaneous measurement of the B0 meson lifetime and mixing frequency with B0 → D*- l+ nu/l decays,” arXiv:hep-ex/0207071.

71. B. Aubert et al. [BABAR Collaboration], “Measurement of sin(2beta) in B0 → Phi K0(S),” arXiv:hep-ex/0207070.

72. B. Aubert et al. [BABAR Collaboration], “A search for B+ → K+ nu anti-nu,” arXiv:hep- ex/0207069.

73. B. Aubert et al. [BABAR Collaboration], “Search for CP violation in B0 / anti-B0 decays to pi+ pi- pi0 and K+- pi-+ pi0 in regions dominated by the rho+- resonance,” arXiv:hep-ex/0207068.

74. B. Aubert et al. [BABAR Collaboration], “Measurement of the branching fraction for B+- → chi/c0 K+-,” arXiv:hep-ex/0207066.

75. B. Aubert et al. [BABAR Collaboration], “Measurements of branching fractions and direct CP asymmetries in pi+ pi0, K+ pi0 and K0 pi0 B decays,” arXiv:hep-ex/0207065.

76. B. Aubert et al. [BABAR Collaboration], “A search for the decay B0 → pi0 pi0,” arXiv:hep- ex/0207063.

77. B. Aubert et al. [BABAR Collaboration], “A study of time dependent CP asymmetry in B0 → J/psi pi0 decays,” arXiv:hep-ex/0207058.

78. B. Aubert et al. [BABAR Collaboration], “A study of the rare decays B0 → D/s(*)+ pi- and B0 → D/s(*)- K+,” arXiv:hep-ex/0207053.

79. B. Aubert et al. [BABAR Collaboration], “Measurements of charmless two-body charged B decays with neutral pions and kaons,” arXiv:hep-ex/0206053.

80. B. Aubert et al. [BABAR Collaboration], “Measurements of the branching fractions of charmless three-body charged B decays,” arXiv:hep-ex/0206004.

81. B. Aubert et al. [BABAR Collaboration], “Evidence for the b → u transition B0 → D/s+ pi- and asearchforB0→ D/s*+ pi-,” arXiv:hep-ex/0205102.

82. B. Aubert et al. [BABAR Collaboration], “Measurements of branching fractions and CP-violating asymmetries in B0 → pi+ pi-, K+ pi-, K+ K- decays,” arXiv:hep-ex/0205082.

83. A. Refregier et al., “Weak Lensing from Space III: Cosmological Parameters,” arXiv:astro-ph/0304419.

84. R. Massey et al., “Weak Lensing from Space II: Dark Matter Mapping,” arXiv:astro-ph/0304418.

85. C. Peck, “Some Neutrino Oscillation Experiments in the United States”, Presented at “Cosmic Radiations: From Astronomy to Particle Physics, Oujda, MA, Mar. 2001, NATO Science Series, Kluwer Academic Publishers.

86. S. Kyriazopoulou, “Search of Slow Magentic Monopoles with the MACRO Scintillation Detector”, Presented at “Cosmic Radiations: From Astronomy to Particle Physics, Oujda, MA, Mar. 2001, NATO Science Series, Kluwer Academic Publishers. 264 D Preprints and Conference Proceedings — Experiment (2002 – Present)

87. D. Michael, “Neutrino Oscillation Results from SuperKamiokande and Soudan 2”, Presented at “Cosmic Radiations: From Astronomy to Particle Physics, Oujda, MA, Mar. 2001, NATO Science Series, Kluwer Academic Publishers.

88. D. Michael, “Update on MINOS”, Presented at “Cosmic Radiations: From Astronomy to Particle Physics, Oujda, MA, Mar. 2001, NATO Science Series, Kluwer Academic Publishers.

89. D. Michael, “The MINOS Experiment”, Lecture presented at NUclear Physics School on Neutri- nos, Erice, Italy, Sep. 2001.

90. D. Michael, “Neutrino Oscillation Physics”, Invited talk presented at Workshop on Intense Proton Sources, ICFA International Workshop, Fermilab, April 2002.

91. D. Michael, “The MINOS Experiment”, Talk at Neutrino 2002, Munich, Germany, May 2002.

92. D. Michael, “The MINOS Experiment”, Talk at DPF 2000, Columbus, OH, Aug. 2000.

93. D. Michael, “The MINOS Scintillator System”, Talk at DPF 2000, Columbus, OH, Aug. 2000.

94. D. Michael, “Neutrino Induced Upgoing Muons in MACRO”, Talk at DPF 2000, Columbus, OH, Aug. 2000.

95. B. Choudhary, “Production of MINOS Scintillator”, Talk at DPF 2000, Columbus, OH, Aug. 2000.

96. H. Kim, “PMTs for MINOS”, Talk at DPF 2000, Columbus, OH, Aug. 2000.

97. D. Michael, “Search for WIMPs with MACRO”, Presented at Dark Matter 2000, Marina Del Rey, CA, Feb. 2000.

98. B. Barish etal., DOE/NSF High Supanel Report on Long Range Planning for U.S. High-Energy Physics, January, 2002. E. Theses (2002 – Present)

0 + − ∗± ∓ 1. Werner M. Sun, “Observation of B → KSπ π and B → K π and measurement of the charge asymmetry in B → K∗±π∓”, Ph.D. thesis, Caltech (2003).

2. Peter Lett, “D-Branes in Anti-Sitter Space,” Ph.D. thesis, Caltech (2003).

3. Jongwon Park, “String/Gauge Duality and Penrose Limit,” Ph.D. thesis, Caltech (2003).

4. Federico M.Spedalieri, “Characterizing Entanglement in Quantum Information,” Ph.D. thesis, Caltech (2003).

5. Lei Xia, “Search for Scalar Leptons at LEP with the L3 Detector”, Ph.D. thesis, Caltech (2002).

6. Sophia Kyriazopoulou, “A Search for Slow Magnetic Monopoles Below the Parker Bound”, Ph.D. thesis, Caltech (2002). 7. John Landahl, “Controlling Quantum Information,” Ph.D. thesis, Caltech (2002).

265