Data Communications Needs: Advancing the Frontiers of Science Through Advanced Networks and Networking Research An ASCAC Report: Ellen Stechel, Chair Bill Wing, Co-Chair February 2008 Extending the Frontiers of Science through Advanced Networks and Networking Research FINAL DRAFT: 12 FEB 2008 Report of the Advanced Scientific Computing Advisory Committee Networking Subpanel Data Communication Needs: Advancing the Frontiers of Science Through Advanced Networks and Networking Research ASCAC Networking Panel Members ELLEN B. STECHEL, Sandia National Laboratories (Chair) WILLIAM R. WING, Oak Ridge National Laboratories (Co-Chair) F. RONALD BAILEY, Advanced Management Technology, Inc. JAMES CORONES, Krell Institute IAN FOSTER, Argonne National Laboratory HARVEY NEWMAN, California Institute of Technology GWENDOLYN HUNTOON, Pittsburgh Supercomputing Center LARRY RAHN, Sandia National Laboratories HORST SIMON, Berkeley National Laboratory JERRY SOBIESKI, Mid-Atlantic Crossroads Submitted to the ASCAC on 12 February 2008 for consideration as a: Report of the Advanced Scientific Computing Advisory Committee March 2008 Page 2 of 58 ASCAC – February 08 Data Communication Needs Extending the Frontiers of Science through Advanced Networks and Networking Research Table of Contents 1 EXECUTIVE SUMMARY ............................................................................................................5 2 CONTEXT.......................................................................................................................................8 2.1 THE CHARGE ..................................................................................................................................8 2.1.1 As Written.............................................................................................................................................8 2.1.2 As Interpreted........................................................................................................................................9 2.2 Background: Networks and Networking Research within SC ....................................................10 2.2.1 Networking Research..........................................................................................................................10 2.2.2 Deployment of Advanced Networks for DOE Science.......................................................................11 3 FORCES and TRENDS................................................................................................................16 3.1 Trends in Computation ...................................................................................................................16 3.2 Trends in Collaborations and Virtual Communities ....................................................................16 3.3 Trends in Data..................................................................................................................................17 3.4 Trends in Society..............................................................................................................................18 3.5 Shifting Paradigm in Scientific Research ......................................................................................18 3.6 Summary...........................................................................................................................................19 4 Key Findings..................................................................................................................................21 4.1 ESnet’s Exemplary Job ...................................................................................................................21 4.1.1 Findings ..............................................................................................................................................21 4.1.2 Basis....................................................................................................................................................21 4.1.3 Discussion...........................................................................................................................................22 4.1.4 Recommendation ................................................................................................................................22 4.1.5 Challenges to Implementing the Recommendations...........................................................................22 4.2 Dependence of Petascale Facilities on High-Performance Networks ..........................................24 4.2.1 Finding................................................................................................................................................24 4.2.2 Basis....................................................................................................................................................24 4.2.3 Discussion...........................................................................................................................................25 4.2.4 Recommendations...............................................................................................................................27 4.3 Advanced Networks as Enablers of New Modes of Scientific Inquiry........................................28 4.3.1 Findings ..............................................................................................................................................28 4.3.2 Basis....................................................................................................................................................29 4.3.3 Discussion...........................................................................................................................................29 4.3.4 Recommendations...............................................................................................................................29 4.3.5 Challenges to Implementing the Recommendations...........................................................................30 4.4 Petascale Facilities as a System of Systems....................................................................................33 4.4.1 Findings ..............................................................................................................................................33 4.4.2 Basis....................................................................................................................................................33 4.4.3 Discussion:..........................................................................................................................................34 4.4.4 Recommendations...............................................................................................................................34 4.5 Research and the “Valley of Death”...............................................................................................35 4.5.1 Findings ..............................................................................................................................................35 4.5.2 Basis....................................................................................................................................................35 Data Communication Needs ASCAC – February 08 Page 3 of 58 Extending the Frontiers of Science through Advanced Networks and Networking Research 4.5.3 Discussion:..........................................................................................................................................35 4.5.4 Recommendation ................................................................................................................................36 4.6 Need for a Coherent Networking Research Program...................................................................38 4.6.1 Findings ..............................................................................................................................................38 4.6.2 Basis....................................................................................................................................................38 4.6.3 Discussion...........................................................................................................................................40 4.6.4 Recommendations...............................................................................................................................41 4.6.5 Challenges to Implementing the Recommendation.............................................................................41 5 CLOSING REMARKS.................................................................................................................43 APPENDICES A. The Charge Letter............................................................................................................................44 B. Acknowledgments ............................................................................................................................45 C. Subpanel Biographies ......................................................................................................................45 D. ASCAC Networking Subcommittee Meeting April, 2007 ............................................................49 E. Selected References..........................................................................................................................53 F. Abbreviations/Acronyms.................................................................................................................54