DOCSLIB.ORG

Big Data and Computing Building a Vision for ARS Information Management

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Big Data and Computing Building a Vision for ARS Information Management Big Data and Computing Building a Vision for ARS Information Management Workshop Summary Feb. 5-6, 2013 USDA Agricultural Research Service TABLE OF CONTENTS TABLE OF CONTENTS ............................................................................................................................. 2 INTRODUCTION ....................................................................................................................................... 3 BIG DATA AND COMPUTING ................................................................................................................ 5 DEFINING RESEARCH NEEDS ............................................................................................................................................................... 5 STEPS TO ACHIEVE THE VISION: SPECIFIC RECOMMENDATIONS .................................................................................................... 6 CONCLUSIONS .......................................................................................................................................... 8 TABLE 1. SUMMARY OF RECOMMENDATIONS. ................................................................................................................................. 9 ACKNOWLEDGEMENTS...................................................................................................................................................................... 10 APPENDIX 1: EXAMPLES OF HOW IMPROVED BIG DATA CAPACITY WILL INCREASE ARS SCIENTIFIC CAPACITY .......................................................................................................................... 11 APPENDIX 2: WORKSHOP AGENDA .................................................................................................. 13 APPENDIX 3. ACRONYMS DEFINED .................................................................................................. 14 2 INTRODUCTION Box 1 On February 5-7, 2013, scientific leaders from the Agricultural The Bigness Research Service (ARS) held a workshop to identify scientific of Big Data information management needs within the agency, and to find solutions to address those needs. Workshop participants represented The phrase ‘Big Data’ scientists from all National Programs and each geographic area. obviously implies volume, but Speakers from industry, academia, and federal agencies provided IT industry analysts and information about their experiences with Big Data. Participants were consultants have long challenged to articulate a vision for ARS information management acknowledged other unconstrained by current ARS capabilities and to formulate a characteristics of data that add strategy to achieve that vision. This document outlines the commensurate challenges. The recommendations arising from that workshop. discussion remains vigorous, to the point of moving from the The ARS is a worldwide leader in agricultural research, providing a description of Big Data to that unique breadth and depth of scientific expertise to address problems of Extreme Data. of national and international importance. ARS scientists use leading- edge methods to solve problems and answer questions across a broad For the purposes of this white array of disciplines related to agriculture. The scope of research paper, Big Data is undertaken by the agency is highlighted by the ARS Mission characterized as having Statement: extreme or variable values of ARS conducts research to develop and transfer solutions to one or more of the following agricultural problems of high national priority and provide characteristics: information access and dissemination to: Volume1 (size) • ensure high-quality, safe food, and other agricultural Variety1 (structure) products • Velocity1 (acquisition rate) • assess the nutritional needs of Americans • Veracity (uncertain quality or • sustain a competitive agricultural economy • provenance) • enhance the natural resource base and the • Variability2 (in meaning) environment, and Complexity3 (in relation- • provide economic opportunities for rural citizens, • ships, sources, etc.) communities, and society as a whole. • However, the nature of the science supporting this mission is 1. Doug Laney. 2001. 3-D Data Management: Controlling Data changing rapidly. In the past, scientific methods were often labor- Volume, Velocity, and Variety. intensive and consumed many person-hours to adequately address a 2. Brian Hopkins. 2011. Blogging From single scientific question. Scientists are now generating vast amounts the IBM Big Data Symposium - Big Is More Than Just Big. 2011. Brian of high-quality data rapidly and relatively inexpensively. This Hopkins. fundamental change in the nature of science is presenting new 3. Valentin Sribar. 2011. ‘Big Data' Is challenges and demanding new approaches to maximize the value Only the Beginning of Extreme Information Management. extracted from these large and complex datasets. This dramatic growth in data volume, variety, and velocity has come to be known as Big Data (Box 1). 3 As a result of these changes, a new paradigm is emerging in science that is characterized by its data intensity. Previous methods for data collection, storage, and analysis are inadequate for handling the scale and complexity of this avalanche of new data. Consumer-oriented services such as Google Maps, Facebook, Wikipedia, and the National Oceanic and Atmospheric Administration (NOAA) weather forecasts, demonstrate the benefits that can be gained by aggregating data from multiple sources. These products have become essential resources intensively adopted by millions of people in an astonishingly brief time. Similarly, the vast data resources produced by ARS scientists could be used to describe the world in ways never imagined, but critical resources and coordination to maximize this impact are missing. A few examples of agricultural research that are enabled by, and even necessitate, such a paradigm shift in scientific information management include: • Increasing the resilience of production systems: Deriving local or regional responses to environmental stressors requires defining the complex interactions among crops, animals, soil, water, weather, and climate. A better understanding of these interactions can be gleaned, even using current data or data collection strategies, but only with an improvement in how the vast amounts of data are integrated across locations. • Improving our understanding of genotype by environment interactions: In all organisms, the environment affects phenotype, and variability in the environment can confound attempts to identify genes underlying important agronomic traits, and even human diseases. Traditional studies typically only identify genes with large effects, but many important traits are driven by multiple genes that are easily affected by the environment. Identifying these genes requires whole genome studies coupled with good environmental data for each individual. Such studies require high computing capacity for analyzing very large datasets. • Enhancing human and animal health: Human and animal nutrition, health, and well- being can be better predicted by understanding the rich interactions among microbial communities through insights gleaned from the genomes of those community members. For a more extensive discussion of these and other examples, see Appendix 1 (p. 11). Enhancements in scientific computing in the ARS are both critical and urgent. The ARS must upgrade computing and data management infrastructure to maintain its status as a world leader in the science of agriculture. The Big Data paradigm shift in science is taking place across disciplines, and action is necessary for ARS scientists to effectively perform key research missions. Enhancements to current systems are vital for maintaining the ARS as an effective, nationally-coordinated, multidisciplinary agency. Current research priorities target objectives that are unattainable without some broad, systematic improvements. ARS scientists cannot perform state of the art research without the appropriate tools, knowledge, and skills. 4 BIG DATA AND COMPUTING Participants at the Big Data Workshop expressed enthusiastic support of the worldwide leadership provided by the ARS in agricultural research and embraced the role of the agency to lead in the collection, storage, analysis, and distribution of scientific data related to agriculture (see Box 2). Workshop participants agreed that the ARS is uniquely positioned to provide long-term, stabile, durable solutions for information management challenges involving Big Data. Participants reported that urgent solutions are needed to address these technology issues to help meet scientific needs, needs that reach across agricultural research institutions and disciplines. This enhanced role for the ARS is consistent with the Big Data initiative recently announced by the White House Office of Science and Technology Policy (OSTP). Box 2 Big Data Vision Statement Scientists in the ARS and elsewhere are generating data at increasing rates. To maximize knowledge extracted from these large and diverse data sets, ARS should support scientific computing to efficiently combine disparate information for scientific discovery, and to enable the transfer of that knowledge quickly and efficiently to other scientists and to the public. The enhancements envisioned here will: • enable or enhance data exchange and analysis • create a more robust network of computer resources • encourage data standardization, and • help to develop a well-trained
Load more

Recommended publications
  • A Bold New Approach for Space Exploration and Discovery
    Office of Science and Technology Policy National Aeronautics and Space Administration FACT SHEET A Bold New Approach for Space Exploration and Discovery • Adds $6 billion to NASA’s budget over five years and draws upon American ingenuity to enable us to embark on an ambitious 21st century program of human space exploration and observation of the Earth and the Universe. • Ends NASA’s Constellation program, which was planning to use an approach similar to the Apollo program to return astronauts to the Moon 50 years after that program’s triumphs. An independent panel found that Constellation was years behind schedule and would require large budget increases to land even a handful of astronauts back on the Moon before 2030. Instead, we are launching a bold new effort that invests in American ingenuity to develop more capable and innovative technologies for future space exploration. • In consultation with our partners, extends operation of the International Space Station likely to 2020 or beyond and enhances its utilization, bringing nations together in a common pursuit of discovery in space. • Initiates several new programs to transform the state of the art in space technologies, including flagship exploration technology development and demonstration programs, investments in early-stage advanced concepts, potential “game-changing” technologies, and new propulsion technologies – all intended to increase the reach and reduce the costs of future human space exploration as well as other NASA, government, and commercial space activities. • Enhances the Nation’s global climate change research and monitoring system, accelerating decadal survey missions and re-flying a satellite that will identify global carbon sources and sinks.
    [Show full text]
  • The U.S. Department of Energy's Ten-Year-Plans for the Office Of
    U.S. DEPARTMENT OF ENERGY The U.S. Department of Energy’s Ten-Year-Plans for the Office of Science National Laboratories FY 2019 FY 2019 Annual Laboratory Plans for the Office of Science National Laboratories i Table of Contents Introduction ................................................................................................................................................................1 Ames Laboratory ........................................................................................................................................................3 Lab-at-a-Glance ......................................................................................................................................................3 Mission and Overview ............................................................................................................................................3 Core Capabilities .....................................................................................................................................................4 Science Strategy for the Future ..............................................................................................................................8 Infrastructure .........................................................................................................................................................8 Argonne National Laboratory .................................................................................................................................
    [Show full text]
  • Agency Information
    Quadrennial Technology Review 2015 Chapter 1: Energy Challenges Supplemental Information Additional Information on Energy Challenges Agency Information Representative DOE Science and Energy Program Workshops U.S. DEPARTMENT OF ENERGY Quadrennial Technology Review 2015 Agency Information Chapter 1: Supplemental Information Agency Information Many Federal Departments and Agencies conduct research, development, demonstration, and deployment (RDD&D) activities, policy and regulatory activities, and other actions related to clean energy technologies, as do many State Agencies. The following websites provide a starting point for examining some of the excellent work done by these Federal Agencies—with a primary focus on RDD&D activities, as well as those providing data and information helpful for informing those activities. State and local activities are not covered here. This is not a complete list across agencies or within agencies, only a starting point. For all of these agencies, searching their websites for the energy-related topics of interest is the best way to get up-to-date information. For regulatory and policy issues, the work done by the Quadrennial Energy Review provides a useful overview and framework.1 AGENCY/DIVISION and URL: Executive Office of the President (EOP) https://www.whitehouse.gov/administration/eop Council on Environmental Quality (CEQ) https://www.whitehouse.gov/administration/eop/ceq National Security Council (NSC) https://www.whitehouse.gov/administration/eop/nsc Office of Management and Budget (OMB) https://www.whitehouse.gov/omb Office of Science and Technology Policy (OSTP) https://www.whitehouse.gov/administration/eop/ostp U.S. Department of Agriculture (USDA) http://www.usda.gov/wps/portal/usda/usdahome http://www.usda.gov/wps/portal/usda/usdahome?navid=ENERGY http://www.usda.gov/oce/energy/ http://www.ars.usda.gov/research/programs/programs.htm?NP_CODE=307 http://www.usda.gov/oce/reports/index.htm#renewable http://www.rd.usda.gov/about-rd/agencies/rural-utilities-service U.S.
    [Show full text]
  • Direct Federal Financial Interventions and Subsidies in Energy in Fiscal Year 2016
    Direct Federal Financial Interventions and Subsidies in Energy in Fiscal Year 2016 April 2018 Independent Statistics & Analysis U.S. Department of Energy www.eia.gov Washington, DC 20585 This report was prepared by the U.S. Energy Information Administration (EIA), the statistical and analytical agency within the U.S. Department of Energy. By law, EIA’s data, analyses, and forecasts are independent of approval by any other officer or employee of the United States Government. The views in this report therefore should not be construed as representing those of the U.S. Department of Energy or other federal agencies. U.S. Energy Information Administration | Financial Interventions and Subsidies i April 2018 Contacts This report, Direct Federal Financial Interventions and Subsidies in Energy in Fiscal Year 2016, was prepared under the general guidance of Ian Mead, Assistant Administrator for Energy Analysis; Jim Turnure at 202/586-1762 (email, [email protected]), Director, Office of Energy Consumption and Efficiency Analysis; and Shirley Neff, Senior Advisor, EIA. Technical information concerning the content of the report also may be obtained from Mark Schipper at 202/586-1136 (email, [email protected]) and technical information on the subsidies and support to the electric power industry may be obtained from Chris Namovicz at 202/586-7120 (email, [email protected]). Contributing authors, by fuel or technology subsidy and support issue areas, are as follows • Richard Bowers and Fred Mayes–renewables (electricity) subsidies and support
    [Show full text]
  • Agricultural Research: Background and Issues
    Agricultural Research: Background and Issues Updated October 2, 2020 Congressional Research Service https://crsreports.congress.gov R40819 SUMMARY R40819 Agricultural Research: Background and Issues October 2, 2020 The U.S. Department of Agriculture (USDA) Research, Education, and Economics (REE) mission area funds billions of dollars annually for biological, physical, and social Genevieve K. Croft science research that is related to agriculture, food, and natural resources. Four agencies Analyst in Agricultural carry out REE responsibilities: the Agricultural Research Service (ARS), the National Policy Institute of Food and Agriculture (NIFA), the National Agricultural Statistics Service (NASS), and the Economic Research Service (ERS). The Under Secretary for REE, who oversees the REE agencies, holds the title of USDA Chief Scientist and is responsible for coordinating research, education, and extension activities across the entire department. The Office of the Chief Scientist (OCS)—a staff office within the Office of the Under Secretary for REE—supports this coordination role. Discretionary funding for the REE mission area totaled approximately $3.4 billion in FY2020, and mandatory funding from the 2018 farm bill adds another $177 million per year on average. USDA administers federal funding to states and local partners through its extramural research agency: NIFA. NIFA administers this extramural funding through capacity grants (allocated to the states based on formulas in statute) and competitive grants (awarded based on a peer-review process). USDA also conducts its own research at its intramural research agencies: ARS, NASS, and ERS. Debates over the direction of public agricultural research and the nature of how it is funded continue. Ongoing issues include whether federal funding is sufficient to support agricultural research, education, and extension activities; the different roles of extramural versus intramural research; and the implications of allocating extramural funds via capacity grants versus competitive grants.
    [Show full text]
  • PCAST Report
    INDUSTRIES OF THE FUTURE INSTITUTES: A NEW MODEL FOR AMERICAN SCIENCE AND TECHNOLOGY LEADERSHIP A Report to the President of the United States of America The President’s Council of Advisors on Science and Technology January 2021 INDUSTRIES OF THE FUTURE INSTITUTES: A NEW MODEL FOR AMERICAN SCIENCE AND TECHNOLOGY LEADERSHIP About the President’s Council of Advisors on Science and Technology Created by Executive Order in 2019, PCAST advises the President on matters involving science, technology, education, and innovation policy. The Council also provides the President with scientific and technical information that is needed to inform public policy relating to the American economy, the American worker, national and homeland security, and other topics. Members include distinguished individuals from sectors outside of the Federal Government having diverse perspectives and expertise in science, technology, education, and innovation. More information is available at https://science.osti.gov/About/PCAST. About this Document This document follows up on a recommendation from PCAST’s report, released June 30, 2020, involving the formation of a new type of multi-sector research and development organization: Industries of the Future Institutes (IotFIs). This document provides a framework to inform the design of IotFIs and thus should be used as preliminary guidance by funders and as a starting point for discussion among those considering participation. The features described here are not intended to be a comprehensive list, nor is it necessary that each IotFI have every feature detailed here. Month 2020 – i – INDUSTRIES OF THE FUTURE INSTITUTES: A NEW MODEL FOR AMERICAN SCIENCE AND TECHNOLOGY LEADERSHIP The President’s Council of Advisors on Science and Technology Chair Kelvin K.
    [Show full text]
  • Doe-Fy2021-Laboratory-Table 1.Pdf
    DOE/CF-0168 Department of Energy FY 2021 Congressional Budget Request Laboratory Tables Preliminary February 2020 Office of Chief Financial Officer DOE/CF-0168 Department of Energy FY 2021 Congressional Budget Request Laboratory Tables Preliminary The numbers depicted in this document represent the gross level of DOE budget authority for the years displayed. The figures include discretionary and supplemental funding. They do not consider revenues/receipts, use of prior year balances, deferrals, rescissions, or other adjustments appropriated as offsets to the DOE appropriations by the Congress. February 2020 Office of Chief Financial Officer Printed with soy ink on recycled paper Table of Contents Laboratory Table by Congressional Control Laboratory Table Summary......................................................................................................................................1 Laboratory Table by Congressional Control4 Ames Laboratory..................................................................................................................................................4 Ames Site Office...................................................................................................................................................5 Argonne National Laboratory...............................................................................................................................6 Argonne Site Office..............................................................................................................................................9
    [Show full text]
  • Undergraduate Internships
    Internship Programs https://internships.fnal.gov/ Undergraduate Internships Community College Internships (CCI) Summer technical training experience C for community college students. CCI is sponsored by the Department of Energy, Office of Science. Helen Edwards Summer Internship A ten-week summer internship for students majoring in physics and engineering at universities in Europe. Lee Teng Internship Accelerator science and engineering internship designed to attract undergraduate students to the exciting and challenging world of particle accelerator physics and technology. Summer Internships in Science and Technology (SIST) Undergraduate sophomores and juniors majoring in physics, engineering (mechanical, electrical and computer), materials science, mathematics and computer science conduct research with Fermilab scientists and engineers. Science Undergraduate Laboratory Internship (SULI) Sponsored by the Department of Energy Office of Science, undergraduate physics or engineering majors work with scientists or engineers on projects at the frontier of particle physics research. VetTech Internship Program Military veterans provide routine technical support for an assigned experiment or support group. The program assumes apprentices level expertise, general military experience and/or knowledge-based on academic study. Managed by Fermi Research Alliance, LLC for the U.S. Department of Energy Office of Science www.fnal.gov Graduate Internships Computational Science Graduate Fellowship (CSGF) CSGF provides outstanding benefits and opportunities
    [Show full text]
  • ARPA-E): Background, Status, and Selected Issues for Congress
    Order Code RL34497 Advanced Research Projects Agency - Energy (ARPA-E): Background, Status, and Selected Issues for Congress Updated August 12, 2008 Deborah D. Stine Specialist in Science and Technology Policy Resources, Science, and Industry Division Advanced Research Projects Agency - Energy (ARPA-E): Background, Status, and Selected Issues for Congress Summary In August 2007, Congress authorized the establishment of the Advanced Research Projects Agency - Energy (ARPA-E) within the Department of Energy (DOE) as part of the America COMPETES Act (P.L. 110-69). Modeled on the Defense Advanced Research Projects Agency (DARPA), ARPA-E would support transformational energy technology research projects with the goal of enhancing the nation’s economic and energy security. Proponents of ARPA-E contend that additional science and technology would help respond to the nation’s need for clean, affordable, and reliable energy. Opponents question whether ARPA-E is necessary to develop new technologies, when existing energy technologies are not fully utilized due to insufficient policies to encourage their implementation. ARPA-E proponents counter that ARPA-E is needed to catalyze the energy marketplace by accelerating research that will bridge the gap between basic research and industrial product development. The Bush Administration questions whether the DARPA model can be used for the energy sector and is concerned that it might redirect funds away from current DOE research activities, particularly the DOE Office of Science. Instead, the President’s FY2009 budget requests funding for six new technology transfer collaborations. ARPA-E proponents doubt that DOE can achieve ARPA-E’s goals with its existing structure and personnel, as opposed to the ARPA-E’s innovative R&D management design.
    [Show full text]
  • De-Foa-0002449
    DEPARTMENT OF ENERGY OFFICE OF SCIENCE BASIC ENERGY SCIENCES MATERIALS AND CHEMICAL SCIENCES RESEARCH FOR QUANTUM INFORMATION SCIENCE FUNDING OPPORTUNITY ANNOUNCEMENT (FOA) NUMBER: DE-FOA-0002449 FOA TYPE: INITIAL CFDA NUMBER: 81.049 FOA Issue Date: December 17, 2020 Submission Deadline for Pre-Applications: January, 27, 2021 at 5:00 PM Eastern A Pre-Application is required Pre-Application Response Date: March 1, 2021 Submission Deadline for Applications: April 14, 2021 at 11:59 PM Eastern Table of Contents SECTION I – FUNDING OPPORTUNITY DESCRIPTION ...................................................... 1 SECTION II – AWARD INFORMATION ................................................................................. 5 A. TYPE OF AWARD INSTRUMENT .................................................................................. 5 B. ESTIMATED FUNDING ................................................................................................... 5 C. MAXIMUM AND MINIMUM AWARD SIZE .................................................................. 5 D. EXPECTED NUMBER OF AWARDS .............................................................................. 6 E. ANTICIPATED AWARD SIZE ......................................................................................... 6 F. PERIOD OF PERFORMANCE .......................................................................................... 6 G. TYPE OF APPLICATION ................................................................................................. 6 SECTION III – ELIGIBILITY
    [Show full text]
  • NRI Cmte Federal Ag & Energy Nov. Newsletter
    Energy & Agriculture Update January 2019 | Volume 8, Issue 1 Top Stories 1/25/19 – Congress and the president reached an agreement to impacted by retaliatory tariffs to apply for trade-relief payments. enact a three-week continuing resolution through Feb. 15, USDA Secretary Sonny Perdue stated that "using existing funding the nine federal agencies (departments of funds, [the agency was] able to keep [Farm Service Agency Transportation, Agriculture, Homeland Security, Treasury, (FSA)] offices open as long as possible, but unfortunately had Commerce, Housing and Urban Development, Justice, State to close them when funding ran out," and that "[the agency] will and Interior, as well as the IRS, National Science Foundation, therefore extend the application deadline for a period of time FDA and EPA) that had been closed for 36 days as part of the equal to the number of business days FSA offices were closed, partial federal government shutdown. It remains unclear if once the government shutdown ends." Producers eligible for Congress and the president will reach a formal agreement trade-relief money previously had until Jan. 15 to apply for concerning federal funds for a barrier on the southern border. direct aid payments, but the government shutdown closed local Stay tuned to NCSL for further details on fiscal year 2019 FSA offices on Dec. 28 preventing new signups. appropriations. 1/6/19 – The Supreme Court of the United States (SCOTUS) 1/18/19 – The U.S. District Court for the District of South declined to take up two challenges to controversial California Carolina ordered the Department of Interior (DOI) to freeze all laws—bans on foie gras and the prohibition of the sale of eggs permit work on seismic surveys of offshore oil and gas deposits from caged hens, in addition to a similar caged-hen law in off the Atlantic coast due to the lack of current appropriations Massachusetts.
    [Show full text]
  • The DOE National Laboratories
    The DOE National Laboratories Brookhaven National Laboratory Community Advisory Council Frank Crescenzo, Manager Brookhaven Site Office September 11, 2014 Today’s DOE Laboratories Why do we have national laboratories? Couldn’t we simply outsource R&D to universities and industry? Community Advisory Council Meeting 2 Why do we have National Laboratories? • Execute long-term government scientific and technological missions, often with complex security, safety, project management, or other operational challenges; • Develop unique, often multidisciplinary, scientific capabilities beyond the scope of academic and industrial institutions, to benefit the Nation’s researchers and national strategic priorities; and • Develop and sustain critical scientific and technical capabilities to which the government requires assured access. Community Advisory Council Meeting 3 Why do we have National Laboratories? • To perform missions of national interest • To perform “government only” missions • To design, build and operate specialized scientific facilities that industry and academia will not/cannot – Usually very expensive to build and operate – Mostly available to user communities (academia, industry, national labs, others) – Designed and operated to meet user community needs Community Advisory Council Meeting 4 DOE and Its Predecessors and the Formation of the National Laboratories . 1942-1946 Manhattan Project, War Department Army Corps of Engineers – Wartime weapons development – Foundations of first DOE multi-purpose national labs . 1946-1974 Atomic Energy Commission created by the 1946 Atomic Energy Act (P.L. 79-585) – Research in basic nuclear processes, nuclear reactor technologies, use of nuclear materials for variety of purposes – Establishment of several DOE national labs . 1974-1977 Energy Research and Development Administration, a new energy R&D agency motivated by Arab oil embargo and created by (P.L.
    [Show full text]