Things You Need to Know About Big Data
Total Page:16
File Type:pdf, Size:1020Kb
Critical Missions Are Built On NetApp NetApp’s Big Data solutions deliver high performance computing, full motion video (FMV) and intelligence, surveillance and reconnaissance (ISR) capabilities to support national safety, defense and intelligence missions and scientific research. UNDERWRITTEN BY To learn more about how NetApp and VetDS can solve your big data challenges, call us at 919.238.4715 or visit us online at VetDS.com. ©2011 NetApp. All rights reserved. Specifications are subject to change without notice. NetApp, the NetApp logo, and Go further, faster are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such. Critical Missions Are Things You NeedBuilt On NetApp to Know About Big Data NetApp’s Big Data solutions deliver high Big data has been making big news in fields rangingperformance from astronomy computing, full motionto online video advertising. (FMV) and intelligence, surveillance and By Joseph Marks The term big data can be difficult to pin down because it reconnaissanceshows up in (ISR) so capabilities many toplaces. support Facebook national safety, defense and intelligence crunches through big data on your user profile and friend networkmissions and to scientific deliver research. micro-targeted ads. Google does the same thing with Gmail messages, Search requests and YouTube browsing. Companies including IBM are sifting through big data from satellites, the Global Positioning System and computer networks to cut down on traffic jams and reduce carbon emissions in cities. And researchers are parsing big data produced by the Hubble Space Telescope, the Large Hadron Collider and numerous other sources to learn more about the nature and origins of the universe. These processes all involve large amounts of information that were once too vast and messy for even computers to analyze. Now that data can be mined for patterns and insights, some of which could spawn major advances in everything from theoretical physics to basic government services. In other words, big data is a chance to take all the things we don’t know we know and finally know them. To learn more about how NetApp and VetDS can solve your big data Here are five important things to know about big data. challenges, call us at 919.238.4715 or visit us online at VetDS.com. ©2011 NetApp. All rights reserved. Specifications are subject to change without notice. NetApp, the NetApp logo, and Go further, faster are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such. RETHINKING BASIC SERVICES The paradigm shift of big data is to invest less in producing new data and more in maximizing the data that already exists. Health researchers, for instance, can mix anonymous information from electronic health records with information about pollution, traffic and purchasing habits in different regions to learn about risk factors for certain diseases. That same data can be brought to bear on the industry as well, pointing out the most common medical errors and who benefits most and least from certain treatments. Big data is at the root of a predictive analytics program that officials expect could drastically cut the roughly $50 billion lost each year to fraudulent and improper Medicare and Medicaid claims. “The old way of collecting data was to only collect it . when a human generates it,” says Amr Awadallah, co-founder and chief technology officer of Cloudera, a company that maintains and manages big data systems. “We called that an explicit transaction. Now we’re collecting implicit information. We have all these sensors around humans in mobile devices and satellites taking images and there are Web services collecting information about you all the time nonstop.” THE BUILDING BLOCKS At the root of most big data crunching systems is the open source software Apache Hadoop. Its major innovations include the ability to link multiple computers and servers, either in a proprietary data center or a computer cloud, and make them work like one huge network that can scale up for a major task. Another breakthrough is the ability to sort through unstructured data as well as structured data. Unstructured data is essentially information that isn’t already slotted into categories, as in a spreadsheet. It can be anything from social media posts to the text in old PDF reports to video from a satellite feed. A BIG BET ON BIG DATA The Obama administration announced in March that it’s putting big money behind big data in the form of $200 million in research funding across multiple agencies. The initiative was sparked by a June 2011 report from the President’s Council of Advisors on Science and Technology, which found a gap in the private sector’s investment in big data research and development. The National Institutes of Health, for instance, announced a plan to put a data set from the human genome project in Amazon’s EC2 computer cloud with tools to make the information easily accessible to researchers. Assistant Secretary of Defense for Research and Engineering Zachary Lemnios announced plans to develop predictive and learning tools that can use big data to make “truly autonomous” defense systems that “can learn from experience with very little training and learn the limits of their own knowledge.” Other agencies involved in the initiative include the U.S. Geological Survey, the Defense Advanced Research Projects Agency and the Energy Department. GIANT STEPS FOR SCIENCE One of the most powerful new tools in science, big data is driving discoveries in areas such as astronomy, genetics, climate science and epidemiology. The cost of sequencing a human genome, for example, has dropped from many millions of dollars when it was first completed in 2003 to just $8,000, partly due to the ease with which analytical tools can crawl through huge amounts of genetic data. Big data mining has effectively transformed astronomy from an observational science into a lab science, according to Johns Hopkins University physics and astronomy professor Alex Szalay. It allows astronomers to model astronomical phenomena and determine what might have happened in the past. In one example, astronomers have used big data to model resonance frequencies the universe emits—similar to monitoring beats on a drum—to deduce how the Big Bang might have played out. THE PLACES You’ll GO Big data isn’t just making its mark in the hard sciences. Researchers in the social sciences and humanities are using text mining to plow through millions of scanned and digitized books, journals and pamphlets to find certain words, phrases and themes. At its most basic level, etymologists can use text mining to trace the origin and changing use of language. Scholars could even compare how certain expressions made their way through literary traditions. Historians also can use the tools to gather quantitative insights into who was discussing disunion in advance of the Civil War or who had protectionist sympathies before World War II to supplement the more common qualitative studies. Critical Missions Are Built On NetApp NetApp’s Big Data solutions deliver high performance computing, full motion video (FMV) and intelligence, surveillance and reconnaissance (ISR) capabilities to support national safety, defense and intelligence missions and scientific research. To learn more about how NetApp and VetDS can solve your big data challenges, call us at 919.238.4715 or visit us online at VetDS.com. ©2011 NetApp. All rights reserved. Specifications are subject to change without notice. NetApp, the NetApp logo, and Go further, faster are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such..