Vanamala Venkataswamy
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
LNCS 8147, Pp
Mahasen: Distributed Storage Resource Broker K.D.A.K.S. Perera1, T. Kishanthan1, H.A.S. Perera1, D.T.H.V. Madola1, Malaka Walpola1, and Srinath Perera2 1 Computer Science and Engineering Department, University Of Moratuwa, Sri Lanka {shelanrc,kshanth2101,ashansa.perera,hirunimadola, malaka.uom}@gmail.com 2 WSO2 Lanka, No. 59, Flower Road, Colombo 07, Sri Lanka [email protected] Abstract. Modern day systems are facing an avalanche of data, and they are be- ing forced to handle more and more data intensive use cases. These data comes in many forms and shapes: Sensors (RFID, Near Field Communication, Weath- er Sensors), transaction logs, Web, social networks etc. As an example, weather sensors across the world generate a large amount of data throughout the year. Handling these and similar data require scalable, efficient, reliable and very large storages with support for efficient metadata based searching. This paper present Mahasen, a highly scalable storage for high volume data intensive ap- plications built on top of a peer-to-peer layer. In addition to scalable storage, Mahasen also supports efficient searching, built on top of the Distributed Hash table (DHT) 1 Introduction Currently United States collects weather data from many sources like Doppler readers deployed across the country, aircrafts, mobile towers and Balloons etc. These sensors keep generating a sizable amount of data. Processing them efficiently as needed is pushing our understanding about large-scale data processing to its limits. Among many challenges data poses, a prominent one is storing the data and index- ing them so that scientist and researchers can come and ask for specific type of data collected at a given time and in a given region. -
An Interoperable & Optimal Data Grid Solution For
An Interoperable & Optimal Data Grid Solution for Heterogeneous and SOA based Grid- GARUDA Payal Saluja, Prahlada Rao B.B., ShashidharV, Neetu Sharma, Paventhan A. Dr. B.B Prahlada Rao [email protected] s IPDPS 2010, Atlanta 2010, IPDPS s ’ 19 April 2010 IPDPS10 System Software Development Group Centre for Development of Advanced Computing HPGC Workshop, IEEE Workshop, HPGC C-DAC Knowledge Park, Bangalore, India GARUDA DataGrid Solutions:GSRM – Prahlada Rao.. et all 1 Presentation Outline • Grid Storage Requirements • Various Data Grid Solutions • Comparision of Grid data Solutions • Indian National Grid –GARUDA • Data Management Challenges for GARUDA • GARUDA Data Storage Solution - GSRM • GSRM Highlights • GSRM Architecture Integration with GARUDA s IPDPS 2010, Atlanta 2010, IPDPS s middleware components ’ • GSRM Usage Scenario for Par. applications • Conclusions HPGC Workshop, IEEE Workshop, HPGC GARUDA DataGrid Solutions:GSRM – Prahlada Rao.. et all 2 Grid Storage Requirements • Data Availability • Security • Performance & Latency • Scalability • Fault Tolerance s IPDPS 2010, Atlanta 2010, IPDPS s ’ HPGC Workshop, IEEE Workshop, HPGC GARUDA DataGrid Solutions:GSRM – Prahlada Rao.. et all Data Grid Solutions & Supported Storage Systems Data Grid Storage Systems Storage Grid File Storage Resource System Resource iRODS WS-DAI Broker Manager Gfarm StoRM GPFS DPM Lustre dCache Xtreemfs Grid Storage Storage Grid Solutions Bestman NFSv4 1.File Systems 1.File systems 1. File systems 1.File Systems AMG 2. Archives 2.Parallel File 2. Parallel File 2. Archives A WS- 3. Storage Area Systems Systems 3. Storage DAI s IPDPS 2010, Atlanta 2010, IPDPS s ’ Network (SAN) 3.Object 3.Object storage Area Network 4. Data Bases storage device (SAN) WS- 4. -
Grid Resource Broker
GRID RESOURCE BROKER ABHISHEK SINGH CSE Deptt. University at Buffalo Fall - 2006 Agenda • Grid Resource Broker- Architecture and Algorithm • Adaptable Resource Broker ( Drawback and Proposed solution) • SDSC Storage Resource Broker (SRB) • Future Enhancements A Sample Grid Grid Resource Broker INTERFACE Resource Resource Information Filter Lookup Services App/User Resource Resource Ranker Make match Fig 1. Resource Broker Architecture showing main components of the Design. Information Service : GIIS Tell me about the computing resources belonging to the HPC Lab that are uniprocessor Linux workstation, with low CPU load and available memory < 250 Mbyte XYZ.abc GRIS GRIS GIIS GRIS GRIS WEB Information Service: GRIS Tell me about the features of “xyz.abc” WEB GRIS Xyz.abc Grid Resource Broker INTERFACE Resource Resource Information Filter Lookup Services App/User Resource Resource Ranker Make match Fig 1. Resource Broker Architecture showing main components of the Design. Resource Brokering Algorithm • Input :- one or more job request • Action :- Select and submit job to most appropriate resources. • Output: none 1. Contact GIIS server(s) to obtain a list of available clusters. 2. Contact each resource's GRIS for static for static and dynamic resource information( hardware and software characteristics, current queue and load, etc.) 3. For each job :- (a) Select the cluster to which the job will be submitted. i. Filter out clusters that do not fulfill the requirements on memory, disk space architecture etc, and clusters that the user are not authorize to use. ii. Estimate the Total Time to Delivery (TTD) for each remaining resource. iii. Select the cluster with the shortest predicted TTD. -
DISCOVER DISCOVER CYBERINFRASTRUCTURE Terashake: Simulating “The Big One” COVER STORY
San Diego Supercomputer Center SPECIAL ISSUE FOR SC2004 DISCOVER DISCOVER CYBERINFRASTRUCTURE TeraShake: Simulating “the Big One” COVER STORY TERASHAKE EARTHQUAKE SIMULATION Ground surface velocity predicted for earthquake waves in the large-scale SCEC/CME, GEOFFREY ELY, UCSD TeraShake simulation of magnitude 7.7 earthquake on the southern San Andreas fault. The simulation ran for more than four days on SDSC’s 10 teraflops IBM DataStar supercomputer, producing an unprecedented 47 terabytes of data. Figure displays detailed fault-parallel velocity, showing the intense back- and-forth shaking predicted, with blue indicating upward motion and red indicating downward motion. The north-to-south fault rupture (thick white line) produced especially intense ground motion in the sediment-filled basin of the Coachella Valley which “shook like a bowl of jelly,” reaching peak velocities of more than 2 meters per second. TeraShake research will advance basic earthquake science and eventually help design more earthquake-resistant structures. 4 Terashake: Simulating the “Big One” on the San Andreas Fault THE SAN DIEGO SUPERCOMPUTER CENTER SDSC cyberinfrastructure Founded in 1985, the San Diego Supercomputer Center models 7.7 earthquake in (SDSC) has a long history of enabling science and engineering discoveries. Continuing this legacy into the unprecedented detail. next generation, SDSC’s mission is to “extend the reach” of researchers and educators by serving as a core resource for cyberinfrastructure— providing them with high-end hardware technologies, integrative software technologies, and deep interdisciplinary expertise. SDSC is an organized research unit of the University of California, San Diego and is primarily funded by the National Science Foundation (NSF). -
Data Grid Management Systems
Arun Jagatheesan Reagan Moore San Diego Supercomputer Center (SDSC) University of California, San Diego {arun, moore} @sdsc.edu University of Florida 1 San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure Storage Resource Broker • Distributed data management technology • Developed at San Diego Supercomputer Center (Univ. of California, San Diego) • 1996 - DARPA Massive Data Analysis • 1998 - DARPA/USPTO Distributed Object Computation Testbed • 2000 to present - NSF, NASA, NARA, DOE, DOD, NIH, NLM, NHPRC • Applications • Data grids - data sharing • Digital libraries - data publication • Persistent archives - data preservation • Used in national and international projects in support of Astronomy, Bio- Informatics, Biology, Earth Systems Science, Ecology, Education, Geology, Government records, High Energy Physics, Seismology University of Florida 2 San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure Acknowledgement: SDSC SRB Team ¾ Arun Jagatheesan ¾ George Kremenek ¾ Sheau-Yen Chen ¾ Arcot Rajasekar ¾ Reagan Moore ¾ Michael Wan ¾ Roman Olschanowsky ¾ Bing Zhu ¾ Charlie Cowart Not In Picture: ¾ Wayne Schroeder ¾ Tim Warnock(BIRN) ¾ Lucas Gilbert ¾ Marcio Faerman (SCEC) ¾ Antoine De Torcy Students: Emeritus: Xi (Cynthia) Sheng Vicky Rowley (BIRN) Allen Ding Qiao Xin Grace Lin Daniel Moore Jonathan Weinberg Ethan Chen Yufang Hu Reena Mathew Yi Li Erik Vandekieft Ullas Kapadia University of Florida 3 San Diego Supercomputer Center National Partnership for Advanced