Solving the HPC I/O Bottleneck: Sun Lustre Storage System
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
System Administration Storage Systems Agenda
System Administration Storage Systems Agenda Storage Devices Partitioning LVM File Systems STORAGE DEVICES Single Disk RAID? RAID Redundant Array of Independent Disks Software vs. Hardware RAID 0, 1, 3, 5, 6 Software RAID Parity done by CPU FakeRAID Linux md LVM ZFS, btrfs ◦ Later Hardware RAID RAID controller card Dedicated hardware box Direct Attached Storage SAS interface Storage Area Network Fiber Channel iSCSI ATA-over-Ethernet Fiber Channel Network Attached Storage NFS CIFS (think Windows File Sharing) SAN vs. NAS PARTITIONING 1 File System / Disk? 2 TB maybe… 2TB x 12? 2TB x 128 then? Partitioning in Linux fdisk ◦ No support for GPT Parted ◦ GParted Fdisk Add Partition Delete Partition Save & Exit Parted Add Partition Change Units Delete Partition No need to save Any action you do is permanent Parted will try to update system partition table Script support parted can also take commands from command line: ◦ parted /dev/sda mkpart pri ext2 1Mib 10Gib Resize (Expand) 1. Edit partition table ◦ Delete and create with same start position 2. Reload partition table ◦ Reboot if needed 3. Expand filesystem Resize (Shrink) 1. Shrink filesystem ◦ Slightly smaller than final 2. Edit partition table ◦ Delete and create with same start position 3. Reload partition table ◦ Reboot if needed 4. Expand filesystem to fit partition No Partition Moving LOGICAL VOLUME MANAGER What is LVM? A system to manage storage devices Volume == Disk Why use LVM? Storage pooling Online resizing Resize any way Snapshots Concepts Physical Volume ◦ A disk or partition Volume Group ◦ A group of PVs Logical Volume ◦ A virtual disk/partition Physical Extent ◦ Data blocks of a PV Using a partition for LVM Best to have a partition table 1. -
ORACLE MAGAZINE Jonathan Vincenzo, Dan Vlamis Is PROVIDED on an “As Is” Basis
JULY/AUGUST 2010 Oracle Enterprise Manager 11g New release delivers business-driven IT management / 21 Deriving and Sharing BI Metadata Integrate Oracle Warehouse Builder 11g Release 2 information with Oracle Business Intelligence Enterprise Edition / 59 Automating Parallelism Take the guesswork out of setting the degree of parallelism for queries / 63 On Connecting, Pivoting, and Learning New Things Our technologist bypasses TNSNAMES.ORA, turns rows to columns, and ORACLE.COM/ORACLEMAGAZINE meets the NO_DATA_FOUND exception / 69 THE VIRTUAL ENTERPRISE From desktop to datacenter, Oracle o ers a complete virtualization solution Strategize. Plan. Execute. Gain more insight with Oracle’s enhanced enterprise performance management solution Java Hits the Road Learn the latest from Java luminaries as the Java Bus goes from code to coast One City, One Week, Three Conferences Oracle OpenWorld, JavaOne, and Oracle Develop converge on San Francisco September 19–23 JA10_cover_R1.indd 2 5/25/10 2:30:41 PM Untitled-2 1 3/22/10 8:42 PM Untitled-2 2 3/22/10 8:42 PM Oracle development PL/SQL Developer by Allround Automations. everywhere... PL/SQL Developer is the Oracle development tool that gives you maximum productivity, ease of use and all the features you need, for a reasonable price. Visit our website for additional details: www.allroundautomations.com/plsqldev 4 FEATURED CONTENT VOLUME XXIV - ISSUE 4 CONTENTS THE VIRTUAL ENTERPRISE /36 Virtualization—of desktop, server, or storage resources—requires a reliable infrastructure, solid plan, and strategic management. The payoffs include IT flexibility and efficiency, as well as server consolidation and lower energy costs. Learn how Oracle’s desktop-to-datacenter solutions can virtualize your enterprise. -
Mysql Network
MySQL Bests practices on Linux Serge Frezefond [email protected] Dir. Technique SUN / MySQL France Solution Linux 2009 Paris, 02-04-2009 Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 1 Agenda • Architecture MySQL • DRBD Heartbeat • IO / File systems / ZFS • LVM for backups • Secure replication : checksum through SSL • Virtualization • Monitoring / Tuning • Linux memory : locking , swappiness • Dev MySQL on Linux. • Conclusion / Q&A Copyright 2009 SUN MySQL The World’s Most Popular Open Source Database 2 MySQL Architecture Copyright 2009 SUN MySQL The World’s Most Popular Open Source Database 3 MySQL : Storage Engines Partenaires InnoDB SolidBD for MySQL En cours : InfoBright – Brighthouse DWH ScaleDB NitroEDB AmazonS3 MemCache PrimeBase XT Moteur de stockage de Thinking Networks OpenOLAP Communautaire Copyright 2009 SUN MySQL The World’s Most Popular Open Source Database 4 MySQL easy install on linux • Rpm + repo • Yum install • Apt-get install • Or install from a tar.gz – allow any case of installation : multiple base dir / multiple instances per base dir • Automatic startup : – /etc/init.d/ + chkconfig (gestion des niveaux de rcx.d) Copyright 2009 SUN MySQL The World’s Most Popular Open Source Database 5 MySQL, Heartbeat & DRBD Cluster IP Management Active Passive DRBD DRBD Server Server Synchronous Block Replication Copyright 2009 SUN MySQL The World’s Most Popular Open Source Database 6 DRBD Service Service File System File System Buffer Cache Raw Device Raw Device Buffer Cache DRBD DRBD TCP/IP TCP/IP -
Sun SPARC Enterprise T5440 Servers
Sun SPARC Enterprise® T5440 Server Just the Facts SunWIN token 526118 December 16, 2009 Version 2.3 Distribution restricted to Sun Internal and Authorized Partners Only. Not for distribution otherwise, in whole or in part T5440 Server Just the Facts Dec. 16, 2009 Sun Internal and Authorized Partner Use Only Page 1 of 133 Copyrights ©2008, 2009 Sun Microsystems, Inc. All Rights Reserved. Sun, Sun Microsystems, the Sun logo, Sun Fire, Sun SPARC Enterprise, Solaris, Java, J2EE, Sun Java, SunSpectrum, iForce, VIS, SunVTS, Sun N1, CoolThreads, Sun StorEdge, Sun Enterprise, Netra, SunSpectrum Platinum, SunSpectrum Gold, SunSpectrum Silver, and SunSpectrum Bronze are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the United States and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. UNIX is a registered trademark in the United States and other countries, exclusively licensed through X/Open Company, Ltd. T5440 Server Just the Facts Dec. 16, 2009 Sun Internal and Authorized Partner Use Only Page 2 of 133 Revision History Version Date Comments 1.0 Oct. 13, 2008 - Initial version 1.1 Oct. 16, 2008 - Enhanced I/O Expansion Module section - Notes on release tabs of XSR-1242/XSR-1242E rack - Updated IBM 560 and HP DL580 G5 competitive information - Updates to external storage products 1.2 Nov. 18, 2008 - Number -
Sun Ultratm 25 Workstation & Sun Ultra 45 Workstation Just the Facts
Sun UltraTM 25 Workstation & Sun Ultra 45 Workstation Just the Facts SunWIN Token# 473547 SunWIN Token# 460409 Copyrights © 2006 Sun Microsystems, Inc. All Rights Reserved. Sun, Sun Microsystems, the Sun logo, Ultra, Sun Blade, Java, Solaris, Java, NetBeans, Sun Fire, Sun StorEdge, SunLink, SunSpectrum, SunSpectrum Platinum, SunSpectrum Gold, SunSpectrum Silver, SunSpectrum Bronze, SunSolve, SunPCi, and SunVTS are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the United States and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. UNIX is a registered trademark in the United States and other countries, exclusively licensed through X/Open Company, Ltd. Ultra 25/45 JTF - 12/10/07 Sun Confidential – Internal Use Only 2 Table of Contents Positioning.....................................................................................................................................................................4 Introduction...............................................................................................................................................................4 Product Family Placement .......................................................................................................................................5 Sun Ultra 45 vs Sun Ultra 25 Workstation...............................................................................................................5 -
Sun Blade 1000 and 2000 Workstations
Sun BladeTM 1000 and 2000 Workstations Just the Facts Copyrights 2002 Sun Microsystems, Inc. All Rights Reserved. Sun, Sun Microsystems, the Sun logo, Sun Blade, PGX, Solaris, Ultra, Sun Enterprise, Starfire, SunPCi, Forte, VIS, XGL, XIL, Java, Java 3D, SunVideo, SunVideo Plus, Sun StorEdge, SunMicrophone, SunVTS, Solstice, Solstice AdminTools, Solstice Enterprise Agents, ShowMe, ShowMe How, ShowMe TV, Sun Workstation, StarOffice, iPlanet, Solaris Resource Manager, Java 2D, OpenWindows, SunCD, Sun Quad FastEthernet, SunFDDI, SunATM, SunCamera, SunForum, PGX32, SunSpectrum, SunSpectrum Platinum, SunSpectrum Gold, SunSpectrum Silver, SunSpectrum Bronze, SunSolve, SunSolve EarlyNotifier, and SunClient are trademarks, registered trademarks, or service marks of Sun Microsystems, Inc. in the United States and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the United States and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. UNIX is a registered trademark in the United States and in other countries, exclusively licensed through X/Open Company, Ltd. FireWire is a registered trademark of Apple Computer, Inc., used under license. OpenGL is a trademark of Silicon Graphics, Inc., which may be registered in certain jurisdictions. Netscape is a trademark of Netscape Communications Corporation. PostScript and Display PostScript are trademarks of Adobe Systems, Inc., which may be registered in -
Rethink Storage with Sun Storage 7000 Systems
An Oracle White Paper January 2010 Rethink Storage with Sun Storage 7000 Unified Storage Systems Oracle White Paper⎯Rethink Storage with Sun Storage 7000 Unified Storage Systems Introduction ......................................................................................... 1 Open Storage...................................................................................... 3 What Is Open Storage?................................................................... 3 Advantages of Open Source Software............................................ 4 Sun Storage 7000 Unified Storage Systems....................................... 4 Breaking Through Economic and Performance Barriers................. 5 Taking Advantage of Solid-State Disks with Oracle Solaris ZFS .... 6 Hybrid Storage Pools ...................................................................... 6 Radically Simple Storage ................................................................ 7 Most Comprehensive Self-Healing Storage System ....................... 8 Business Value of Hybrid Storage Pools ........................................ 9 Sun Storage 7000 Unified Storage Systems Configurations......... 10 Conclusion ........................................................................................ 12 Oracle White Paper⎯Rethink Storage with Sun Storage 7000 Unified Storage Systems Introduction The rapid growth of new digital data demands new storage architectures that offer more flexibility and radically different storage economics. IDC, a technology research firm, estimates -
Sun Fire X4500 Server Linux and Solaris OS Installation Guide
Sun Fire X4500 Server Linux and Solaris OS Installation Guide Sun Microsystems, Inc. www.sun.com Part No. 819-4362-17 March 2009, Revision A Submit comments about this document at: http://www.sun.com/hwdocs/feedback Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved. This distribution may include materials developed by third parties. Sun, Sun Microsystems, the Sun logo, Java, Netra, Solaris, Sun Ray and Sun Fire X4500 Backup Server are trademarks or registered trademarks of Sun Microsystems, Inc., and its subsidiaries, in the U.S. and other countries. This product is covered and controlled by U.S. Export Control laws and may be subject to the export or import laws in other countries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether direct or indirect, are strictly prohibited. Export or reexport to countries subject to U.S. embargo or to entities identified on U.S. export exclusion lists, including, but not limited to, the denied persons and specially designated nationals lists is strictly prohibited. Use of any spare or replacement CPUs is limited to repair or one-for-one replacement of CPUs in products exported in compliance with U.S. export laws. Use of CPUs as product upgrades unless authorized by the U.S. Government is strictly prohibited. Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés. Cette distribution peut incluire des élements développés par des tiers. Sun, Sun Microsystems, le logo Sun, Java, Netra, Solaris, Sun Ray et Sun Fire X4500 Backup Server sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc., et ses filiales, aux Etats-Unis et dans d'autres pays. -
Repository Design Report with Attached Metadata Plan
National Digital Information Infrastructure and Preservation Program Preserving Digital Public Television Project REPOSITORY DESIGN REPORT WITH ATTACHED METADATA PLAN Prepared by Joseph Pawletko, Software Systems Architect, New York University With contributions by Nan Rubin, Project Director, Thirteen/WNET-TV Kara van Malssen, Senior Research Scholar, New York University 2010-03-19 Table of Contents Executive Summary ………………………………………………………………………………………… 2 1. Background ……………………………………………………………………………………………..... 3 2. OAIS Reference Model and Terms …………………………………………………………………… 3 3. Preservation Repository Design ……………………………………………………………………….. 3 4. Core Archival Information Package Structure …………………………………………………….. 11 5. PDPTV Archival Information Package Structure …………………………………………………... 14 6. PDPTV Submission Information Packages …………………………………………………………... 17 7. PDPTV Archival Information Package Generation and Metadata Plan ……………………... 18 8. Content Transfer to the Library of Congress .……………………………………………………….. 22 9. Summary …………………………………………………………………………………………………… 23 10. Remaining Work …………………………………………………………………………………………. 30 11. Acknowledgements ……………………………………………………………………………………. 31 Appendix 1 Glossary and OAIS Operation ……………………………………………...…………… 32 Appendix 2 WNET NAAT 002405 METS File ……………………………………………………………. 37 Appendix 3 WNET NAAT 002405 PBCore File …………………………………………………………. 41 Appendix 4 WNET NAAT 002405 SD Broadcast Master PREMIS File ...……………………………. 46 Appendix 5 WNET NAAT 002405 SD Broadcast Master METSRights File …………………………. 48 Appendix 6 WGBH -
Services for Oracle Sun Blade Servers
ORACLE DATA SHEET SERVICES FOR ORACLE SUN BLADE SERVERS Oracle's Sun Blade modular systems integrate x86 and SPARC-based servers, INSTALL, CONFIGURE, AND OPTIMIZE ORACLE SUN BLADE SERVERS storage, and advanced networking capabilities to support a complex and dynamic mix of IT workloads. And with Oracle Advanced Customer Support KEY FEATURES Services you get the guidance you need to plan, deploy and keep your Oracle • Technical knowledge transfer tailored to Sun Blade technology optimized for peak performance. your IT environment and business needs With intimate knowledge of Oracle tools and best practices, Oracle Advanced • Comprehensive, system hardware installation including site audit, installation Customer Support Services provides the right knowledge at the right time to help and configuration planning documentation, mitigate risk and to maximize the value of your Oracle technology investment. testing and handover • Trusted systems knowledge and proven IT delivery methodologies Pre-production Services for Oracle Sun Blade Servers • Reviews and recommendations for Oracle Advanced Customer Support Services delivers optimization services that help you optimized deployment planning, system install, configure, optimize and support your Oracle technology environment. Whether you are configuration and meeting availability requirements initiating a technology refresh project, optimizing an existing environment through the use of virtualization technologies, or looking to drive better datacenter standardization and operational best practices, -
Repository and Preservation Storage Architecture
Repository and Preservation Storage Architecture Keith Rajecki Sun Microsystems, Inc. 15 Network Circle Menlo Park, CA 94025 USA [email protected] Abstract enables digital workflow and the content archive provides While the Open Archive Information System (OAIS) model permanent access to digital content files. has become the de facto standard for preservation archives, the With SAM software, the files are stored, tracked, and retrieved design and implementation of a repository or reliable long term based on the archival requirements. Files are seamlessly and archive lacks adopted technology standards and design best transparently available to other services. SAM software creates practices. This paper is intended to provide guidelines and virtually limitless capacity. Its scalability allows for continual recommendations for standards implementation and best growth throughout the archive with support for all data types. practices for a viable, cost effective, and reliable repository and The policy based SAM software stores and manages data for preservation storage architecture. This architecture is based on compliance and non-compliance archives using a tiered storage a combination of open source and commercially supported approach with integrated disk and tape into a seamless storage software and systems. solution, SAM software simplifies the archive storage. Allows you to automate data management policies based on file Although several operating systems currently exist, the logical attributes. You can manage data according to the storage and choice for an archive storage system is an open source access requirements of each user on the system and decide how operating system, of which there are two primary choices data is grouped, copied, and accessed based on the needs of the today: Linux and Solaris. -
Sun Blade™ 150 Workstation
Datasheet Sun Blade 150 Workstation On the Web http://www.sun.com/sunblade150 Sun Blade™ 150 Workstation The price leader in 64-bit UNIX® workstations. Key feature highlights Entry-level workstation designed with full features to deliver exceptional performance. • 550-MHz or 650-MHz UltraSPARC® IIi For technical and business computing environments looking to balance advanced 64-bit capabili- processor with 512 KB L2 on-die cache ties, workstation performance, support for the Solaris Operating System, and cost considerations, • Up to 2 GB of SDRAM memory with Error Sun has the ideal solution: The Sun Blade™ 150 workstation. This powerful, low-cost system boasts Correction a full 64-bit architecture with the latest high-performance UltraSPARC IIi CPU, supporting up to 2 • Up to two ATA-66 80-GB, 7200-rpm hard GB of memory and up to 160 GB of internal storage. Features such as a fast PCI bus and the latest disk drives I/O technologies including USB and IEEE-1394 provide connectivity to a wide variety of latest- • 1394a (FireWire) generation peripherals. And with graphics options that include the high-resolution, on-board Sun™ • Three PCI slots PGX64 2-D, Sun™ XVR-100 2-D, or high-end Sun™ XVR-500 3-D graphics, the Sun Blade 150 worksta- • 10/100 BaseT Ethernet tion delivers advanced workstation capabilities with exceptional performance at an affordable • Solaris™ 8 HW 5/03 and Solaris 9 HW 4/03 price. With only a 20 percent increase in megahertz, the doubling of the cache to 512 KB, plus Sun Operating System pre-installed XVR-500 graphics, the Sun Blade 150 workstation cranks up the performance.