IBM Platform Computing Solutions

Total Page：16

File Type：pdf, Size：1020Kb

Front cover IBM Platform Computing Solutions Describes the offering portfolio Shows implementation scenarios Delivers add-on value to current implementations Dino Quintero Scott Denham Rodrigo Garcia da Silva Alberto Ortiz Aline Guedes Pinto Atsumori Sasaki Roger Tucker Joanna Wong Elsie Ramos ibm.com/redbooks International Technical Support Organization IBM Platform Computing Solutions December 2012 SG24-8073-00 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. First Edition (December 2012) This edition applies to IBM Platform High Performance Computing (HPC) v3.2, IBM Platform Cluster Manager Advanced Edition v3.2, IBM Platform Symphony v5.2, IBM Platform Load Sharing Facility (LSF) v8.3, RedHat Enterprise Linux v6.2/v6.3, Hadoop v1.0.1, Sun Java Development Kit (JDK) v1.6.0_25, and Oracle Database Express Edition v11.2.0. © Copyright International Business Machines Corporation 2012. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . vii Trademarks . viii Preface . ix The team who wrote this book . ix Now you can become a published author, too! . xi Comments welcome. xi Stay connected to IBM Redbooks . xii Chapter 1. Introduction to IBM Platform Computing . 1 1.1 IBM Platform Computing solutions value proposition . 2 1.1.1 Benefits . 3 1.1.2 Cluster, grids, and clouds . 4 1.2 History . 5 1.2.1 IBM acquisition . 6 Chapter 2. Technical computing software portfolio. 7 2.1 IBM Platform Computing product portfolio . 8 2.1.1 Workload management. 8 2.1.2 Cluster management. 8 2.2 IBM Platform Computing products overview . 10 2.2.1 IBM Platform Load Sharing Facility family . 10 2.2.2 IBM Platform Message Passing Interface. 12 2.2.3 IBM Platform Symphony family. 12 2.2.4 IBM Platform HPC. 13 2.2.5 IBM Platform Cluster Manager . 13 2.3 Current user roadmap . 14 Chapter 3. Planning . 15 3.1 Hardware setup for this residency. 16 3.1.1 Server nodes. 16 3.1.2 Shared storage . 18 3.1.3 Infrastructure planning . 20 3.2 Software packages . 23 Chapter 4. IBM Platform Load Sharing Facility (LSF) product family . 27 4.1 IBM Platform LSF overview. 28 4.2 IBM Platform LSF add-on products. 36 4.2.1 IBM Platform Application Center. 36 4.2.2 IBM Platform RTM. 40 4.2.3 IBM Platform Process Manager . 41 4.2.4 IBM Platform License Scheduler. 44 4.3 Implementation . 49 4.3.1 IBM Platform LSF implementation . 50 4.3.2 IBM Platform Application Center implementation . 54 4.3.3 IBM Platform RTM implementation . 65 4.3.4 IBM Platform Process Manager implementation. 98 4.4 References . 109 © Copyright IBM Corp. 2012. All rights reserved. iii Chapter 5. IBM Platform Symphony . 111 5.1 Overview . 112 5.1.1 Architecture. 117 5.1.2 Target audience . 118 5.1.3 Product versions . 118 5.2 Compute-intensive and data-intensive workloads. 119 5.2.1 Basic concepts . 120 5.2.2 Core components . 121 5.2.3 Application implementation . 124 5.2.4 Application Deployment . 125 5.2.5 Symexec . 126 5.3 Data-intensive workloads . 126 5.3.1 Data affinity . 127 5.3.2 MapReduce. 128 5.4 Reporting. 134 5.5 Getting started. 135 5.5.1 Planning for Symphony. 136 5.5.2 Installation preferred practices . 139 5.6 Sample workload scenarios . 156 5.6.1 Hadoop . 156 5.7 Symphony and IBM Platform LSF multihead environment . 164 5.7.1 Planning a multihead installation . 167 5.7.2 Installation. 167 5.7.3 Configuration. 171 5.8 References . 179 Chapter 6. IBM Platform High Performance Computing . 181 6.1 Overview . ..

370

Copy Link

Recommended publications

Ioncube Ltd. Presentation to Kent University 2015 Hello!
ionCube Ltd. Presentation to Kent University 2015 Hello! I am Nick Lindridge MD ionCube Ltd. - Software company ioncube.com ioncube24.com Overview About ionCube What ionCube offers Who are we looking for Technologies we use Questions - anything not covered? Demo 1. About ionCube About ionCube - where are we? Located close to Canterbury On bus routes About ionCube - we had a unit built About ionCube - offices About ionCube - what we do Develop #infosec solutions. ionCube PHP Encoder - protect/license PHP code online PHP Encoder ionCube24 - real-time protection against website vulnerability exploits Top countries USA, Germany, UK, China, Russia, Netherlands, Indonesia, Turkey, India, Italy, France, 100+. Products - PHP Encoder PHP remains highly popular - used by 81.6% of websites - w3techs.com, 8 dec 2015 Many commercial developers want to protect and license code Website owners (should) want to protect database passwords Products - PHP Encoder Compiles PHP to modified VM bytecode ionCube Loader PHP extension with modified execution engine Encrypt code sections with runtime generated encryption keys to solve static key issue // @ioncube.dk g(14) -> "octoberon" function fn($p) Licensing solutions Products - PHP Encoder Encoder/Loader codebase - C GUI - C++ / wxWidgets Windows, Linux, FreeBSD, OS X Updated to support new versions of PHP and run code from previous Encoders, plus new features “ Five a day Website vulnerabilities Customer reports 5 website vuln exploits per day for their small hosting business with 1000
[Show full text]
Introduction Use Runit with Traditional Init (Sysvinit)
2021/07/26 19:10 (UTC) 1/12 Runit Runit Introduction runit is a UNIX init scheme with service supervision. It is a cross-platform Unix init scheme with service supervision, a replacement for sysvinit, and other init schemes and supervision that are used with the traditional init. runit is compatible with djb's daemontools. In Unix-based computer operating systems, init (short for initialization) is the first process started during booting of the computer system. Init is a daemon process that continues running until the system is shut down. Slackware comes with its own legacy init (/sbin/init) from the sysvinit package, that used to be included in almost all other major Linux distributions. The init daemon (or its replacement) is characterised by Process ID 1 (PID 1). To read on the benefits of runit, see here: http://smarden.org/runit/benefits.html * Unless otherwise stated, all commands in this article are to be run by root. Use runit with traditional init (sysvinit) runit is not provided by Slackware, but a SlackBuild is maintained on https://slackbuilds.org/. It does not have any dependencies. As we do not want yet to replace init with runit, run the slackbuild with CONFIG=no: CONFIG=no ./runit.SlackBuild Then install the resulting package and proceed as follows: mkdir /etc/runit/ /service/ cp -a /usr/doc/runit-*/etc/2 /etc/runit/ /sbin/runsvdir-start & Starting via rc.local For a typical Slackware-stlyle service, you can edit /etc/rc.d/rc.local file if [ -x /sbin/runsvdir-start ]; then /sbin/runsvdir-start & fi and then edit write /etc/rc.d/rc.local_shutdown #!/bin/sh SlackDocs - https://docs.slackware.com/ Last update: 2020/05/06 08:08 (UTC) howtos:slackware_admin:runit https://docs.slackware.com/howtos:slackware_admin:runit RUNIT=x$( /sbin/pidof runsvdir ) if [ "$RUNIT" != x ]; then kill $RUNIT fi Then give rc.local_shutdown executive permission: chmod +x /etc/rc.d/rc.local_shutdown and reboot Starting via inittab (supervised) Remove the entries in /etc/rc.d/rc.local and /etc/rc.d/rc.local_shutdown described above.
[Show full text]
WEB GUI Configuration
WEB GUI MANUAL Contents INTRODUCTION...................................................................................................................................3 Key Features .....................................................................................................................................3 GLOSSARY.............................................................................................................................................4 Packet loss......................................................................................................................................... 4 VoIPmonitor loss.......................................................................................................................... 4 Packet delay variation PDV............................................................................................................. 4 VoIPmonitor Packet delay variation..............................................................................................5 Jitter buffer....................................................................................................................................... 5 MOS score......................................................................................................................................... 5 VoIPmonitor MOS prediction....................................................................................................... 7 INSTALLATION...................................................................................................................................
[Show full text]
Steve French IBM – Linux Technology Center
IBM Server & Technology Group Linux Kernel Clients: A Year in Review “Man-Eating Seals of Antiquity” to “Sheep on Meth” Steve French IBM – Linux Technology Center Linux Technology Center © 2010 IBM Corporation SNIA SDC 2010 Legal Statement –This work represents the views of the author(s) and does not necessarily reflect the views of IBM Corporation –A full list of U.S. trademarks owned by IBM may be found at http://www.ibm.com/legal/copytrade.shtml. –Linux is a registered trademark of Linus Torvalds. –Other company, product, and service names may be trademarks or service marks of others. 2 IBM Linux Technology Center © 2010 IBM Corporation SNIA SDC 2010 Who am I? –Steve French ([email protected] or [email protected]) –Author and maintainer of Linux cifs vfs (for accessing Samba, Windows and various SMB/CIFS based NAS appliances) –Wrote initial SMB2 kernel client prototype –Member of the Samba team, coauthor of CIFS Technical Reference and former SNIA CIFS Working Group chair –Architect: Filesystems/NFS/Samba for IBM LTC IBM Linux Technology Center © 2010 IBM Corporation SNIA SDC 2010 • Last September Kernel version 2.6.31: • [Since 2.6.34] Now we have • “Man-Eating Seals of Antiquity” 2.6.36-rc4 – 2.6.31 came out on September 9th, 2009 • “Sheep on Meth” 4 IBM Linux Technology Center © 2010 IBM Corporation SNIA SDC 2010 Development on Linux kernel clients is very active • 187 kernel changesets for cifs since 2.6.31 (typical year) • But … improvements to related tools have accelerated dramatically: in a few month period early this year 100+ changesets in mount helper and related utilities were merged • And smb2 development accelerated … – A month after SambaXP last year, public git tree for smb2 development created on kernel.org – More than 130 kernel changesets for smb2 since January – SMB2 module now close to full function but 30% smaller (17188 lines of code, CIFS is now 24447 LOC).
[Show full text]
Technical Support Services for Linux from IBM Robust Linux Support Solutions from an Industry Leader
IBM Technical Support Services Technical Support Services for Linux from IBM Robust Linux support solutions from an industry leader IBM Technical Support Services (TSS) — Highlights Linux Support Summary • IBM’s Operational Support Services — Support Line for Linux is IBM Technical Support Services offers comprehensive, flexible support for Linux a remotely delivered support service designed to answer how-to and environments across all IBM hardware diagnostic questions. Services include: platforms as well as OEM Intel. – Assistance for Linux installation, configuration, usage, Business Impact technical questions, and code-related defects • Helps maximize the return on your Linux – Standard problem determination/resolution investment through increased reliability – Defect support and change team for emergency fixes • Designed to increase IT staff productivity with installation, configuration and – Call-back response from technicians in specialist area within how-to support 2 hours prime shift and 4 hours off shift • Leverage IBM skills and infrastructure to help – Per environment support based on total server count lower your overall Linux operational costs – Support for all IBM hardware platforms and OEM Intel IBM Value – Red Hat or SUSE Linux, IBM Director, VMware and US • Unmatched breadth, depth, reach Citizens support available • Unparalleled speed to resolution – Linux subscription options available • Access to relevant OEM vendors and IBM Linux Technology Center • Enhanced Services (i.e. Account Advocate, Enhanced Technical Support,
[Show full text]
Troubleshooting Guide
Java Platform, Standard Edition Troubleshooting Guide Release 9 E61074-05 October 2017 Java Platform, Standard Edition Troubleshooting Guide, Release 9 E61074-05 Copyright © 1995, 2017, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency- specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S.
[Show full text]
Platform LSF Foundations Chapter 1
Platform LSF Version 9 Release 1.3 Foundations SC27-5304-03 Platform LSF Version 9 Release 1.3 Foundations SC27-5304-03 Note Before using this information and the product it supports, read the information in “Notices” on page 21. First edition This edition applies to version 9, release 1 of IBM Platform LSF (product number 5725G82) and to all subsequent releases and modifications until otherwise indicated in new editions. Significant changes or additions to the text and illustrations are indicated by a vertical line (|) to the left of the change. If you find an error in any Platform Computing documentation, or you have a suggestion for improving it, please let us know. In the IBM Knowledge Center, add your comments and feedback to any topic. You can also send your suggestions, comments and questions to the following email address: [email protected] Be sure include the publication title and order number, and, if applicable, the specific location of the information about which you have comments (for example, a page number or a browser URL). When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you. © Copyright IBM Corporation 1992, 2014. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Chapter 1. IBM Platform LSF: An Job submission .............12 Overview ..............1 Job scheduling and dispatch .........13 Introduction to IBM Platform LSF .......1 Host selection .............15 LSF cluster components...........2 Job execution environment .........15 Chapter 2.
[Show full text]
Cubismo: Decloaking Server-Side Malware Via Cubist Program Analysis
Cubismo: Decloaking Server-side Malware via Cubist Program Analysis Abbas Naderi-Afooshteh, Yonghwi Kwon, Anh Nguyen-Tuong, Mandana Bagheri-Marzijarani, and Jack W. Davidson Department of Computer Science, University of Virginia {abiusx;yongkwon;nguyen;mb3wz;jwd}@virginia:edu ABSTRACT 1 INTRODUCTION Malware written in dynamic languages such as PHP routinely Web-based malware, particularly server-side malware, is one employ anti-analysis techniques such as obfuscation schemes and of the most prevalent security threats nowadays. Numerous re- evasive tricks to avoid detection. On top of that, attackers use ports describe the prevalence of server-side malware. Sucuri, a firm automated malware creation tools to create numerous variants specializing in managed security and system protection, analyzed with little to no manual effort. 34,371 infected websites and reported that 71% of those contained This paper presents a system called Cubismo to solve this press- PHP-based, hidden backdoors [52]. Incapsula discovered that out ing problem. It processes potentially malicious files and decloaks of 500 infected websites detected on their network, the majority their obfuscations, exposing the hidden malicious code into multiple of them contained PHP malware [27]. Verizon’s 2017 Data Breach files. The resulting files can be scanned by existing malware detec- reported that a sizable number of web server compromises are a tion tools, leading to a much higher chance of detection. Cubismo means to an end, allowing attackers to set up for other targets [26]. achieves improved detection by exploring all executable statements This prevalence is in part because server-side malware is typ- of a suspect program counterfactually to see through complicated ically equipped with advanced anti-analysis and anti-debugging polymorphism, metamorphism and, obfuscation techniques and techniques such as obfuscation and metamorphism.
[Show full text]
IBM Platform Computing Cloud Service Ready to Use Platform LSF & Symphony Clusters in the Softlayer Cloud
Platform Computing IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud February 25, 2014 1 © 2014 IBM Corporation Platform Computing Agenda v Mapping clients needs to cloud technologies v Addressing your pain points v Introducing IBM Platform Computing Cloud Service v Product features and benefits v Use cases v Performance benchmarks 2 © 2014 IBM Corporation Platform Computing HPC cloud characteristics and economics are different than general-purpose computing • High-end hardware and special purpose devices (e.g. GPUs) are typically used to supply the needed processing, memory, network, and storage capabilities • The performance requirements of technical computing and service-oriented workloads means that performance may be impacted in a virtualized cloud environment, especially when latency or I/O is a constraint • HPC cluster/grid utilization is usually in the 70-90% range, removing a major potential advantage of a public cloud service provider for stable workload volumes HPC Workloads Recommended for Private Cloud HPC Workloads with Best Potential for Virtualized Public & Hybrid Cloud Primary HPC Workloads 3 © 2014 IBM Corporation Platform Computing IBM’s HPC cloud strategy provides a flexible approach to address a variety of client needs Private Hybrid Public Clouds Clouds Clouds Evolve existing infrastructure to Enable integrated HPC Cloud to enhance approach to improve Access additional responsiveness, HPC cost and HPC capacity with flexibility, and capability variable cost model
[Show full text]
Blueprints: Installing Linux Distributions on Multipath Devices
Blueprints Installing Linux Distributions on Multipath Devices Blueprints Installing Linux Distributions on Multipath Devices Note Before using this information and the product it supports, read the information in Appendix D, “Notices,” on page 27. First Edition (August 2008) © Copyright International Business Machines Corporation 2008. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Preface . .v Appendix B. Additional multipath Introduction . .v configuration information for IBM storage . .23 Installing Linux Distributions on Multipathed Devices . .1 Appendix C. Related information . .25 What is Multipath Connectivity . .1 Hardware Setup for Test Environment . .2 Appendix D. Notices . .27 Multipath installation on Red Hat Enterprise Linux Trademarks . .28 5.2 . .2 Terms and conditions . .29 Multipath installation on SUSE Linux Enterprise Server 10 SP2 . .11 Appendix A. Troubleshooting tips . .21 © Copyright IBM Corp. 2008 iii iv Blueprints: Installing Linux Distributions on Multipath Devices Preface Introduction This blueprint provides step by step instructions for installing Red Hat Enterprise Linux 5.2 and SUSE Linux Enterprise Server 10 SP2 on a LUN in a multipath disk storage environment. The procedure demonstrated here is performed on a System x host connected to a DS6000 storage server through Fibre Channel Fabric, but it can be adapted for installing either of the Linux distribution onto other supported models of storage devices. Intended audience This document is intended to be used by Linux system administrators who have experience in installing Red Hat Enterprise Linux 5 or SUSE Linux Enterprise Server 10 and have a moderate level of knowledge in Device Mapper (DM) multipath.
[Show full text]
Container Technologies
Zagreb, NKOSL, FER Container technologies Marko Golec · Juraj Vijtiuk · Jakov Petrina April 11, 2020 About us ◦ Embedded Linux development and integration ◦ Delivering solutions based on Linux, OpenWrt and Yocto • Focused on software in network edge and CPEs ◦ Continuous participation in Open Source projects ◦ www.sartura.hr Introduction to GNU/Linux ◦ Linux = operating system kernel ◦ GNU/Linux distribution = kernel + userspace (Ubuntu, Arch Linux, Gentoo, Debian, OpenWrt, Mint, …) ◦ Userspace = set of libraries + system software Linux kernel ◦ Operating systems have two spaces of operation: • Kernel space – protected memory space and full access to the device’s hardware • Userspace – space in which all other application run • Has limited access to hardware resources • Accesses hardware resources via kernel • Userspace applications invoke kernel services with system calls User applications E.g. bash, LibreOffice, GIMP, Blender, Mozilla Firefox, etc. System daemons: Windowing system: User mode Low-level system systemd, runit, logind, X11, Wayland, Other libraries: GTK+, Qt, EFL, SDL, SFML, Graphics: Mesa, AMD components networkd, PulseAudio, SurfaceFlinger FLTK, GNUstep, etc. Catalyst, … … (Android) C standard library Up to 2000 subroutines depending on C library (glibc, musl, uClibc, bionic) ( open() , exec() , sbrk() , socket() , fopen() , calloc() , …) About 380 system calls ( stat , splice , dup , read , open , ioctl , write , mmap , close , exit , etc.) Process scheduling Memory management IPC subsystem Virtual files subsystem Network subsystem Kernel mode Linux Kernel subsystem subsystem Other components: ALSA, DRI, evdev, LVM, device mapper, Linux Network Scheduler, Netfilter Linux Security Modules: SELinux, TOMOYO, AppArmor, Smack Hardware (CPU, main memory, data storage devices, etc.) TABLE 1 Layers within Linux Virtualization Virtualization Concepts Two virtualization concepts: ◦ Hardware virtualization (full/para virtualization) • Emulation of complete hardware (virtual machines - VMs) • VirtualBox, QEMU, etc.
[Show full text]
Open OVF Proposal
Linux Technology Center Open OVF Proposal . Open-source library and tools to support the OVF – http://xml.coverpages.org/ni2007-09-11-a.html . IBM will be donating code and invites all to form an open community around OVF . Eclipse Public License – http://www.eclipse.org/legal/eplfaq.php . Contact: – Mike D. Day <[email protected]> 1 IBM Open Virtualization © 2008 IBM Corporation Linux Technology Center Open Virtual Appliance Format (OVF) ● DMTF Standard, Open Packaging format for Virtual Appliances ● Meta-data, configuration, file systems, disk images, memory images ● XML-encoded ● Security and licensing features ● Enables cross-hypervisor Appliance portability ● Enables a neutral Appliance Ecosystem OVF Container App App App Middleware Middleware OS Hypervisor BIOS/EFI © 2008 IBM Corporation Linux Technology Center Why open-ovf? . Leverage development resources by cooperating on common plumbing – Finish the plumbing faster, keep more resources focused on the higher-value solution stack . Build out the cross-hypervisor components of OVF – Prevent fragmentation of the OVF across the industry – Provide a reference architecture . Establish conventions in OVF usage that promote transportability of virtual machines . Promote the usage of OVF and its adoption as a standard VM Appliance format 3 IBM Open Virtualization © 2008 IBM Corporation Linux Technology Center OVF Characteristics . Tar file with following contents: – .ovf descriptor • xml-encoded, basic schema, appliance meta-data • Contains URI references to other files – disk images, iso, cpio archives, etc. • Contains virtual hardware resource descriptions – Memory, network interfaces, storage devices, etc. – .mf manifest file • Digest of sha-1 hashes of referenced files – .cert certificate • Signature of the manifest © 2008 IBM Corporation Linux Technology Center Missing from OVF: Linkage to an actual Hypervisor "The OVF format supports portable, efficient, and secure distribution of virtual machine templates and virtual appliances.
[Show full text]