Sun Grid Engine Update Daniel Gruber Software Engineer Sun Microsystems Deutschland GmbH Sun is a wholly-owned subsidiary of Oracle 1 Content What's new in SGE? DRMAA Customer Feedback 2 Sun Grid Engine Releases Release Announcement Some Features... 6.2 major 23.09.2008 SDM, scalability (> 60000 cores), AR, IJS 6.2 update 1 18.12.2008 maintenance release GUI Installer, JSV, Per Job Resources, 6.2 update 2 31.03.2009 jemalloc SGE Inspect, SDM Cloud Adapter, 6.2 update 3 23.06.2009 Exclusive Host 6.2 update 4 23.10.2009 maintenance release Slotwise Preemption, Core Binding, 6.2 update 5 22.12.2009 enhanced Inspect, Java JSV, Array Job Throttling, Hadoop Support Sun Confidential: Internal Only 3 SDM – Service Domain Manager Grid Grid Grid Engine Engine Engine A B C Service Domain Manager Zzzzz Zzzzz Power Saving Spare Pool (via IPMI) Spare Pool CloudService Sun Confidential: Internal Only 4 JSV – Job Submission Verifier • Administrator (or users) can reformulate (insert, delete) job submission parameters based on a JSV scripts • Jobs can be rejected based on parameters • bash, csh, tcl, perl and JSV scripts are supported Sun Confidential: Internal Only 5 GUI Installer • Installs a complete SGE cluster Sun Confidential: Internal Only 6 Slot-wise preemption • Slot limit per host • Suspends jobs from subordinate queues in order to get high priority jobs to run • Suspends longest/shortest running jobs • Multiple layers (suspend trees) possible • Per layer: Order definable Sun Confidential: Internal Only 7 Core Binding • Job submission extension • Binds job on specific cores and sockets, which prevents from migration through OS sheduler • Significant performance increase possible • Stable runtime results in case of oversubscription • Job isolation • Available for DRMAA applications (native_specification) Sun Confidential: Internal Only 8 Core Binding • qsub -binding striding:2:2 job.sh • qsub -binding linear:4 job.sh Host Sockets Cores Sun Confidential: Internal Only 9 DRMAA Customer Feedback • Number of customers and ISVs using DRMAA • Most successful: Java language binding • Join DRMAAv2.0 sessions • Example for applications in complex workflows: KNIME – Konstanz Information Miner www.knime.org Sun Confidential: Internal Only 10 The KNIME Platform Sun Confidential: Internal Only 11 KNIME Cluster Execution Task Splitting and DRMAA Submission Sun Confidential: Internal Only 12 Thank you! Sun Grid Engine Update Daniel Gruber [email protected] 13.