Sun Grid Engine Update Daniel Gruber Software Engineer Sun Microsystems Deutschland GmbH Sun is a wholly-owned subsidiary of Oracle
1 Content
What's new in SGE?
DRMAA Customer Feedback
2 Sun Grid Engine Releases
Release Announcement Some Features... 6.2 major 23.09.2008 SDM, scalability (> 60000 cores), AR, IJS 6.2 update 1 18.12.2008 maintenance release GUI Installer, JSV, Per Job Resources, 6.2 update 2 31.03.2009 jemalloc SGE Inspect, SDM Cloud Adapter, 6.2 update 3 23.06.2009 Exclusive Host 6.2 update 4 23.10.2009 maintenance release Slotwise Preemption, Core Binding, 6.2 update 5 22.12.2009 enhanced Inspect, Java JSV, Array Job Throttling, Hadoop Support
Sun Confidential: Internal Only 3 SDM – Service Domain Manager
Grid Grid Grid Engine Engine Engine A B C
Service Domain Manager
Zzzzz Zzzzz
Power Saving Spare Pool (via IPMI) Spare Pool CloudService
Sun Confidential: Internal Only 4 JSV – Job Submission Verifier • Administrator (or users) can reformulate (insert, delete) job submission parameters based on a JSV scripts • Jobs can be rejected based on parameters • bash, csh, tcl, perl and JSV scripts are supported
Sun Confidential: Internal Only 5 GUI Installer • Installs a complete SGE cluster
Sun Confidential: Internal Only 6 Slot-wise preemption • Slot limit per host • Suspends jobs from subordinate queues in order to get high priority jobs to run • Suspends longest/shortest running jobs • Multiple layers (suspend trees) possible • Per layer: Order definable
Sun Confidential: Internal Only 7 Core Binding • Job submission extension • Binds job on specific cores and sockets, which prevents from migration through OS sheduler • Significant performance increase possible • Stable runtime results in case of oversubscription • Job isolation • Available for DRMAA applications (native_specification)
Sun Confidential: Internal Only 8 Core Binding • qsub -binding striding:2:2 job.sh • qsub -binding linear:4 job.sh
Host Sockets
Cores
Sun Confidential: Internal Only 9 DRMAA Customer Feedback • Number of customers and ISVs using DRMAA • Most successful: Java language binding • Join DRMAAv2.0 sessions
• Example for applications in complex workflows:
KNIME – Konstanz Information Miner www.knime.org
Sun Confidential: Internal Only 10 The KNIME Platform
Sun Confidential: Internal Only 11 KNIME Cluster Execution Task Splitting and DRMAA Submission
Sun Confidential: Internal Only 12 Thank you!
Sun Grid Engine Update Daniel Gruber [email protected]
13