Sun Grid Engine Update Daniel Gruber Software Engineer Deutschland GmbH Sun is a wholly-owned subsidiary of Oracle

1 Content

What's new in SGE?

DRMAA Customer Feedback

2 Sun Grid Engine Releases

Release Announcement Some Features... 6.2 major 23.09.2008 SDM, scalability (> 60000 cores), AR, IJS 6.2 update 1 18.12.2008 maintenance release GUI Installer, JSV, Per Job Resources, 6.2 update 2 31.03.2009 jemalloc SGE Inspect, SDM Cloud Adapter, 6.2 update 3 23.06.2009 Exclusive Host 6.2 update 4 23.10.2009 maintenance release Slotwise Preemption, Core Binding, 6.2 update 5 22.12.2009 enhanced Inspect, Java JSV, Array Job Throttling, Hadoop Support

Sun Confidential: Internal Only 3 SDM – Service Domain Manager

Grid Grid Grid Engine Engine Engine A B

Service Domain Manager

Zzzzz Zzzzz

Power Saving Spare Pool (via IPMI) Spare Pool CloudService

Sun Confidential: Internal Only 4 JSV – Job Submission Verifier • Administrator (or users) can reformulate (insert, delete) job submission parameters based on a JSV scripts • Jobs can be rejected based on parameters • bash, csh, tcl, and JSV scripts are supported

Sun Confidential: Internal Only 5 GUI Installer • Installs a complete SGE cluster

Sun Confidential: Internal Only 6 Slot-wise preemption • Slot limit per host • Suspends jobs from subordinate queues in order to get high priority jobs to run • Suspends longest/shortest running jobs • Multiple layers (suspend trees) possible • Per layer: Order definable

Sun Confidential: Internal Only 7 Core Binding • Job submission extension • Binds job on specific cores and sockets, which prevents from migration through OS sheduler • Significant performance increase possible • Stable runtime results in case of oversubscription • Job isolation • Available for DRMAA applications (native_specification)

Sun Confidential: Internal Only 8 Core Binding • qsub -binding striding:2:2 job.sh • qsub -binding linear:4 job.sh

Host Sockets

Cores

Sun Confidential: Internal Only 9 DRMAA Customer Feedback • Number of customers and ISVs using DRMAA • Most successful: Java language binding • Join DRMAAv2.0 sessions

• Example for applications in complex workflows:

KNIME – Konstanz Information Miner www.knime.org

Sun Confidential: Internal Only 10 The KNIME Platform

Sun Confidential: Internal Only 11 KNIME Cluster Execution Task Splitting and DRMAA Submission

Sun Confidential: Internal Only 12 Thank you!

Sun Grid Engine Update Daniel Gruber [email protected]

13