Mach in the Darwin Kernel

Mac OS X Mach in the Darwin Kernel

http://www.maths.mq.edu.au/~steffen/talks/

COMP342 4/5/06

Daniel A. Steffen Mathematics Department Macquarie University [email protected] Mach in the Darwin Kernel

• Overview of Mac OS X & Darwin Kernel • Darwin Kernel Services • Mach History & Goals • Mach Abstractions in detail Mac OMacS X OS X System Architecture Darwin technology is at the core

Aqua • High Level APIs Classic Java Carbon Cocoa “Frameworks” Application Services Quartz OpenGL QuickTime • • Core Services Darwin • Darwin Kernel “CoreOS” Darwin Kernel Architecture Darwin Kernel • “hybrid” kernel: Aqua single address Application Classic Java Carbon Cocoa Environments space for BSD Quartz OpenGL QuickTime and Mach (not a true microkernel Darwin File System Network KEXT Drivers system) Darwin BSD Kernel IOKit • processor architecture Mach Kernel independent • open-source Darwin Kernel Architecture

• Origins - Mach 3.0 (OSFMK73) - Virtual Memory, Scheduling, IPC - 4.4BSD-Lite 2 - Process model, security, ﬁle access - FreeBSD 3.2 & 4.4 - Networking, NFS Darwin Kernel Services Darwin Kernel Services Emphasis on the services, not the layering • cross-layer view • everything built on File System Net KEXT top of Mach Drivers primitives Process KEXT Services e.g. BSD process Darwin Threads Files IPC IOKit - Kernel is Mach task + Scheduler VM Security extra structure • lower-layer API not always safe to use on higher-layer objects Mach Kernel

• microkernel architecturally, but tightly integrated • CPU and memory abstractions • manages processor and memory resources • process & thread scheduling • SMP & realtime support • memory protection, management (VM) • message based: all interaction based on mach ports • policy neutral BSD Kernel

• tight integration with Mach and IOKit • process model: env vars, signals, reaping • security policy: users, filesystem permissions • filesystem & networking architecture • POSIX/BSD API: pthreads, select, kqueues, … • plugin architecture for NKEs (e.g. filters) • enhanced VFS design (metadata, ACLs, unicode, large files, UBC) IO Kit

• abstraction level for driver and device access • object-oriented framework: common behavior & protocols among device types abstracted into families, only device-speciﬁcs in drivers • implemented in Embedded C++ • modular and extensible, many families already implemented in OS, support for userspace drivers • dynamic device management, on-demand driver matching & (un)loading, power management Mach/Darwin History

• 1985-1994: Mach Research project at CMU • 1990-1996: Mach 3 effort continued at OSF RI: OSF/1 ! DEC Digital UNIX ! Compaq Tru64 • 1989-1995: NeXTSTEP on Mach 2.5 & 4.3BSD • 1996-1998: MkLinux on Mach 3.0 by Apple & OSF • 1997: Apple acquires NeXT • 2000: Darwin 1.0 (Mac OS X DP4): kernel transitioned to Mach 3.0 & 4.4BSD-Lite 2 Mach Goals • simple, extensible communication kernel; BSD compatible (“drop-in” replacement for BSD kernel) • move as much functionality outside of kernel as possible (microkernel) • small number of abstractions • exploit parallelism in both OS and user applications • extensive process to process communication (IPC) • large, sparse address space support with flexible VM • portability, multiprocessor & heterogeneous system support, distributed operation Mach Abstractions • Task: unit of resource ownership, has virtual address space & collection of port rights, contains threads • Thread: unit of execution (CPU utilization), lightweight, shares containing task’s resources • Port: communication channel endpoint, accessible only via send / receive capabilities (port rights) • Port set: group of ports on which a thread can block waiting to receive messages • Message: typed collection of data objects/pointers • Memory object: internal unit of memory management, tasks map portions into their address space Mach Task ServicesMach Task A task is the unit of reso•urcollectionce owner ofsh systemip in M raesourch ces, •Prowhichvide a (exceptn enviro fnorm adendrt ess space) Mach Task are referenced by ports and may be T T T in which threads run: h h h shared with other tasks if they have r r r e e e • Acorddrreespondingss space port rights a a a d d d Communications •• provides a large, potentially sparse • Eadxcdrepesstion space handli,n porg tions of which Virtual Memory task_tmay be shared through inheritance Space tasork_c externalreate(), tas memork_terminayt emanagement() thread_create() Port Name tascontainsk_suspen dsome(), task _nrumberesume() of threads Space •task_swap_exception_ports() • defines task wide exception handling Mach ThreadMach Servic Thresead A thread is the unit of ex•ecpointution of in flo Mwa ccontrh ol in a task, defines “how” and “where” of Mach Thread •A Mexachecution, thread obutwn nots no thereso ruesourrces ces to •Defexineecutes the “withhow” and “where” of Register State execution, but not the resources to e•xecouwnste w noith resources, has access to all elements of the containing task •Defines thread-specific exception handling• potentially executes in parallel with Scheduling other threads, even threads within Attributes thread_tthe same task hasthre aminimald_create( )state, threa dfor_te rlomiwna toe(v)erhead Exception Handlers • thread_suspend(), thread_resume() • definesthread_s wthrapead-specific_exception_por texceptions() handling Mach SchedulerMach Scheduler Assigns runnable thread•s assignsto pro crunnableesors threads to processors Mach Host •Symmetric Multi-Processor (SMP) SMP: threads run on any available • • Threads run on any CPU CPU processoravailable, pprroefceerssenceor for affinity

127 full• Pyr eprfeeemptivrence fore a fschedulingfinity Time • •Fully Preby ehighermptive priority Scheduling threads when 95 Kernel - 79 Core • runningBy highe rin pr iuserority tmodehreads 63 when running in user mode GUI - by higher priority real-time • By higher priority real-time 31 threads while running in kernel modethreads while running in App kernel mode 0 • time constraint threads watched to assure good behavior Mach Port

• unidirectional communication channel between a client requesting a service and a server providing that service • has a single receiver and (potentially) multiple senders • state associated with a port: message queue, count of port rights referencing it • kernel has services to allocate ports, and every Mach entity (except virtual memory ranges) is named by a port, so ports are created implicitly during creation of such entities Mach Port Right

• a port can only be accessed via a port right: an entity that indicates the right to access a speciﬁc port in a speciﬁc way • three types of port rights: - receive: allows to receive messages from a port - send: allows to send messages to a port - send-once: allows to send a single message to a port • port rights can be copied and moved between tasks via Mach messages • can acquire certain port rights from a global service Mach IPC ServicesMach Message Messaging across Mach ports •M•acha ptypedorts a collectionre endpoin tofs data passed Mach IPC betw• Meseensage twquoeu entitieses • not• Se ma aMachphore sentity, Locks inets its own right •Capabilities represented by• Macmessagesh port rights are queued, hold state between the time a message is sent • Send, send-once Viirttuall Memory and when it is received Viirttuall Memory Receive Space Space • •M•essagesmessages carry can carry data (inline and out-of-line),• Data (inline porand to urights,t-of-lin evir) tual Task Task memor• Additioyn rangesal Mach pandort rsenderights identity inf• Sormationender identity information Mach Security Services

• implements mechanism, not policy • no authentication checking • capabilities in the form of Mach port rights • subdivide capabilities to create extra privileges • sender identity tagging on each message • policy decisions implemented at the BSD level Mach Memory Object

• represents non-resident state of given memory ranges • memory manager: task implementing a memory object (responding to messages sent to its port), responsible for filling main memory with e.g. contents of backing store • main memory is a cache for the contents of the various memory objects, kernel maintains cache by sending messages to the memory object ports • Mach has no concept of files: BSD vnodes implemented on top of memory objects Mach VirtualMach Memory Address ServicesSpace Mach controls most asp•ecdefinests of v ithertu asetl m ofem validory virtual •Layoutaddr fleessesxibility that a thread executing in Mach VM Services a task is allowed to reference • Each task’s address space can be Address Address • layoutcons flexibilitytructed with a unique layout Space Space •P•roprtecotectionted addr eagainstss spac eunauthorizeds . access• Guar orded update against unauthorized . . access or update . • copy-on-write . . •Copy-on-Write optimization . . •C•oncontrtrolleolledd sha rsharinging VM Object • semantics• Single page, implemented mapped file, through Cache actionsto shar ofed memor complexy rmanageregions for memory object of a memory range References

• Apple developer information on Darwin - http://developer.apple.com/documentation/Darwin/ - http://developer.apple.com/opensource/ • Darwin source code - http://www.opensource.apple.com/darwinsource/ - http://darwinsource.opendarwin.org/ • Darwin Kernel Programming Guide on Mach - http://developer.apple.com/documentation/Darwin/Conceptual/ KernelProgramming/Mach/chapter_6_section_1.html - http://developer.apple.com/documentation/Darwin/Conceptual/ KernelProgramming/boundaries/chapter_14_section_4.html References

• CMU Mach project - http://www.cs.cmu.edu/afs/cs/project/mach/public/www/ mach.html • OSF Mach 3.0 manuals - http://www.cs.cmu.edu/afs/cs/project/mach/public/www/doc/ osf.html • Wikipedia: Mach kernel - http://en.wikipedia.org/wiki/Mach_kernel • kernelthread.com - http://www.kernelthread.com/mac/oshistory/8.html • M.K. McKusick et al, The Design and Implementation of the 4.4 BSD Operating System, Addison-Wesley, 1996 WWDC Scholarships

• to attend Apple’s World Wide Developers Conference Aug 7-11 2006 - http://developer.apple.com/wwdc/ • Apple University Consortium scholarships for Australian university students and staff: (appls close 23/5/06) - http://auc.edu.au/audf/wwdc06/ • Apple Developer Connection (ADC) Student program scholarships: (appls close 19/5/06) - http://developer.apple.com/wwdc/students/ Acknowledgements

• The speaker gratefully acknowledges support to attend the Apple World Wide Developers Conference WWDC 2004 by the Apple University Consortium Australia - http://auc.uow.edu.au/ • Graphics Credit - Jim Magee, Core OS Kernel Team, Apple Computer: - Slides from WWDC 2002 Session 107 “The Darwin Kernel”