Caladan: a Distributed Meta-OS for Data Center Disaggregation

Caladan: a Distributed Meta-OS for Data Center Disaggregation

Caladan: A Distributed Meta-OS for Data Center Disaggregation Lluís Vilanova Lina Maudlej Matthias Hille Imperial College London Technion Technische Universität Dresden Nils Asmussen Michael Roitzsch Mark Silberstein Barkhausen Institut Barkhausen Institut Technion ABSTRACT Data center resource disaggregation promises cost savings by pool- ing compute, storage and memory resources into separate, net- worked nodes. The benefits of this model are clear, but a closer look shows that its full performance and efficiency potential cannot be easily realized. Existing systems use CPUs pervasively to interface ar- bitrary devices with the network and to orchestrate communication among them, reducing the benefits of disaggregation. In this paper we present Caladan, a novel system with a trusted uni- Figure 1: Example machine learning service with a main versal resource fabric that interconnects all resources and efficiently user application (App) processing external requests, a file offloads the system and application control planes to SmartNICs, system service (FS and SSD), and a machine learning accel- freeing server CPUs to execute application logic. Caladan offers erator (MLAccel). The numbered arrows (used in the text) three core services: capability-driven distributed name space, virtual correspond to network messages across resources in the data devices, and direct inter-device communications. These services center. are implemented in a trusted meta-kernel that executes in per-node SmartNICs. Low-level device drivers running on the commodity host MLAccel resources that does not involve App. Second, the MLAccel OS are used for setting up accelerators and I/O devices, and exposing resource runs on a hardware accelerator; the model inference thus them to Caladan. Applications run in a distributed fashion across does not require any CPU computations. Third, the whole system CPUs and multiple accelerators, which in turn can directly perform operates in a form of a dataflow where resources are invoked ina I/O, i.e., access files, other accelerators or host services. Our dis- pipeline by their predecessors in the flow graph. While App lever- tributeddataflowruntimerunsontopofthissubstrate.Itorchestrates ages the same logical execution path for each request, it may invoke the distributed execution, connecting disaggregated resources using different hardware resources each time, allocated on-demand bythe data transfers and inter-device communication, while eliminating system to meet the load. the performance bottlenecks of the traditional CPU-centric design. Unfortunately, the idealized design of Figure 1 is not what existing systems software supports. First, with MLAccel, FS and SSD all located 1 INTRODUCTION in different nodes, each node deploys its own CPUs to execute device Operators are moving towards a data center model where compute, access logic, provisioned to enable efficient control of the devices and storage and memory resources are disaggregated in order to improve scalable RPC interfaces for applications. Second, current software TCO [5, 10, 11, 13, 16, 18, 20, 21, 23, 24, 27, 29]. The primary bene- stacks would force centralized management of all the resources (FS, fits stem from pooling resources onto separate nodes, which allows SSD and MLAccel) from the application’s CPU (App). This 1-to-all more flexible allocation and sharing of hardware, thereby leading control and data path topology would involve more network trans- to improved utilization and reduced hardware redundancy. fers to run the model inference task compared to the peer-to-peer While at the high-level the cost benefits of the disaggregated re- topology in Figure 1. These two factors imply that a practical disag- source model are clear, current software stacks do not allow realizing gregated system will quickly lose its performance and TCO benefits; the model’s full performance and efficiency potential. For example, centralized resource management requires an increased number of consider a network service that performs image classification using network messages, and device access logic requires additional CPUs a machine learning model. Its high-level organization is shown in to execute the system’s and application’s control plane, instead of Figure 1. The application runs as a regular CPU process (App). For using them to execute critical application business logic. each classification request, it sends out a read request to the file The crux of the problem is the inherently CPU-centric model of system ( 1 and 2 ; running in FS and SSD) to read the appropriate existing systems; CPUs play a central role in managing devices and model from a file (if it has not been cached already); e.g., to classify in orchestrating operations across them. This is because only CPUs types of cats or dogs it reads a cat or a dog model respectively. The knowadevice’swireprotocol(i.e.,howtooperateitsbare-metalinter- SSD then sends the file contents into a custom machine learning faces). Instead, peer-to-peer interactions assume that every device in accelerator to load the model ( 3 ; in MLAccel), computes the result the system will know the protocol of every other device, which seems and sends it back to the application ( 4 ; in App). unrealistic. For example, we cannot expect every SSD vendor to im- We make three primary observations about an idealized disaggre- plement the logic to push requests into our custom MLAccel device. gated system. First, achieving high performance in such an appli- Ideally, we would expose all disaggregated resources through the cation requires direct data and control path between the SSD and network by co-locating networked device access logic with every Lluís Vilanova, Lina Maudlej, Matthias Hille, Nils Asmussen, Michael Roitzsch, and Mark Silberstein resource. This would allow efficient implementations of a wealth of use-cases like secure direct device assignment of disaggregated resources (e.g., directly accessing file contents in SSD from App), ap- plication and device RPC interfaces, or even distributed dataflow exe- cution models [30] like the one shown in Figure 1. In fact, distributed dataflow is increasingly used in data center-scale programming frameworks [2, 3, 12, 32], making its optimized execution particu- larly useful in a disaggregated setting. In this paper we describe Caladan, a data center-scale distributed meta-OS that eliminates the limitations of existing CPU-centric de- vices using a novel universal resource fabric. Caladan’s fabric interconnects all resources in the system, and exposes generic de- vice access logic through a small set of abstract resource access and messaging primitives. Importantly, it regains the lost benefits of the Figure 2: Placement of the hardware and software compo- disaggregated resource model without the need for additional CPUs nents of the application in Figure 1; slices correspond to logic nor changes in the devices it interconnects. components, and numbers to the messages used by them Caldan’smeta-OStakestheroleofsecurely managingaccesstoand in Figure 1. Dark gray components are part of Caladan’s communication among resources anywhere in the data center, while meta-kernel, trusted by the data center operator. Light gray its users can take advantage of existing OSes and software stacks to components are higher-level services and interfaces trusted deploy their application logic and device-specific drivers. The fabric by the application only if used. is provided as a trusted component that executes in per-node Smart- NICs [22], to which device access logic is offloaded. All resources are globally addressed, routing operations to them regardless of their critical path operations of its universal resource fabric are offloaded physical location and resource type. Since multiple tenants can share into per-node SmartNICs [22]. the same data center, the fabric also has strong security guarantees Caladan follows well-known `kernel principles in its design. through the use of object capabilities [9], which can be seen as pro- The data center operator must trust the Caladan meta-kernel, Cal- tected resource handles (akin to file descriptors in POSIX systems). adan’s trusted computing base (TCB), which implements the basic re- Caladan follows well-known `kernel principles and offers three source abstraction and messaging primitives of Caladan’s universal basic objects that form its universal resource fabric: memory, de- resource fabric. Higher-level components like device drivers, system vice slices and device requests. Device slices are a virtual instance services (e.g., management of resource placement and scheduling), of a resource, like a CPU process, a portion of an SSD disk that and even constructs for complex execution models like dataflow an application can directly access, or even a virtual function of a reside outside of the meta-kernel.§3 further develops the reason- PCIe-attached device [26]. Device requests are the only messaging ing and affordances behind these principles, while§4 contains a primitive in Caladan, and contain immediate values (e.g., used to description of physical device drivers in Caladan. represent a device-specific request) and references to other resources Caladan’s universal resource fabric abstracts all resources as ob- or requests using capabilities. jects. Caladan’s primitives must be secure, since multiple tenants The contributions of this paper are a description of the security must co-exist in the data center. The meta-kernel provides a sound aspects and functional

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    7 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us