SONIC: Application-Aware Data Passing for Chained Serverless Applications
SONIC: Application-aware Data Passing for Chained Serverless Applications Ashraf Mahgoub Karthick Shankar Subrata Mitra Ana Klimovic Purdue University Carnegie Mellon University Adobe Research ETH Zurich Somali Chaterji Saurabh Bagchi Purdue University Purdue University Abstract introduced serverless workflow services such as AWS Step Data analytics applications are increasingly leveraging Functions [4], Azure Durable Functions [45], and Google serverless execution environments for their ease-of-use and Cloud Composer [24], which provide easier design and orches- pay-as-you-go billing. The structure of such applications tration for serverless workflow applications. Serverless work- is usually composed of multiple functions that are chained flows are composed of a sequence of execution stages, which can be represented as a directed acyclic graph (DAG) [17,53]. together to form a workflow. The current approach of ex- 1 changing intermediate (ephemeral) data between functions DAG nodes correspond to serverless functions (or ls ) and is through a remote storage (such as S3), which introduces edges represent the flow of data between dependent ls (e.g., significant performance overhead. We compare three data- our video analytics application DAG is shown in Fig.1). passing methods, which we call VM-Storage, Direct-Passing, Exchanging intermediate data between serverless functions and state-of-practice Remote-Storage. Crucially, we show that is a major challenge in serverless workflows [34,36,47]. By no single data-passing method prevails under all scenarios design, IP addresses and port numbers of individual ls are not and the optimal choice depends on dynamic factors such as exposed to users, making direct point-to-point communication the size of input data, the size of intermediate data, the appli- difficult.
[Show full text]