Designing an Efficient Decomposition Method for the Deployment of Machine Learning Models on Serverless Platforms
Abstract
In recent years, Serverless computing has emerged as a persuasive paradigm aiming to reshape the cloud
computing landscape considerably. Serverless offers a scalable and cost-effective deployment model where
users can run applications without the need to manage or provision servers. The underlying infrastructure
is entirely abstracted and has the ability to scale automatically in a flexible manner, while the users are
charged exclusively for the resources they use. In parallel, we have witnessed a surge in the adoption
of Artificial Intelligence and Machine Learning (ML) technologies in various application domains. Since
Serverless architectures are not tailored to address the unique challenges posed by resource-intensive
jobs, combining ML with Serverless proves to be a complex undertaking.
In this thesis, we propose a solution for deploying ML models on Serverless platforms, specifically for
inference jobs. Our model-agnostic approach is based on a flexible decomposition of such models into sub-
models, referred to as slices, and the execution of inferences in a workflow of Serverless functions. We rely
on conducting a thorough investigation of the limitations affecting the most popular Serverless platforms
on the market and devising strategies to overcome them. Our experimental evaluations are performed
on AWS, considering the ONNX open source format for ML model representation. Our results show
that our decomposition method enables running ML inference on Serverless, regardless of the model size,
benefiting from the high scalability of this architecture while lowering the strain on computing resources
such as required runtime memory.
Metadata
Links
