Designing an Efficient Decomposition Method for the Deployment of Machine Learning Models on Serverless Platforms
Abstract
In recent years, Serverless computing has emerged as a persuasive paradigm aiming to reshape the
cloud computing landscape considerably. Serverless offers a scalable and cost-effective deployment
model where users can run applications without the need to manage or provision servers. The
underlying infrastructure is entirely abstracted and has the ability to scale automatically in a
flexible manner, while the users are charged exclusively for the resources they use. In parallel, we
have witnessed a surge in the adoption of Artificial Intelligence and Machine Learning (ML)
technologies in various application domains. Since Serverless architectures are not tailored to
address the unique challenges posed by resource-intensive jobs, combining ML with Serverless proves
to be a complex undertaking.
In this thesis, we propose a solution for deploying ML models on Serverless platforms, specifically
for inference jobs. Our model-agnostic approach is based on a flexible decomposition of such models
into sub-models, referred to as slices, and the execution of inferences in a workflow of Serverless
functions. We rely on conducting a thorough investigation of the limitations affecting the most
popular Serverless platforms on the market and devising strategies to overcome them. Our
experimental evaluations are performed on AWS, considering the ONNX open source format for ML model
representation. Our results show that our decomposition method enables running ML inference on
Serverless, regardless of the model size, benefiting from the high scalability of this architecture
while lowering the strain on computing resources such as required runtime memory.
Metadata
Links
