Introduction
In the 3D Computing Continuum, applications unify edge, cloud, and space computing needs, requiring the integration of multiple AI tasks such as object detection, time-series analytics, and natural language processing into Compound AI systems. These systems must meet stringent Service Level Objectives (SLOs) regarding accuracy, latency, and cost.
Runtime Model Selection
A key mechanism for maintaining SLO compliance in Compound AI systems is runtime model selection, which allows for the dynamic switching of AI models tailored to each workflow task. However, existing distributed and compound AI frameworks do not natively support this functionality.
PLAIground Framework
We present PLAIground, a framework designed to enable runtime model selection for Compound AI systems. PLAIground introduces the Compoundable AI Model (CAIM) abstraction, which decouples task semantics from AI model implementations through Task and Data Contracts, allowing model switching without changes to the workflow. Additionally, PLAIground features the Pixie algorithm, which dynamically selects the most suitable model for each task based on SLOs during execution.
Performance Evaluation
Our evaluation on two realistic Compound AI workflows shows that Pixie achieves up to 91.3% accuracy while maintaining SLO compliance, whereas fixed-model strategies can violate cost and latency budgets by up to 21 times or miss accuracy targets by 4%.
Blogger's Review: PLAIground significantly enhances the flexibility and efficiency of Compound AI systems in multi-task environments through the introduction of CAIM abstraction and Pixie algorithm, highlighting the importance of dynamic model selection in complex computing scenarios, providing a new perspective for the integration of edge and cloud computing in the future.