NeFut Logo NeFut
Admin Login

[CS.AI] Designing Agentic AI Workflow Portfolios under Imperfect Selection and Compute Cost

Published at: 2026-09-17 22:00 Last updated: 2026-09-18 00:46
#AI #Machine Learning #optimization

Agentic AI systems often tackle the same task using multiple workflows that differ in reasoning strategy, verification structure, and compute cost. Selecting the workflow with the highest average performance for deployment seems natural but can be suboptimal because different workflows succeed on different instances. To address this, we propose a portfolio‑selector paradigm: a firm runs several workflow executions in parallel and selects the final answer after observing their outputs. Additional executions may uncover correct answers missed by the best standalone workflow, yet they consume compute and introduce plausible distractors that complicate selection.

We formalize this as a workflow portfolio problem, where the firm jointly decides the total run size and the allocation across workflow types. To quantify selector quality we introduce the odds‑lift index (OL), defined as $OL = \frac{P(\text{correct}\mid\text{selector})}{P(\text{correct})}$, which measures the selector’s improvement over random choice. Using this metric we derive sharp upper and lower bounds on the value of workflow diversity.

For finite workflow pools we provide exact integer programming formulations, linear programming relaxations, and randomized rounding procedures, together with computable performance certificates. For large implicit workflow classes we derive a finite‑dimensional dual and employ an ellipsoid method with a pricing oracle to identify workflows that maximize weighted accuracy net of recurring compute cost. Under a mild condition the method reaches a near‑optimal solution to the relaxation with only polynomially many oracle calls.

We evaluate the framework on three datasets—ABCD, Schema‑Guided Dialogue, and HotpotQA. Compared with the best standalone workflow, portfolio optimization improves held‑out selector accuracy by 3.1, 7.5, and 0.9 percentage points respectively. Dual‑guided workflow generation adds another 3.5 points on ABCD and 24.1 points on HotpotQA, with no extra gain on Schema‑Guided Dialogue.

Review: The paper offers a comprehensive treatment of workflow diversity under imperfect selection and compute constraints, delivering both theoretical bounds and practical algorithms. Empirical results demonstrate that even when strong single‑workflow baselines exist, a well‑designed portfolio can yield notable performance gains, providing actionable guidance for real‑world deployments.

Original Source: https://arxiv.org/abs/2609.18126

[h] Back to Home