In partially observable environments, agents must decide when to gather information and which observations are worth the cost. Standard POMDPs evaluate the value of information only based on its eventual effect on rewards. The $ ho$-POMDP framework, however, directly rewards uncertainty reduction through a belief-dependent utility $ ho$, with both its choice and weight often manually tuned for each task.
We demonstrate that active inference entirely removes this tuning. Minimizing Expected Free Energy (EFE) is exactly equivalent to solving a $ ho$-POMDP where the utility is expected information gain, and the exploration weight is fixed at $w=1$, as the variational bound expresses pragmatic and epistemic value in the same units (nats).
We prove this equivalence for observe-then-commit POMDPs and extend it to factored observation POMDPs, a broader class covering interleaved observe-act problems such as non-destructive testing and mobile sensing, where gathering information leaves the hidden state unchanged.
Experimental results support this theory. Across various environments, from the classic Tiger problem to RockSample and a new Structural Inspection benchmark with over 65,000 states, the untuned weight matches or outperforms reward-only planning at the same horizon, avoids the over-exploration of task-tuned bonuses, and is near the reward-maximizing knee of the success-reward Pareto frontier. The practical payoff is an exploration objective that works out of the box.
In applications such as fault detection and medical screening, where every test has a price and every missed fault has a cost, EFE provides a belief-dependent utility that is derived rather than tuned.
Blogger's Review: This study introduces a novel perspective on decision-making through Expected Free Energy, emphasizing the importance of efficient exploration in complex environments. Particularly in cost-sensitive applications, the belief-dependent utility will significantly enhance the reliability and efficiency of decisions, providing a solid foundation for theoretical research while opening new possibilities for practical applications.