Integrating external tools with Large Language Models (LLMs) has emerged as a promising paradigm for accomplishing complex tasks. However, LLMs still struggle to effectively manage large tool collections, prompting researchers to explore retrieval-based methods to pre-select the most relevant options, addressing input length and latency constraints. Existing retrievers are often misaligned with tool-calling LLMs due to their separate training processes.
This paper presents PORTS, a novel odds ratio preference optimization method for training retrievers aimed at tool selection. Our approach fine-tunes a retriever using a perplexity-inspired preference signal from a frozen LLM, optimizing the correlation between selection probabilities and downstream performances while jointly enforcing a contrastive semantic loss between documentation strings. The versatility of PORTS and its ability to significantly improve tool selection accuracy are demonstrated through extensive experiments on six datasets, two encoder models, and three LLMs with diverse prior knowledge. With low computational demands, our alignment process facilitates generalization to new queries and tools, proving valuable for practical applications with evolving toolsets.
Blogger's Review: The PORTS method enhances the accuracy of tool selection by optimizing the training process of retrievers. This innovative approach provides robust support for the practical applications of LLMs, demonstrating excellent adaptability and efficiency when faced with diverse toolsets.