Financial QA systems are usually enhanced before deployment by improving retrieval, prompting, or agent coordination, after which their behavior remains fixed. In real use, new SEC‑filing questions repeatedly expose heterogeneous errors across time periods, entities, evidence usage, and calculations. Existing self‑improvement methods can turn failures into new behaviors but offer little control over where corrections apply and risk breaking previously correct answers.
We therefore frame post‑deployment improvement as controlled behavioral maintenance: recurring failures should become scoped skill patches, and each patch must be deployed without introducing regressions. Based on this view we built FINSKILLOPS, a multi‑agent system for SEC filing QA. It derives reusable skills from evidence‑grounded, typed failure diagnoses and manages them through targeted validation, protected regression checks, negative controls, and versioned replacement or retirement.
Across six financial QA benchmarks, a single frozen skill registry achieves the highest verdict‑weighted correctness and reference consistency among evaluated systems. Evolved skills raise correctness from 3.70 to 4.55 on the enhanced benchmark. In a separate 12‑round operational study, only six of 33 proposed skills are promoted, while the monitoring non‑correct rate drops from 20.0% to 12.5%. These results demonstrate that controlled skill scope, admission, and lifecycle management form the foundation for reliable self‑improvement.
Review