Abstract
Large metal-organic framework (MOF) databases support simulation, screening, and machine learning through crystallographic information files (CIFs). Subtle chemical and structural errors in these inputs can compromise downstream results and hinder manual inspection. Advances in large language models (LLMs) in computational chemistry offer paths beyond predictive screening toward fine-grained diagnosis with evidence-grounded explanations. However, two challenges remain:
- Limited Fine-Grained Attribution: MOF-specific validators and machine-learning models scale detection but provide fixed checks, readiness scores, or coarse labels rather than evidence-grounded explanations.
- Unreliable CIF Reasoning: Direct LLM auditing is costly and unreliable because chemical evidence is implicit across atom-site records, requiring geometric, connectivity, occupancy, and charge calculations. Both stem from weak coupling between chemical evidence and language-model explanation.
We introduce MOF-Sleuth, a reinforcement-guided CIF auditing agent with two modules:
- Deterministic Forensic Lab: Derives composition, geometry, connectivity, occupancy, coordination, and charge evidence.
- Sleuth Reasoning Engine: Uses this evidence to produce an evidence-grounded explanation, error types, and a binary decision.
Reward-guided reinforcement learning (RL) turns tool measurements into chemical explanation-level supervision, rewarding not only the final answer but also cited chemical evidence and evidence-supported diagnoses. We introduce Chemically Grounded Diagnosis (Chem-GD), a metric that assesses whether a correct diagnosis is explained by factual, relevant CIF-derived evidence. Across four benchmarks, MOF-Sleuth establishes state-of-the-art performance among LLM-based approaches and MOF-specific machine-learning methods, demonstrating gains in detection, attribution, and grounded explanation quality.
Blogger's Review: The introduction of MOF-Sleuth offers an innovative solution for CIF auditing, enhancing the interpretability of chemical evidence through reinforcement learning. This integration of advanced computational chemistry and machine learning paves the way for more reliable and explainable scientific data, as evidenced by its superior performance across various benchmarks, indicating a future of more precise and efficient MOF-related research.