Aggregating noisy and conflicting free‑text hypotheses into a reliable consensus is a core challenge for real‑world NLP deployments. A monolithic large language model (LLM) offers expressive power but suffers from context limits, compounded hallucinations, and high inference latency. Traditional weak supervision provides statistical rigor yet is confined to discrete classes, making it unsuitable for open‑ended text.
Loom bridges these paradigms. Modular heuristics—diagnostic templates populated with episode‑specific entities, timestamps, and metrics—first generate open‑form hypotheses. These hypotheses are projected into a continuous embedding space. The heart of the method is an iterative centroid‑based reweighting algorithm: each iteration computes the centroid of all embeddings, then adjusts hypothesis weights according to their similarity to the centroid, suppressing contradictory signals while amplifying agreement.
The resulting weighted set feeds a single lightweight LLM synthesis step, drastically reducing the number of model calls. On the OpenRCA benchmark, Loom matches a state‑of‑the‑art autonomous agent on the Bank and Market‑2 datasets, trails slightly on Market‑1 and Telecom, yet achieves roughly 26× faster inference overall and about 33× faster when using an 8B‑parameter synthesizer.
Our deployment experience highlights several lessons: deeper agentic pipelines capture more nuance but incur prohibitive latency; attempts at redundancy detection via simple similarity thresholds yielded negative results; and deterministic consensus markedly improves trust among subject‑matter experts (SMEs).
Review