Knowledge graph completion is usually evaluated by ranking observed triples above randomly corrupted ones, which implicitly treats every unobserved fact as false. This evaluation breaks down when the target is a description logic (DL) knowledge base rather than a plain graph, because the open‑world assumption and deductive closure introduce three logical statuses for a candidate axiom: entailed, contradictory, or undetermined. A model that cannot separate a logically impossible axiom from a plausible novel one is not merely less accurate—it is semantically wrong.
We propose a hierarchy of four increasingly strict faithfulness criteria for KBC models:
- Discrimination – the model distinguishes true triples from random negatives.
- Logical Admissibility – the model never assigns high scores to axioms that contradict the knowledge base.
- Monotonic Logical Faithfulness – after adding a new axiom, scores for existing axioms do not decrease.
- Probabilistic Logical Faithfulness – the model’s score corresponds to the relative model count $$P(\alpha\mid\mathcal{O}) = \frac{\#(\mathcal{O}\cup\{\alpha\})}{\#(\mathcal{O})}$$ which yields 1 for entailed axioms, 0 for contradictory ones, and a value in (0,1) for undetermined axioms.
We prove that these criteria form a strict implication chain: Discrimination ⇒ Logical Admissibility ⇒ Monotonic Logical Faithfulness ⇒ Probabilistic Logical Faithfulness.
In experiments we evaluate several knowledge‑graph and logic‑geometric embedding models on $ EL$ ontologies. Test sets—generated by a reasoner—contain entailed, contradictory, and undetermined axioms. Results show that high ranking accuracy does not guarantee any of the faithfulness criteria, and none of the evaluated models satisfies the full hierarchy.
The code is released at https://github.com/bio-ontology-research-group/kbc.
Review: The paper offers a rigorous, layered definition of logical faithfulness for knowledge‑base completion and demonstrates that current embedding approaches fall short, establishing a clear benchmark for future work.