An architecture (TEGA) and analysis for making transformer models perform generalizable first-order logical entailment - deciding whether a first-order conclusion follows from premises parameterized in the model weights - cast as knowledge-graph query answering. It maps the concept-shift and covariate-shift notions of out-of-distribution generalization onto unseen knowledge and unseen query types, exposes a mismatch between conventional positional encodings and logical reasoning, and adds logic-aware design choices that improve entailment accuracy on unseen query types.
This paper studies whether transformer models can perform generalizable first-order logical entailment - deciding whether a first-order conclusion follows from a set of premises - using knowledge par…