Tensor Logic: A Unified Language for Neural and Symbolic AI
AI lacks a proper programming language. Python dominates AI development, but was never designed for it—PyTorch and TensorFlow bolt on automatic differentiation and GPU support, while automated reasoning and knowledge acquisition remain afterthoughts. Tensor logic solves this by unifying neural and symbolic AI at a mathematical level, using a single construct: the tensor equation.
The Core Insight
Tensor logic rests on one observation: logical rules and Einstein summation are the same operation. A Datalog rule joins relations and projects out shared variables; an einsum joins tensors and sums over repeated indices. Apply a step function to the result, and you convert between the two worlds.
Consider the rule Aunt(x,z) ← Sister(x,y), Parent(y,z). Treating each relation as a Boolean matrix, this becomes:
A[x,z] = H(S[x,y] * P[y,z])
The einsum S[x,y] * P[y,z] implements the join on y. The Heaviside function H collapses any sum greater than zero to 1. Logic and tensor algebra are one.
A tensor logic program is a set of tensor equations. The left-hand side names the tensor being computed. The right-hand side chains tensor joins, projects out unneeded indices, and applies an optional nonlinearity. Nothing else exists in the language.
Implementing Major AI Paradigms
Neural Networks
A single-layer perceptron:
Y = step(W[i] X[i])
A multilayer perceptron, with i ranging over layers and j, k over units:
X[i,j] = sig(W[i,j,k] X[i-1,k])
A transformer requires roughly a dozen equations. Attention computes queries, keys, and values by multiplying the residual stream by learned weight matrices, then compares each query against all keys:
Query[b,h,p,dk] = W_Q[b,h,dk,d] Stream[b,p,d]
Comp[b,h,p,p'.] = softmax(Query[b,h,p,dk] Key[b,h,p',dk] / sqrt(Dk))
Attn[b,h,p,dv] = Comp[b,h,p,p'] Val[b,h,p',dv]
Graph neural networks follow the same pattern—graph structure is a Boolean tensor Neig[n,n'], and message passing is a join:
Agg[n,l,d] = Neig(n,n') Z[n',l,d]
Since RNNs are Turing-complete and tensor logic implements RNNs, tensor logic is Turing-complete.
Symbolic AI
Any Datalog program is a valid tensor logic program. Symbolic reasoning, planning, and knowledge representation work directly. The language accepts Datalog syntax; parentheses instead of square brackets signal a Boolean tensor stored as facts.
Kernel Machines and Graphical Models
A kernel machine maps to a single recursive equation. Graphical models map cleanly: factors are tensors, marginalization is projection, pointwise products are joins, and belief propagation is forward chaining.
Reasoning in Embedding Space
The most powerful capability tensor logic enables is sound reasoning directly in embedding space.
Embed each object as a random unit vector. Store embeddings in Emb[x,d]. The superposition of a set of objects is their sum in embedding space. A dot product then tests membership approximately—similar to a Bloom filter, with error probability decreasing as embedding dimension increases.
Extend this to relations by embedding each tuple as the tensor product of its arguments’ embeddings:
EmbR[i,j] = R(x,y) Emb[x,i] Emb[y,j]
Querying whether (A,B) belongs to relation R:
D[A,B] = EmbR[i,j] Emb[A,i] Emb[B,j]
This returns approximately 1 if (A,B) is in the relation, 0 otherwise. The commutativity and associativity of einsums guarantee the derivation is valid.
Embed rules by replacing their antecedents and consequents with their embeddings. Forward or backward chaining over embedded rules then performs reasoning in embedding space. The key control is temperature T in a sigmoid applied to each equation:
- T = 0: Purely deductive reasoning. Unlike LLMs, the system cannot hallucinate at zero temperature.
- T > 0: Analogical reasoning. Similar objects borrow inferences from each other, weighted by their embedding dot products.
This combines the scalability and generalization of neural networks with the reliability of symbolic reasoning. Intermediate tensors are inspectable at any point, making the reasoning transparent—a direct contrast to LLM-based reasoning models.
Learning
Automatic differentiation in tensor logic is straightforward because only one statement type exists. The derivative of Y[...] = T[...] X1[...] ... Xn[...] with respect to T is simply X1[...] ... Xn[...]. The gradient of an entire program is itself a tensor logic program.
Tensor decomposition generalizes predicate invention from inductive logic programming. Learning the Tucker decomposition of a relation tensor discovers latent structure; thresholding the factor matrices into Booleans converts that structure into symbolic predicates.
Scaling
Dense tensor operations run directly on GPUs. Sparse tensors use two approaches:
- Separation of concerns: sparse operations go to a database query engine, dense subtensors to GPUs, joined at boundaries.
- Tucker decomposition: convert sparse tensors to dense via learned embeddings. This is exponentially more efficient, scales seamlessly with the learning and reasoning algorithms, and introduces only a controllable probability of error.
What This Enables
Tensor logic addresses the two failures that have split AI for decades. Neural approaches scale and learn but cannot reason reliably. Symbolic approaches reason correctly but cannot scale or generalize. Tensor logic’s embedding-space reasoning inherits both sets of properties.
The immediate next steps are implementing tensor logic directly in CUDA, building open-source libraries, and applying it to domains where hallucination and opacity currently block adoption—mathematical reasoning, formal verification, and coding assistants.
More information at tensor-logic.org.