AS2: Attention-Based Soft Answer Sets for End-to-End Differentiable Neuro-Soft-Symbolic Reasoning

原文链接: arXiv:2603.18436

PDF

摘要

Neuro-symbolic AI systems typically couple a neural perception module to a discrete symbolic solver through a non-differentiable boundary, preventing constraint-satisfaction feedback from reaching the perception encoder during training. This paper introduces AS2 (Attention-Based Soft Answer Sets), a fully differentiable neuro-symbolic architecture that replaces the discrete solver with a soft, continuous approximation of the Answer Set Programming (ASP) immediate consequence operator. AS2 achieves 99.89% cell accuracy and 100% constraint satisfaction on Visual Sudoku, and >99.7% digit accuracy on MNIST Addition across all scales.

1. 问题定义

“The challenge of combining these two capabilities arises from the incompatibility of their computational registers, since neural inference operates over continuous distributions while symbolic reasoning requires discrete variable assignments that satisfy hard logical constraints.”

Perceptual Constraint-Satisfaction Problems (P-CSPs) require mapping raw sensor data to discrete symbolic assignments that collectively satisfy declarative constraints. The dominant paradigm uses a pipeline architecture:

1. Neural network processes inputs
2. Discrete symbolic solver finds globally consistent assignment

This introduces a fundamental bottleneck: the discrete handoff prevents constraint-satisfaction gradients from flowing back into perception modules. State-of-the-art systems (NeurASP, DeepProbLog, Scallop, PBCS) all exhibit this limitation.

Key Challenge: A single confident perception error can render the symbolic grounding unsatisfiable, and the perception module receives no feedback about how its predictions affect downstream reasoning.

2. 方法框架

2.1 Core Innovation: Soft Probabilistic TP Operator

AS2 replaces the discrete ASP solver with a continuous approximation through three key design principles:

1. Continuous Probability Distributions

• Maintains per-position probability distributions over a finite symbol domain throughout the forward pass
• Constraints enforced through differentiable loss derived from fixed-point residual
• No external solver invoked at training or inference time

2. Constraint-Group Membership Embeddings

“AS2 is entirely free of conventional positional embeddings.”

Instead of learned/sinusoidal positional encodings, AS2 encodes problem structure through constraint-group membership embeddings that reflect the declarative ASP specification:

• Each token’s position defined by which constraint groups it participates in
• For Sudoku: rows, columns, and boxes
• Makes representation invariant to arbitrary permutations of position index

3. Probabilistic Lift of TP Operator The classical TP (immediate consequence) operator from logic programming maps interpretations to their immediate consequences. AS2 lifts this to probability distributions:

• Replaces set-theoretic intersection with element-wise products
• Computes constraint violations as squared distance between current distribution and its image under TP
• Loss is zero if and only if predicted distribution is a valid one-hot assignment satisfying all constraints

2.2 Architecture Details

Attention-Based Reasoning: Uses Transformer architecture with:

• Self-attention for constraint propagation
• Cross-attention between perception and reasoning modules
• Constraint-aware attention masking

Declarative Specification: The reasoning module’s structure is fixed entirely by the declarative logic program—not learned from data. This ensures the model respects symbolic semantics while remaining differentiable.

3. 实验结果

3.1 Visual Sudoku

Method	Grid Accuracy	Constraint Satisfaction	External Solver
SATNet	63.2%	Not guaranteed	No
NeurASP	97.3%	100%	Yes (Clingo)
PBCS	99.4%	100%	Yes (CP-SAT)
AS2	99.89%	100%	No

AS2 achieves state-of-the-art grid accuracy with 100% constraint satisfaction verified by Clingo, without requiring an external solver at inference time.

3.2 MNIST Addition

N (Addends)	Digit Accuracy	Task
N=2	>99.7%	Addition of 2 digits
N=4	>99.7%	Addition of 4 digits
N=8	>99.7%	Addition of 8 digits

AS2 maintains high accuracy across all scales, demonstrating generalization beyond Latin-square constraints to arithmetic reasoning.

3.3 Key Advantages

• End-to-end differentiability: Constraint feedback flows directly to perception encoder
• No external solver: Greedy constrained decoding achieves perfect satisfaction
• Structure-aware: Embeddings reflect logical problem structure, not arbitrary positions
• Sound semantics: Loss has no degenerate minima at uniform distribution

4. 优点与局限

优点

• Fully differentiable architecture enables gradient-based training with constraint feedback
• No solver dependency at inference—enables real-time applications
• Declarative grounding ensures model respects symbolic semantics
• Position-agnostic embeddings generalize across problem instances
• Competitive accuracy with state-of-the-art while maintaining differentiability

局限

• Limited to finite symbolic domains (cannot handle continuous variables directly)
• Constraint specification must be provided declaratively
• May not scale to extremely large constraint systems
• Requires careful design of constraint-group embeddings for each problem type

5. 为什么对AI硬件重要

AS2 represents a significant step toward hardware-friendly neuro-symbolic AI:

1. Eliminating Discrete Solvers: By removing the need for external ASP/SAT solvers, AS2 enables purely neural inference that can be accelerated by standard AI hardware (GPUs, TPUs, neural accelerators).

2. Unified Architecture: The fully differentiable design allows the entire system to be implemented as a single neural network, enabling:
   • End-to-end optimization on AI accelerators
   • Reduced memory bandwidth (no solver data movement)
   • Simpler deployment pipelines
3. Constraint-Aware Hardware: The success of constraint-group embeddings suggests opportunities for specialized hardware that:
   • Implements structured attention patterns efficiently
   • Supports declarative constraint specification
   • Optimizes for symbolic-semantic operations

4. Edge Deployment: The elimination of external solvers makes neuro-symbolic AI viable for edge devices with limited compute resources.

5. Hybrid AI Chips: AS2’s architecture suggests a path for AI accelerators that natively support both:
   • Neural perception (standard deep learning)
   • Symbolic reasoning (constraint propagation, logical inference) In a unified, differentiable framework.

参考文献

1. AbdAlmageed, W. (2026). AS2 – Attention-Based Soft Answer Sets: An End-to-End Differentiable Neuro-Soft-Symbolic Reasoning Architecture. arXiv:2603.18436.
2. Yang, Z., et al. (2020). NeurASP: Embracing Neural Networks into Answer Set Programming. IJCAI.
3. Manhaeve, R., et al. (2018). DeepProbLog: Neural Probabilistic Logic Programming. NeurIPS.
4. Li, J., et al. (2023). Scallop: A Language for Neurosymbolic Programming. PLDI.
5. Wang, P., et al. (2019). SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. ICML.