Artificial Intelligence #z-plane neural networks#bounded geometric activation
Z-Plane Neural Networks Replace ReLU and LayerNorm with Bounded Geometric Activation
Researchers propose Z-Plane Neural Networks, which replace traditional ReLU activations and LayerNorm with a bounded geometric activation called Radial Bounding. This new approach maintains 1-Lipschitz continuity, prevents gradient vanishing, and preserves directional information. A 100-layer Z-Plane MLP achieved 98.34% accuracy on MNIST without any ReLU or LayerNorm, demonstrating numerical stability.
Jun 16, 2026 1 source