Deep neural networks (DNNs) deliver strong performance across many fields but often lack interpretability. Generalized additive models (GAMs) offer a balance, and their neural network-based variants—Neural Additive Models (NAM) and Neural Basis Models (NBM)—use neural networks as nonlinear shape functions to retain interpretability. However, according to a recent paper on arXiv, training NAM and NBM on high-dimensional datasets or when incorporating two-input neural networks to capture feature interactions becomes computationally intractable due to the surge in resource requirements.
The Computational Challenge
NAM and NBM provide interpretability by visualizing each feature's contribution to the prediction, thanks to their GAM-based architecture. Yet, when applied to high-dimensional data or when using two-input NNs to model pairwise interactions, the computational cost and model size grow rapidly. The researchers note that this bottleneck limits the practical deployment of these models in complex real-world scenarios.
Proposed Solution: Feature Selection Layer
The paper introduces a simple but effective mechanism: adding a feature selection layer to both NAM and NBM. This layer learns selection weights during training, allowing the model to prioritize the most relevant features. According to the authors, the method reduces computational costs and model sizes compared to vanilla NAM and NBM. It also makes it feasible to use two-input NNs even on high-dimensional datasets, enabling the capture of feature interactions that were previously too expensive to compute.
Performance and Comparison
The proposed models are computationally efficient while maintaining or improving predictive performance. The researchers report that they exhibit better or comparable performance with state-of-the-art GAMs. The table below summarizes the key differences between vanilla and proposed models:
| Aspect | Vanilla NAM/NBM | Proposed with Feature Selection |
|---|---|---|
| Interpretability | High (GAM-based) | High (GAM-based) |
| Computational Cost | High for high-dimensional or interaction scenarios | Reduced |
| Model Size | Large | Reduced |
| Feature Interactions | Limited by computational complexity | Enabled even in high-dimensional data |
| Performance | High | Better or comparable to state-of-the-art GAMs |
Implications for Enterprise AI
For organizations deploying interpretable machine learning on complex, high-dimensional datasets—whether in finance, healthcare, or industrial automation—this research offers a path to more efficient models without sacrificing insight. The ability to capture feature interactions at manageable computational cost could make GAM-based neural models more practical for large-scale applications. The paper's approach is straightforward to implement, suggesting a low barrier for adoption in existing machine learning pipelines.
In summary, the integration of feature selection into neural additive and basis models addresses a key computational bottleneck, expanding the feasibility of interpretable AI for high-dimensional problems. The work was conducted by researchers Kishimoto, Yasutoshi; Yamanishi, Kota; Matsuda, Takuya; and Shirakawa, Shinichi, and is available on arXiv under a Creative Commons license.