Black-box optimization problems where most parameters of a good solution are exactly zero are common in fields from control systems to machine learning. Existing sparse black-box optimizers rely on hand-designed sparsity operators, bi-level schemes, or zeroing thresholds—defeating a key advantage of estimation-of-distribution algorithms (EDAs). According to a paper on arXiv by Faust, Andreas; Nitzsche, Sven; and Becker, Juergen, a new approach using multivariate zero-inflated Gaussian (ZIG) distributions as EDA sampling laws closes this gap.
The Sparsity Challenge in Black-Box Optimization
EDAs are a powerful class of evolutionary methods that fit a probability distribution to the best individuals and sample the next generation from it, avoiding hand-crafted mutation and crossover operators. They are well established on continuous parameter spaces but have not previously been generalized to sparse ones, where most coefficients are exactly zero. Existing sparse black-box optimizers reintroduce exactly what EDAs were designed to avoid: hand-crafted sparsity operators, bi-level schemes that alternate between support set and active values, and zeroing thresholds.
Zero-Inflated Gaussian Distributions as EDA Sampling Laws
The proposed method uses a latent Gaussian model with separate indicator and value dimensions to represent sparsity patterns, correlations among active parameters, and interactions between the two. This allows sparsity patterns and active values to be optimized jointly, without a hierarchy. The paper shows that the latent parameters of this model are identifiable from observed samples, unlike in missing-data settings where related constructions originate.
Identifiability and Amortized Inversion-Based Estimators
To make the approach practical, the authors introduce amortized inversion-based estimators for the latent parameters. These estimators accurately recover latent correlation structures, enabling the ZIG-EDA to learn effective sparse representations during optimization.
Lunar Lander Benchmark Performance
On the Lunar Lander benchmark, the ZIG-EDA converges faster and reaches higher final returns than three alternatives: a dense Gaussian EDA, a hand-crafted sparse evolutionary algorithm, and an ad-hoc sparse EDA. Critically, the ZIG-EDA finds controllers with only a small fraction of parameters active, demonstrating that sparsity is learned automatically.
This work represents a significant step in unifying sparsity and black-box optimization without design-specific assumptions. By eliminating the need for hand-crafted sparsity mechanisms, ZIG-EDAs could enable more efficient optimization in problems where parsimonious solutions matter—from robotics to network design.
