Classification problems typically assume each instance is a single point in a vector space. But many real-world scenarios—from image histograms to sensor readings—represent instances as distributions. A new preprint on arXiv by Jia and Lin tackles this challenge by introducing a dimension reduction technique within the Wasserstein metric space.
The paper, "Canonical Variates in Wasserstein Metric Space", addresses classification algorithms based on pairwise distances using the Wasserstein metric between distributions. The core innovation is a novel approach grounded in maximizing Fisher's ratio—defined as the quotient of between-class variation to within-class variation. The directions that maximize this ratio are termed discriminant coordinates or canonical variates axes.
Dimension Reduction for Distributional Data
In practice, both between-class and within-class variations are defined as the average squared Wasserstein distances between pairs of distributions, with pairs either belonging to the same class or to different classes. This ratio optimization is achieved through an iterative algorithm that alternates between optimal transport and maximization steps within the vector space. The goal is to reduce dimensionality while preserving class separability.
Algorithm and Empirical Performance
The iterative algorithm alternates between computing optimal transport maps and updating the projection matrix. Empirical studies assess the algorithm's convergence and demonstrate that the dimension reduction technique substantially enhances classification performance. According to the paper, the new method outperforms well-established algorithms that operate on vector representations derived from distributional data. It also exhibits robustness to variations in how instances are summarized by distributions, such as the number of components in a Gaussian mixture model (GMM) representation.
Implications for Machine Learning
This work extends canonical variate analysis—traditionally used in multivariate statistics—to the non-Euclidean geometry of the Wasserstein space. By using the Wasserstein metric, the method naturally handles distributions with different shapes, locations, and scales. The ability to reduce dimensions while retaining discriminatory information could benefit applications in signal processing, computer vision, and any domain where data is naturally distributional.
While the paper does not explicitly address supply chain or logistics, the technique is directly applicable to tasks such as demand pattern recognition, sensor data classification, and anomaly detection from distributional inputs. For enterprise technology leaders, this represents a mathematically rigorous tool for improving model accuracy when working with aggregated or probabilistic data.
The full preprint is available on arXiv (arXiv:2405.15768) under the Statistics > Machine Learning category.