Options
Convergence properties of natural gradient descent for minimizing KL divergence
Citation Link: https://doi.org/10.15480/882.15749
Publikationstyp
Journal Article
Date Issued
2025-07-30
Sprache
English
TORE-DOI
Issue
7
Start Page
1
End Page
28
Citation
Transactions on machine learning research (7): 1-28 (2025)
Publisher Link
Scopus ID
Publisher
OpenReview.net
The Kullback-Leibler (KL) divergence plays a central role in probabilistic machine learning, where it commonly serves as the canonical loss function. Optimization in such settings is
often performed over the probability simplex, where the choice of parameterization significantly impacts convergence. In this work, we study the problem of minimizing the KL divergence and analyze the behavior of gradient-based optimization algorithms under two dual coordinate systems within the framework of information geometry− the exponential
family (θ coordinates) and the mixture family (η coordinates). We compare Euclidean gradient descent (GD) in these coordinates with the coordinate-invariant natural gradient
descent (NGD), where the natural gradient is a Riemannian gradient that incorporates the intrinsic geometry of the underlying statistical model. In continuous time, we prove that
the convergence rates of GD in the θ and η coordinates provide lower and upper bounds, respectively, on the convergence rate of NGD. Moreover, under affine reparameterizations
of the dual coordinates, the convergence rates of GD in η and θ coordinates can be scaled to 2c and 2 c , respectively, for any c > 0, while NGD maintains a fixed convergence rate of
2, remaining invariant to such transformations and sandwiched between them. Although this suggests that NGD may not exhibit uniformly superior convergence in continuous time,
we demonstrate that its advantages become pronounced in discrete time, where it achieves faster convergence and greater robustness to noise, outperforming GD. Our analysis hinges on bounding the spectrum and condition number of the Hessian of the KL divergence at the optimum, which coincides with the Fisher information matrix.
often performed over the probability simplex, where the choice of parameterization significantly impacts convergence. In this work, we study the problem of minimizing the KL divergence and analyze the behavior of gradient-based optimization algorithms under two dual coordinate systems within the framework of information geometry− the exponential
family (θ coordinates) and the mixture family (η coordinates). We compare Euclidean gradient descent (GD) in these coordinates with the coordinate-invariant natural gradient
descent (NGD), where the natural gradient is a Riemannian gradient that incorporates the intrinsic geometry of the underlying statistical model. In continuous time, we prove that
the convergence rates of GD in the θ and η coordinates provide lower and upper bounds, respectively, on the convergence rate of NGD. Moreover, under affine reparameterizations
of the dual coordinates, the convergence rates of GD in η and θ coordinates can be scaled to 2c and 2 c , respectively, for any c > 0, while NGD maintains a fixed convergence rate of
2, remaining invariant to such transformations and sandwiched between them. Although this suggests that NGD may not exhibit uniformly superior convergence in continuous time,
we demonstrate that its advantages become pronounced in discrete time, where it achieves faster convergence and greater robustness to noise, outperforming GD. Our analysis hinges on bounding the spectrum and condition number of the Hessian of the KL divergence at the optimum, which coincides with the Fisher information matrix.
DDC Class
600: Technology
Publication version
publishedVersion
Loading...
Name
4787_Convergence_Properties_of.pdf
Type
Main Article
Size
1016.62 KB
Format
Adobe PDF