Ay, NihatNihatAyOostrum, Jesse vanJesse vanOostrumDatar, AdwaitAdwaitDatar2026-02-172026-02-172025-09Journal of Machine Learning Research 26: 1-37 (2025)https://hdl.handle.net/11420/61564This article studies the Fisher-Rao gradient, also referred to as the natural gradient, of the evidence lower bound (ELBO) which plays a central role in generative machine learning. It reveals that the gap between the evidence and its lower bound, the ELBO, has essentially a vanishing natural gradient within unconstrained optimization. As a result, maximization of the ELBO is equivalent to minimization of the Kullback-Leibler divergence from a target distribution, the primary objective function of learning. Building on this insight, we derive a condition under which this equivalence persists even when optimization is constrained to a model. This condition yields a geometric characterization, which we formalize through the notion of a cylindrical model .en1533-79282025137Microtome Publishinghttps://creativecommons.org/licenses/by/4.0/Evidence lower boundvariational gapnatural gradientinformation geometryvariational inferenceComputer Science, Information and General Works::006: Special computer methods::006.3: Artificial IntelligenceNatural Sciences and Mathematics::519: Applied Mathematics, ProbabilitiesJournal Articlehttps://doi.org/10.15480/882.16720https://jmlr.org/papers/v26/24-0606.html10.15480/882.16720Journal Article