Ay, NihatNihatAyvan Oostrum, JesseJessevan OostrumDatar, AdwaitAdwaitDatar2026-03-132026-03-132025Journal of Machine Learning Research 26: 222 (2025)https://hdl.handle.net/11420/62115This article studies the Fisher-Rao gradient, also referred to as the natural gradient, of the evidence lower bound (ELBO) which plays a central role in generative machine learning. It reveals that the gap between the evidence and its lower bound, the ELBO, has essentially a vanishing natural gradient within unconstrained optimization. As a result, maximization of the ELBO is equivalent to minimization of the Kullback-Leibler divergence from a target distribution, the primary objective function of learning. Building on this insight, we derive a condition under which this equivalence persists even when optimization is constrained to a model. This condition yields a geometric characterization, which we formalize through the notion of a cylindrical model.en1533-79282025Evidence lower boundinformation geometrynatural gradientvariational gapvariational inferenceTechnology::600: TechnologyJournal ArticleJournal Article