Description
Variational Monte Carlo is a powerful approach to tackle the exponential complexity of quantum many body physics. Taking the variational principle in quantum physics as a starting point, VMC transforms the problem of finding the ground state from an eigenvalue problem to an optimization problem. Then, it relies on two key elements to solve this optimization problem efficiently: an ansatz, i.e a parametric family of states, to reduce the search space and Monte Carlo sampling to efficiently estimate the objective function and other related quantities like its gradient. Regarding the ansatz, neural networks have become one of the most popular candidates due to multiple reasons: their expressivity, their polynomial scaling with the number of particles, their capacity to integrate informative priors in the form of inductive biases, their ability to be optimized using gradient based methods and automatic differentiation, and the fact that they leverage the advances of a rapidly advancing field like deep learning. On the other hand, regarding the optimization methods, stochastic reconfiguration, i.e. natural gradient descent, and its variants, like minSR or SPRING, have become the standard approach to navigate the complex energy landscape of quantum systems. However, the computational cost of these second order methods is still an important bottleneck. In this work, we make explicit the connection of NQS with Reinforcement Learning and take advantage of it to leverage advances from this field to overcome the limitations of SR. In particular, we introduce a new first order optimization method for VMC that can outperform SR both in terms of computational cost and quality of the final solution across different scenarios, allowing a massive scaling of architectures and systems.