Massachusetts Undergraduate Research Conference (MassURC)

We introduce a class of mathematically justified regularizers for training score-based generative models (SGM) based on its connections to mean-field games and Hamilton-Jacobi-Bellman (HJB) equations. Recently it has been shown that score-based generative models can be formulated in terms of a mean-field game (MFG). This MFG formulation admits alternate characterizations of the SGM based on its optimality conditions, which are a set of coupled nonlinear partial differential equations (PDE). The first PDE is a controlled Fokker-Planck equation which is equivalent to the denoising process in SGM, while the second PDE is a HJB equation that characterizes the optimal controller, whose solution is related to the score function. Based on this mathematically rigorous connection, we propose a new objective function for the score function, and develop a PDE-based theory for explaining and understanding the role of latent space in SGM. In addition to the implicit score-matching objective, we include two regularizers, the first based on measuring the discrepancy in the HJB equation, the second based on measuring discrepancy in satisfying the terminal condition. The terminal condition is determined by the training data. By including these regularizers, we are informing the structure of the score function, thereby constraining the search space. This strategy is well-grounded through the theory of mean-field games and HJB equations. Through experiments we will show that the HJB regularizer helps learn the score function in a more stable way, while the terminal condition regularizer is associated with higher quality samples.

Research Area	Presenter	Title	Keywords
Probability, Statistics, and Machine Learning	Waghe, Shreyas	Efficient Convex Optimization in Fair Machine Learning	Fair Machine Learning (0.758621), Machine Learning (0.846154)
Engineering	Liousas, Demetri	Predicting Timescales in Dynamical Systems with Explainable Machine Learning Models	machine learning
Computer Science	Thornton, Peter David	Advancing Reasoning Abilities in AI through Local Open Source Methods	machine learning
Computer Science	Berduo, Alan Jesse	PC Building Assistant	Machine learning
Engineering	Li, Agnes	Using Physics Informed Machine Learning to Bridge the Knowledge-Data Gap for Pavement Modeling	Machine learning

Research Area

Presenter

Title

Keywords

Probability, Statistics, and Machine Learning

Waghe, Shreyas

Efficient Convex Optimization in Fair Machine Learning

Fair Machine Learning (0.758621), Machine Learning (0.846154)

Engineering

Liousas, Demetri

Predicting Timescales in Dynamical Systems with Explainable Machine Learning Models

machine learning