Recently, Yoshua Bengio, one of the world's top experts in the field of AI and the winner of the Turing Award , made a connection between the Generative Flow Network (GFlowNet) and the deep generation model. introduce.
GFlowNet is a new network generation method proposed by Bengio, involving "reinforcement learning, deep generation modeling and energy-based probability modeling", which is also related to variational models and inference.
Bengio mentioned on his personal website that he is rarely so passionate about new research directions, one of which is GFlowNet.
Figure | Yoshua Bengio (Source: Bengio Personal Website)
This research paper was "Unifying Generative Models with GFlowNets" on September 6 (Unifying Generative Models with GFlowNets) Submit the title on arXiv .
He is considered one of the few people who promoted the development of deep learning in the 1990s and 2000s, and in 2022 he became the world's highest H-index computer scientist. (Source: Bengio Personal Website)
First, give a brief introduction to Bengio. He is a professor in the Department of Computer and Operations Research at the University of Montreal, Canada and the founder and director of science at the Mira Quebec Institute of Artificial Intelligence.
In 2018, due to pioneering work and important contributions in deep learning, Bengio, professor of Computer Science at the University of Toronto, Canada, and Likun Yang, Vice President and Chief AI Scientist at Meta (Yann LeCun) together won the Turing Award (often known as the "Nobel Prize in Computing") from the International Computer Society. The three of them are sometimes called the " artificial intelligence godfather" and the "deep learning godfather".
It is understood that Bengio received his PhD in Computer Science from McGill University in Canada in 1991, and then worked as a postdoctoral fellow at MIT and AT&T Bell Laboratory . Since joining University of Montreal in 1993, he has been to the present. He has authored books and papers such as "Deep Learning (Adaptive Computation and Machine Learning)" and "Towards Biologically Plausible Deep Learning".
2021, Bengio published an important paper about GFlowNet "GFlowNet Foundations" (GFlowNet Foundations) as a writer.
At present, GFlowNet has been introduced into an active learning environment to sample various candidate sets. It also provides a new perspective on nonparametric Bayesian modeling and supervised learning of abstract representations. "It is trained to make them sample approximately in proportion to a given reward function," the paper mentioned.
In addition to understanding the explanatory causal factors and related mechanisms, GFlowNet is especially helpful in implementing system inductive bias. GFlowNet is also a new and difficult field of research, and in order to understand and apply it, appropriate optimization algorithms are still developing rapidly. Its concept is gradually expanding.
. In this study, the paper mentioned: "There are many frameworks for deep generation modeling, and each framework has its own specific training algorithms and inference methods. We use the perspective of Markov trajectory learning to generate the deep generation model for deep generation through the perspective of Markov trajectory learning. The connection with the GFlowNet framework gives a unified view. This provides a method for unified training and inference algorithms and provides a path for building generative model aggregation. "
From the perspective of probability modeling, GFlowNet is a generative model whose purpose is to sample x according to the proportion of the given reward function R(x).
Specifically, a GFlowNet will sample a Markov trajectory τ=(S0, S1,…, Sn) of length n. If not specified, the symbol X=Sn is used to indicate the final state of the track.
This process has a natural connection with reinforcement learning. All states s construct a directed non-cyclic graph in the latent state space. Each track starts with the same (abstract) initial state S0 and runs to a different endpoint Sn. Ideally, the traffic to x is expected to be equal to the given reward.
In the paper's "Learning reward functions from data" section, the research team mentioned: "Energy-based model (EBM, Energy-based model) can be used as a (negative logarithmic) reward function trained by GFlowNet. We can use any GFlowNet modeling, and the two models (EBM and GFlowNet) are trained together. "
In addition, the generative adversarial network (GAN, Generative Adversarial Network) is closely related to EBM, but its algorithm is more computationally efficient. However, while it may be reasonable for the first time, it is not possible to use the discriminator D(x) directly as a reward for GFlowNet training.
If so, at the end of a perfect training, an optimal discriminator and optimal GFlowNet generator distribution will be obtained. To fill this gap, Bengio designed some more meaningful algorithms.
The above figure illustrates why the word "flow" is used in GFlowNet. This takes into account the flow of denormalized probability, similar to the amount of water flowing out from the initial state (0 on the left) in a directed acyclic graph (maybe exponential, and does not need to be explicitly stated in the computer), with its trajectory corresponding to All possible sequences of actions (i.e. actions that determine state transitions) in order to construct complex objects, such as molecular graphs, causal graphs, interpretations of scenes or ideas in our minds. The final conclusion of the paper
The final conclusion mentioned: "Today's generative models can be understood as GFlowNet with differentiated strategies on sample trajectories. This is the overlapping part between existing generative modeling frameworks and the general algorithms that train them Relationship provides some perspectives.
This unification means a method to build generative modeling of different types of clusters, and due to the superiority of inference and training, GFlowNet can be used as a general purpose adhesive in it. "
Reference :
https://arxiv.org/abs/2209.02606
https://yoshuabengio.org/2022/03/05/generative-flow-networks/
https://en.wikipedia.org/wiki/Yoshua_Bengio
