Nos partenaires

CNRS

Rechercher





Accueil du site > Français > Evénements > Séminaires

Séminaires

 

L’IRIT étant localisé sur plusieurs sites, ses séminaires sont organisés et ont lieu soit à l’Université Toulouse 3 Paul Sabatier (UT3), l’Université Toulouse 1 Capitole (UT1), l’INP-ENSEEIHT ou l’Université Toulouse 2 Jean Jaurès (UT2J).

 

ARM: Augment-REINFORCE-merge gradient for discrete latent variable models

Mingyuan ZHOU - University of Texas at Austin (Etats-Unis)

Jeudi 5 Juillet 2018, 16h30 - 17h30
INP-ENSEEIHT, Salle F218
Version PDF :

Abstract

To propagate the gradients through discrete stochastic layers, we encode the true gradients into zeros combined with spikes, which are distributed over a random subset of iterations and amenable to backpropagation. To modulate the frequencies, amplitudes, and signs of the spikes to capture the temporal evolution of the true gradients, we propose the augment-REINFORCE-merge (ARM) estimator, which combines data augmentation, the score-function estimator, and variance reduction for Monte Carlo integration using common random numbers. The ARM estimator provides low-variance and unbiased gradient estimates for the parameters of discrete distributions, leading to state-of-the-art performance in both auto-encoding variational Bayes and maximum likelihood inference, for discrete latent variable models with one or multiple discrete stochastic layers.

 

Retour