Dimensionality Reduction for Representing the Knowledge of Probabilistic Models
May 1, 2019 / GlobalAbstract
Most deep learning models rely on expressive high-dimensional representations to achieve good performance on tasks such as classification. However, the high dimensionality of these representations makes them difficult to interpret and prone to over-fitting. We propose a simple, intuitive and scalable dimension reduction framework that takes into account the soft probabilistic interpretation of standard deep models for classification. When applying our framework to visualization, our representations more accurately reflect inter-class distances than standard visualization techniques such as t-SNE. We show experimentally that our framework improves generalization performance to unseen categories in zero-shot learning. We also provide a finite sample error upper bound guarantee for the method.
Authors
Marc T Law, Jake Snell, Amir-massoud Farahmand, Raquel Urtasun, Richard S. Zemel
Conference
ICLR 2019
Full Paper
Uber ATG
