Efficient Learning of Sparse Distributed Feature Representations

Informative image representations are important in achieving
state-of-the-art performance in object recognition tasks. Among
feature learning algorithms that are used to develop image
representations, restricted Boltzmann machines (RBMs) have good
expressive power and build effective representations. However, the
difficulty of training RBMs has been a barrier to their wide use. To
address this difficulty, we show the connections between mixture models and RBMs and present an efficient training method for RBMs that utilize these connections. To the best of our knowledge, this is the first work showing that RBMs can be trained with almost no hyperparameter tuning to provide classification performance similar to or significantly better than mixture models (e.g., Gaussian mixture models). Along with this efficient training, we evaluate the importance of convolutional training that can capture a larger spatial context with less redundancy, as compared to nonconvolutional training. Overall, our method achieves stateof-the-art performance on both Caltech 101 / 256 datasets using a single type of feature.