A Generalized Maximum Entropy Approach to Bregman Co-clustering and Matrix Approximations

Arindam Banerjee, Inderjit Dhillon, Joydeep Ghosh, Srujana Merugu

Abstract:   An important task in unsupervised learning is maximum likelihood mixture estimation (MLME) for exponential families. In this paper, we prove a mathematical equivalence between this MLME problem and the rate distortion problem for Bregman divergences. We also present new theoretical results in rate distortion theory for Bregman divergences. Further, an analysis of the problems as a trade-off between compression and preservation of information is presented that yields the information bottleneck method as an interesting special case.

Download: pdf

Citation

  • A Generalized Maximum Entropy Approach to Bregman Co-clustering and Matrix Approximations (pdf)
    A. Banerjee, I. Dhillon, J. Ghosh, S. Merugu.
    In International Conference on Machine Learning (ICML), pp. 57-64, July 2004.

    Bibtex: