Deflation Methods for Sparse PCA

TitleDeflation Methods for Sparse PCA
Publication TypeConference Proceedings
Year of Conference2008
AuthorsMackey, Lester
Conference NameNeural Information Processing Systems (NIPS'08)
Date Published12/2008
Conference LocationVancouver, Canada

In analogy to the PCA setting, the sparse PCA problem is often solved by iter-
atively alternating between two subtasks: cardinality-constrained rank-one vari-
ance maximization and matrix deflation. While the former has received a great
deal of attention in the literature, the latter is seldom analyzed and is typically
borrowed without justification from the PCA context. In this work, we demon-
strate that the standard PCA deflation procedure is seldom appropriate for the
sparse PCA setting. To rectify the situation, we first develop several deflation al-
ternatives better suited to the cardinality-constrained context. We then reformulate
the sparse PCA optimization problem to explicitly reflect the maximum additional
variance objective on each round. The result is a generalized deflation procedure
that typically outperforms more standard techniques on real-world datasets.