PAC-Bayes Learning of Conjunctions and Classification of Gene-Expression Data

Abstract

We propose a “soft greedy ” learning algorithm for building small conjunctions of simple threshold functions, called rays, defined on single real-valued attributes. We also propose a PAC-Bayes risk bound which is minimized for classifiers achieving a non-trivial tradeoff between sparsity (the number of rays used) and the mag-nitude of the separating margin of each ray. Finally, we test the soft greedy algorithm on four DNA micro-array data sets. 1