Bayesian Ying-Yang System and Theory as a Unified Statistical Learning Approach (I): for Unsupervised and Semi-Unsupervised Learning
For Unsupervised, Semi-unsupervised Learning, Lei Xu +3 more
. A unified statistical learning approach called Bayesian YingYang (BYY) system and theory has been developed by the present author in recent years. This paper is the first part of a recent effort on systematically summarizing this theory. In this paper, we show how the theory functions as a general theory for unsupervised learning and its semi-unsupervised extension on parameter learning, regularization, structural scale or complexity selection, architecture design and data sampling. Specifically, it is shown how the general theory provides new theories for unsupervised pattern recognition and clustering analysis, factorial encoding, data dimension reduction, and independent component analysis, such that not only several existing popular unsupervised learning approaches, (e.g., finite mixture with the EM algorithm, K-means clustering algorithm, Helmholtz machine, principal component…