General Comments
Definitions Generative, Gammah Ff =k B T, Unsupervised Learning
> c we must encode this value as a bit string. : : : Shannon's coding theorem implies that x can be communicated at a cost that is bounded below by \\Gamma log p(x) bits. 1 Moreover, by using block coding techniques we can get arbitrarily close to this bound : : : For coding real values to within a quantization width of t, the cost of coding the value x is then \\Gamma log t + 0:5 log 2ßoe 2 + x 2 =2oe 2 . The simple method of communicating the training vectors is generally very wastful so the data is first converted to some other representation before communicating it. The essence of the MDL principle is that the best model of the data is the one that minimizes the total number of bits required to communicate