MEAN-SQUARE ERRORS OF ESTIMATORS:
Mit Opencourseware, Information Inequalities
Suppose we have a parametric family of probability distributions with a likelihood function f(x, θ) for one observation, where f(x, θ) is a probability mass function for a discrete distribution or a probability density function for a continuous distribution. Let Eθ denote expectation, and Pθ probability, when θ is the true value of the parameter. Let X = (X1,..., Xn) be a vector of i.i.d. observations with distribution Pθ. Suppose g = g(θ) is a real-valued function of the parameter θ. One criterion for choosing an estimator T = T(X) of g(θ) is to minimize the mean-squared error (MSE) Eθ((T(X) − g(θ)) 2). Recall that T is called an unbiased estimator of g(θ) if EθT(X) = g(θ) for all θ. More generally, the bias of T as an estimator of g(θ) is defined by bT(θ): = bT,g(θ): = EθT − g(θ) for all θ. Thus T is unbiased as an estimator of g(θ) if and only if bT(θ) = 0 for all θ. If Eθ(T 2)…