Mathematics · Statistics and Probability
An Illustration of the Least Median Squares (LMS) Regression Using PROGRESS
See on the mapAbstract
Classical regression analyses were based on least mean squares (DS) methods which minimized sum of residual squares in a linear regression (Casella & Berger, 1990). Weisberg (1985) noted, One main qualification of least squares estimation is that it has been used successfully for over 150 years (P. 251). Meanwhile, many researchers expressed concerns regarding sensitivity of LS estimators to in real analyses (e.g., Birkes & Dodge, 1993; Carroll & Ruppert, 1988; Montgomery & Peck, 1982; Rawlings, 1988). Rousseeuw and Leroy (1987) cautioned: Outliers occur very frequently in real data, and they often go unnoticed because nowadays much is processed by computers, without careful inspection or screening (p. vii). Although some researchers suggested that Outlier detection procedures should be considered before any formal testing is done (Cook & Weisberg, 1982, p. 2), they also acknowledged that If a set of has more than one outlier, cases may mask each other, making finding difficult (Weisberg, 1985, p. 117). Alternatively, Weisberg (1985) asserted, we can think of using statistical methods that can tolerate or accommodate some proportion of bad or outlying data (p. 116). Birkes and Dodge (1993) suggested that LMS [Least Median Squares] estimate is simple to describe and is very robust against outliers (P. 207). Thus far, LMS approach has been developed in a computer software entitled Program for RObust reGRESSion (PROGRESS), and according to Rousseeuw (1984), resulting estimator can resist effect of nearly 50% of contamination in data (p. 871). This development was identified a frontier in statistics (Carroll & Ruppert, 1988), and as a result, PROGRESS has been integrated in workstation package S-PLUS of Statistical Sciences (Rousseeuw & Leroy, 1987). However, few researchers in educational research community are aware of Least Median Squares (LMS) method. The purpose of this study is to illustrate some advantages of LMS regression through empirical analyses. Literature Review The LS method produced best linear unbiased estimators (BLUE) under normal error distribution (Birkes & Dodge, 1993). For real not meeting normality assumption, LS may not be optimal. In particular, a single outlier in a set can have profound impact on LS estimates (Weisberg, 1985). Chatterjee and Hadi (1988) reviewed: Several procedures exist for detection of a single outlier in linear regression. These procedures usually assume that there is at most one outlier in a given set and require that label of outlying observation is unknown. (p. 80) To remedy contamination in larger proportions, robust approaches were developed to fit a regression that does justice to majority of data (Rousseeuw & Leroy, 1987, p. vii). Birkes and Dodge (1993) elaborated: The robustness of an estimate against heavier contamination is measured by it breakdown point, which is least proportion of that can occur in a sample without entailing possibility of arbitrarily large bias. (P. 207) In LS estimation, single point far removed from other points can have almost as much influence on regression results as all other points combined (Rawlings, 1988, p. 241). Thus, LS estimate can be seriously disturbed by contamination because of zero breakdown point in LS modeling (Rousseeuw & Leroy, 1987). A higher breakdown point in robust regression was an important feature ameliorating weakness in outlier diagnostics. Cook and Weisberg (1982) acknowledged, the use of robust methods does not abrogate usefulness of diagnostics in general, although it may render certain of them unnecessary (p. …