Rater agreement and the generalized Rudas-Clogg-Lindsay index of fit.

Rudas, Clogg and Lindsay proposed a new index of fit for contingency table analysis. Using the two-component mixture, where the first component with weight (1-w) represents the model to be tested and the second component with weight w is unstructured, the RCL index of lack of fit was defined to be the smallest mixing weight w(*) being compatible with the two-component mixture to be saturated. This index of fit, which is not sensitive to sample size, is applied to the problem of assessing agreement between two raters whereby three hypotheses (pure agreement, quasi-independence, independence) are considered. As quasi-independence comprises the two other hypotheses, a natural generalization of the RCL index of fit results from assuming the model itself to be composed of two submodels (pure agreement, independence) while the third component remains unstructured. Two examples demonstrate the application of this generalized RCL index of fit, with the first 3x3 contingency table having no empty cells, and the second 4x4 table having five of them. In both cases estimating the parameters and determining w(*) was possible without problems within the linear logistic framework for latent class analysis. A further analysis of the 4x4 table assumes the model to be the two-class latent class model that certainly does not belong to the family of standard models for contingency table analysis. Thus, in contrast to the recommendations given originally, the following conclusions seem to be justified: (i) the number of components representing the model may exceed one (generalized RCL index); (ii) the application of the original RCL index of fit may be extended to more complex models; and (iii) empty cells bear no problems so that this approach may be recommended in the case of both large and small sample sizes.