Examining the Dynamic of Clustering Effects in Multilevel Designs: A Latent Variable Method Application.

This note is concerned with the study of temporal development in several indices reflecting clustering effects in multilevel designs that are frequently utilized in educational and behavioral research. A latent variable method-based approach is outlined, which can be used to point and interval estimate the growth or decline in important functions of level-specific variances in two-level and three-level settings. The procedure may also be employed for the purpose of examining stability over time in clustering effects.

Comparison of Multiple-Choice Question Formats in a First Year Medical Physiology Course.

The purpose of this study was to compare student performance and question discrimination of multiple-choice questions (MCQs) that followed a standard format (SF) versus those that do not follow a SF, termed here as non-standard format (NSF). Medical physiology exam results of approximately 500 first-year medical students collected over a five-year period (2020-2024) were used. Classical test theory item analysis indices, e.

A Note on Comparing the Bifactor and Second-Order Factor Models: Is the Bayesian Information Criterion a Routinely Dependable Index for Model Selection?

This note demonstrates that the widely used Bayesian Information Criterion (BIC) need not be generally viewed as a routinely dependable index for model selection when the bifactor and second-order factor models are examined as rival means for data description and explanation. To this end, we use an empirically relevant setting with multidimensional measuring instrument components, where the bifactor model is found consistently inferior to the second-order model in terms of the BIC even though the data on a large number of replications at different sample sizes were generated following the bifactor model. We therefore caution researchers that routine reliance on the BIC for the purpose of discriminating between these two widely used models may not always lead to correct decisions with respect to model choice.

Comparing Informants for Mental Health Screening at the Preschool Level.

Universal screening for mental health in preschools provides the opportunity for early identification and early intervention, but guidance regarding which informants to use is needed. Preschoolers' ( = 535) parent and teacher reports across two screening forms were analyzed to determine similarities and discrepancies for classification results and screener scores. The analyses also examined if an additional rater provided incrementally valid information to the prediction of longitudinal kindergarten outcomes.

A randomized controlled trial on the interconnected systems framework for school mental health and PBIS: Focus on proximal variables and school discipline.

This study reviews findings for the first randomized controlled trial (RCT) on the Interconnected Systems Framework (ISF) for school mental health (SMH) and Positive Behavioral Interventions and Supports (PBIS). Since its development in the late 2000s, the ISF has been supported by federally funded centers for SMH and PBIS, and, guided by a national workgroup, is being implemented in >50 communities in the United States. This experimental evaluation of the ISF involved an RCT implemented in 24 schools in two southeastern states, with the ISF implemented in eight schools, PBIS alone implemented in eight schools, and typically co-located PBIS+SMH implemented in eight schools.

Design Effect in Multilevel Settings: A Commentary on a Latent Variable Modeling Procedure for Its Evaluation.

A latent variable modeling-based procedure is discussed that permits to readily point and interval estimate the design effect index in multilevel settings using widely circulated software. The method provides useful information about the relationship of important parameter standard errors when accounting for clustering effects relative to conducting single-level analyses. The approach can also be employed as an addendum to point and interval estimation of the intraclass correlation coefficient in empirical research.

Moving towards Implementation of Universal Mental Health Screening by Examining Attitudes toward School-based Practices.

Universal mental health screening is a proactive approach to identify students who may benefit from prevention or early intervention services. Despite known benefits, few schools are engaging in screening efforts and it is critical to examine factors that may impede or enhance implementation. Following implementation of a universal screening program across five preschools and elementary schools, this study investigated the attitudes of teachers ( = 40) and parents ( = 330) and found strong agreement among stakeholders about the acceptability and appropriateness of universal mental health screening.

A Monte Carlo Study of Confidence Interval Methods for Generalizability Coefficient.

Computing confidence intervals around generalizability coefficients has long been a challenging task in generalizability theory. This is a serious practical problem because generalizability coefficients are often computed from designs where some facets have small sample sizes, and researchers have little guide regarding the trustworthiness of the coefficients. As generalizability theory can be framed to a linear mixed-effect model (LMM), bootstrap and simulation techniques from LMM paradigm can be used to construct the confidence intervals.

A Short Note on Optimizing Cost-Generalizability via a Machine-Learning Approach.

The costs of an objective structured clinical examination (OSCE) are of concern to health profession educators globally. As OSCEs are usually designed under generalizability theory (G-theory) framework, this article proposes a machine-learning-based approach to optimize the costs, while maintaining the minimum required generalizability coefficient, a reliability-like index in G-theory. The authors adopted G-theory parameters yielded from an OSCE hosted by a medical school, reproduced the generalizability coefficients to prepare for optimizing manipulations, applied simulated annealing algorithm to calculate the number of facet levels minimizing the associated costs, and conducted the analysis in various conditions via computer simulation.

Evaluating Restrictive Models in Educational and Behavioral Research: Local Misfit Overrides Model Tenability.

The frequent practice of overall fit evaluation for latent variable models in educational and behavioral research is reconsidered. It is argued that since overall plausibility does not imply local plausibility and is only necessary for the latter, local misfit should be considered a sufficient condition for model rejection, even in the case of omnibus model tenability. The argument is exemplified with a comparison of the widely used one-parameter and two-parameter logistic models.

Fitting Latent Growth Models with Small Sample Sizes and Non-normal Missing Data.

This study investigates the performance of robust ML estimators when fitting and evaluating small sample latent growth models (LGM) with non-normal missing data. Results showed that the robust ML methods could be used to account for non-normality even when the sample size is very small (e.g.

Evaluating SEM Model Fit with Small Degrees of Freedom.

Research has revealed that the performance of root mean square error of approximation (RMSEA) in assessing structural equation models with small degrees of freedom () is suboptimal, often resulting in the rejection of correctly specified or closely fitted models. This study investigates the performance of standardized root mean square residual (SRMR) and comparative fit index (CFI) in small models with various levels of factor loadings, sample sizes, and model misspecifications. We find that, in comparison with RMSEA, population SRMR and CFI are less susceptible to the effects of .

School and Teacher Factors That Promote Adolescents' Bystander Responses to Social Exclusion.

Schools may be one important context where adolescents learn and shape the behaviors necessary for promoting global inclusivity in adulthood. Given the importance of bystanders in halting bullying and peer aggression, the focus of this study is on both moral judgments regarding one type of bullying, social exclusion, and factors that are associated with bystander intervention. The study includes 896 adolescents, who were 6th ( = 450, = 11.

Using Integrative Data Analysis to Investigate School Climate Across Multiple Informants.

Survey research frequently involves the collection of data from multiple informants. Results, however, are usually analyzed by informant group, potentially ignoring important relationships across groups. When the same construct(s) are measured, integrative data analysis (IDA) allows pooling of data from multiple sources into one data set to examine information from multiple perspectives within the same analysis.

Using a linear mixed-effect model framework to estimate multivariate generalizability theory parameters in R.

Multivariate generalizability theory (mG-theory) is an important framework in many behavioral and educational studies, as it describes useful psychometric properties of multidimensional assessments. Nevertheless, the use of mG-theory estimation is limited due to the lack of available software for carrying out the necessary calculations: users rely heavily on independent software programs such as mGENOVA and the BUGS/JAGS suite of programs. Considering the prevalence of R software, this paper presents a solution using the glmmTMB package to accomplish the estimation task.

Fitting Large Factor Analysis Models With Ordinal Data.

A simulation study was conducted to investigate the model size effect when confirmatory factor analysis (CFA) models include many ordinal items. CFA models including between 15 and 120 ordinal items were analyzed with mean- and variance-adjusted weighted least squares to determine how varying sample size, number of ordered categories, and misspecification affect parameter estimates, standard errors of parameter estimates, and selected fit indices. As the number of items increased, the number of admissible solutions and accuracy of parameter estimates improved, even when models were misspecified.

Scaling the Variance of a Latent Variable While Assuring Constancy of the Model.

This paper investigates how the major outcome of a confirmatory factor investigation is preserved when scaling the variance of a latent variable by the various scaling methods. A constancy framework, based upon the underlying factor analysis formula that enables scaling by modifying components through scalar multiplication, is described; a proof is included to demonstrate the constancy property of the framework. It provides the basis for a scaling method that enables the comparison of the contribution of different latent variables of the same confirmatory factor model to observed scores, as for example, the contributions of trait and method latent variables.

The Relationship Between the Standardized Root Mean Square Residual and Model Misspecification in Factor Analysis Models.

We argue that the definition of close fitting models should embody the notion of substantially ignorable misspecifications (SIM). A SIM model is a misspecified model that might be selected, based on parsimony, over the true model should knowledge of the true model be available. Because in applications the true model (i.

Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling.

In this study, we introduce an interval estimation approach based on Bayesian structural equation modeling to evaluate factorial invariance. For each tested parameter, the size of noninvariance with an uncertainty interval (i.e.

School and Family Factors Predicting Adolescent Cognition Regarding Bystander Intervention in Response to Bullying and Victim Retaliation.

Youth aggression occurs at high rates. Aggressive acts can be curbed through bystander intervention; yet, little is known about school and family factors that predict bystander intervention in response to both aggression and victim retaliation. This research examines school and family factors related to standing up to aggression and intervening before possible retaliation occurs.

EASY-An Instrument for Surveillance of Physical Activity in Youth.

Purpose: Physical activity (PA) promotion among youth is a public health priority, and there is a need for robust surveillance systems to help support such initiatives. Existing youth PA self-report instruments that are used for surveillance lack information regarding the types and contexts of activity. Further, these instruments have limited validity with accelerometry.

Examining the BASC-3 BESS Parent Form-Preschool Using Rasch Methodology.

This study examined the Behavior Assessment System for Children-Third Edition Behavioral and Emotional Screening System, Parent Form-Preschool, which is used to identify children at risk for emotional/behavioral problems. Rasch rating scale methods were used to provide information about psychometric properties of items, parent raters, and the response scale. Using the norming sample of 459 children, we investigated the structure of the screener, performance of the scale, and coverage of the maladaptive behavior construct to identify children with emerging behavioral difficulties.