Customizing Bayesian multivariate generalizability theory to mixed-format tests.

Mixed-format tests, which typically include dichotomous items and polytomously scored tasks, are employed to assess a wider range of knowledge and skills. Recent behavioral and educational studies have highlighted their practical importance and methodological developments, particularly within the context of multivariate generalizability theory. However, the diverse response types and complex designs of these tests pose significant analytical challenges when modeling data simultaneously.

Many nonnormalities, one simulation: Do different data generation algorithms affect study results?

Monte Carlo simulation studies are among the primary scientific outputs contributed by methodologists, guiding application of various statistical tools in practice. Although methodological researchers routinely extend simulation study findings through follow-up work, few studies are ever replicated. Simulation studies are susceptible to factors that can contribute to replicability failures, however.

Evaluating Close Fit in Ordinal Factor Analysis Models With Multiply Imputed Data.

Multiple imputation (MI) is one of the recommended techniques for handling missing data in ordinal factor analysis models. However, methods for computing MI-based fit indices under ordinal factor analysis models have yet to be developed. In this short note, we introduced the methods of using the standardized root mean squared residual (SRMR) and the root mean square error of approximation (RMSEA) to assess the fit of ordinal factor analysis models with multiply imputed data.

One step at a time: A statistical approach for distinguishing mediators, confounders, and colliders using direction dependence analysis.

In observational data, understanding the causal link when estimating the causal effect of an independent variable () on a dependent variable () often requires researchers to identify the role of a third variable in the → relationship. Mediation, confounding, and colliding are three key third-variable effects that yield different theoretical and methodological implications for drawing causal conclusions. Commonly used covariance-based statistical methods, such as linear regression and structural equation modeling, cannot distinguish these effects in practice, however.

Detecting heterogeneity in the causal direction of dependence: A model-based recursive partitioning approach.

Methods of causal discovery and direction of dependence to evaluate causal properties of variable relations have experienced rapid development. The majority of causal discovery methods, however, relies on the assumption of causal effect homogeneity, that is, the identified causal structure is expected to hold for the entire population. Because causal mechanisms can vary across subpopulations, we propose combining methods of model-based recursive partitioning and non-Gaussian causal discovery to identify such subpopulations.

The NEAT Equating Via Chaining Random Forests in the Context of Small Sample Sizes: A Machine-Learning Method.

The part of responses that is absent in the nonequivalent groups with anchor test (NEAT) design can be managed to a planned missing scenario. In the context of small sample sizes, we present a machine learning (ML)-based imputation technique called chaining random forests (CRF) to perform equating tasks within the NEAT design. Specifically, seven CRF-based imputation equating methods are proposed based on different data augmentation methods.

Is the Area Under Curve Appropriate for Evaluating the Fit of Psychometric Models?

In the literature of modern psychometric modeling, mostly related to item response theory (IRT), the fit of model is evaluated through known indices, such as χ, M2, and root mean square error of approximation (RMSEA) for absolute assessments as well as Akaike information criterion (AIC), consistent AIC (CAIC), and Bayesian information criterion (BIC) for relative comparisons. Recent developments show a merging trend of psychometric and machine learnings, yet there remains a gap in the model fit evaluation, specifically the use of the area under curve (AUC). This study focuses on the behaviors of AUC in fitting IRT models.

Factor Structure of the Teacher Strengths and Difficulties Questionnaire in a Large Community-Based Sample: An Investigation of Alternative Measurement Models.

The Strengths and Difficulties Questionnaire (SDQ) is a screening measure commonly used to assess behavioral and emotional symptoms and strengths among children and adolescents. However, despite its frequent use, its underlying factor structure remains an important area of inquiry. Whereas the original five-factor structure has often been supported through exploratory factor analysis, results from confirmatory analyses continue to yield mixed results.

Applying Negative Binomial Distribution in Diagnostic Classification Models for Analyzing Count Data.

Diagnostic classification models (DCMs) have been used to classify examinees into groups based on their possession status of a set of latent traits. In addition to traditional item-based scoring approaches, examinees may be scored based on their completion of a series of small and similar tasks. Those scores are usually considered as count variables.

The Pull-to-Sit Task: Examining Infant Postural Development in Autism Spectrum Disorder.

Objective: To evaluate the predictive relationship between early trajectories of postural and head control during a pull-to-sit task and later autism diagnostic and developmental outcomes.

Study Design: Using a prospective longitudinal design, postural skills of 100 infants at elevated and low familial likelihood of autism spectrum disorder (ASD) were evaluated using a pull-to-sit task monthly from age 1 month to 6 months. At age 24 months, infants were seen for a developmental and diagnostic evaluation completed by examiners masked to participant group.

Diagnostic Classification Models for a Mixture of Ordered and Non-ordered Response Options in Rating Scales.

When developing ordinal rating scales, we may include potentially unordered response options such as "Neither Agree nor Disagree," "Neutral," "Don't Know," "No Opinion," or "Hard to Say." To handle responses to a mixture of ordered and unordered options, Huggins-Manley et al. (2018) proposed a class of semi-ordered models under the unidimensional item response theory framework.

A Monte Carlo Study of Confidence Interval Methods for Generalizability Coefficient.

Computing confidence intervals around generalizability coefficients has long been a challenging task in generalizability theory. This is a serious practical problem because generalizability coefficients are often computed from designs where some facets have small sample sizes, and researchers have little guide regarding the trustworthiness of the coefficients. As generalizability theory can be framed to a linear mixed-effect model (LMM), bootstrap and simulation techniques from LMM paradigm can be used to construct the confidence intervals.

Teacher-student relationships and adolescents' school satisfaction: Behavioural engagement as a mechanism of change.

Background: Teacher-student relationships have been linked to various aspects of students' school functioning, including social-emotional well-being in school, but the underlying mechanisms need more investigation.

Aims: In this study, we analysed longitudinal data to test if students' classroom behavioural engagement was a potential mechanism of change that explained how teacher-student relationships affect student school satisfaction.

Sample: We used an archival dataset with a sample of seventh graders (ages 11-14, M  = 12.

Examining the validity and measurement invariance across gender and race of the Implicit Thoughts, Emotion, and Behavior Questionnaire.

Implicit theory has been relatively well-studied in the areas of intelligence and personality but remains less investigated in mental health. This article aims to analyze the psychometric properties of the Implicit Thoughts, Emotion, and Behavior Questionnaire (ITEB-Q; Schleider & Weisz, 2016a). We tested its factorial validity, measurement invariance across gender and two racial groups, as well as criterion validity in a large, diverse sample of adolescents.

Poisson Diagnostic Classification Models: A Framework and an Exploratory Example.

Assessments with a large amount of small, similar, or often repetitive tasks are being used in educational, neurocognitive, and psychological contexts. For example, respondents are asked to recognize numbers or letters from a large pool of those and the number of correct answers is a count variable. In 1960, George Rasch developed the Rasch Poisson counts model (RPCM) to handle that type of assessment.

Agreement Between Parental Report and Clinician Observation of Infant Developmental Skills.

Understanding the convergence between parent report and clinician observation measures of development is important and became even more critical during the COVID-19 pandemic as clinician contact with families was significantly limited. Previous research points to inconsistencies in the degree of agreement between parents and clinicians and very little research has examined these associations for infants within the first year of life. This study investigated the association between parent report and clinician observation measures of social communication and motor skills in 27 young infants who were assessed at 9 and 12 months of age.

Neonatal neurobehavior in infants with autism spectrum disorder.

Aim: To investigate neurobehavioral maturation for neonates who are later diagnosed with autism spectrum disorder (ASD).

Method: In a prospective longitudinal design, neonatal neurobehavior was examined monthly in 1- to 3-month-old infants at elevated and low familial likelihood of ASD (n=60). At 2 years, infants were seen for a clinical best-estimate evaluation, resulting in 18 infants with ASD and 36 typically developing infants.

A Short Note on Optimizing Cost-Generalizability via a Machine-Learning Approach.

The costs of an objective structured clinical examination (OSCE) are of concern to health profession educators globally. As OSCEs are usually designed under generalizability theory (G-theory) framework, this article proposes a machine-learning-based approach to optimize the costs, while maintaining the minimum required generalizability coefficient, a reliability-like index in G-theory. The authors adopted G-theory parameters yielded from an OSCE hosted by a medical school, reproduced the generalizability coefficients to prepare for optimizing manipulations, applied simulated annealing algorithm to calculate the number of facet levels minimizing the associated costs, and conducted the analysis in various conditions via computer simulation.

Temperamental Shyness and Anger/Frustration in Childhood: Normative Development, Individual Differences, and the Impacts of Maternal Intrusiveness and Frontal Electroencephalogram Asymmetry.

This study used latent growth curve modeling to identify normative development and individual differences in the developmental patterns of shyness and anger/frustration across childhood. This study also examined the impacts of maternal intrusiveness and frontal electroencephalogram (EEG) asymmetry at age 4 on the developmental patterns of shyness and anger/frustration. 180 children (92 boys, 88 girls; M  = 4.

Fitting Latent Growth Models with Small Sample Sizes and Non-normal Missing Data.

This study investigates the performance of robust ML estimators when fitting and evaluating small sample latent growth models (LGM) with non-normal missing data. Results showed that the robust ML methods could be used to account for non-normality even when the sample size is very small (e.g.

Evaluating SEM Model Fit with Small Degrees of Freedom.

Research has revealed that the performance of root mean square error of approximation (RMSEA) in assessing structural equation models with small degrees of freedom () is suboptimal, often resulting in the rejection of correctly specified or closely fitted models. This study investigates the performance of standardized root mean square residual (SRMR) and comparative fit index (CFI) in small models with various levels of factor loadings, sample sizes, and model misspecifications. We find that, in comparison with RMSEA, population SRMR and CFI are less susceptible to the effects of .

A comparison of full information maximum likelihood and multiple imputation in structural equation modeling with missing data.

This article compares two missing data procedures, full information maximum likelihood (FIML) and multiple imputation (MI), to investigate their relative performances in relation to the results from analyses of the original complete data or the hypothetical data available before missingness occurred. By expressing the FIML estimator as a special MI estimator, we predicted the expected patterns of discrepancy between the two estimators. Via Monte Carlo simulation studies where we have access to the original complete data, we compare the performance of FIML and MI estimators to that of the complete data maximum likelihood (ML) estimator under a wide range of conditions, including differences in sample size, percent of missingness, and degrees of model misfit.

[Formula: see text] Social-environmental moderators of neurodevelopmental outcomes in youth born preterm: A systematic review.

: Preterm birth represents a significant medical event that places infants at a markedly greater risk for neurodevelopmental problems and delays. Although the impact of medical factors on neurodevelopment for those born preterm has been thoroughly explored, less is known about how social-environmental factors (e.g.