No packages match

mixedsubjectsirt - Item Response Theory Calibration with a Mixed Subjects Design

Integrates large language model generated item responses into psychometric calibration studies through a mixed-subjects design for unidimensional two-parameter and one-parameter logistic item response theory models. Human pilot responses are augmented with model-generated responses using a prediction-powered inference estimator (Angelopoulos, Bates, Fannjiang, Jordan and Zrnic (2023) <doi:10.1126/science.adi6000>; Angelopoulos, Duchi and Zrnic (2023) <doi:10.48550/arXiv.2311.01453>) adapted to marginal maximum-likelihood estimation, following the mixed-subjects design of Broska, Howes and van Loon (2025) <doi:10.1177/00491241251326865>. The estimator is anchored to the human responses and is asymptotically unbiased for the human item parameters at any tuning weight; the weight on the synthetic responses is chosen to minimize propagated ability-score risk, down-weighting uninformative or biased generated responses. Louis-corrected sandwich standard errors, ability scoring, cross-fitted tuning, and scale linking are also provided.

Last updated

data-augmentationpsychometrics

5.45 score 14 scripts 387 downloads

meow - Unified Framework for Computer Adaptive Testing Simulations

Provides an extensible framework for conducting simulations to compare data generating processes, item selection algorithms, parameter update algorithms, and stopping rules in computer adaptive testing (CAT) applications. Bundled algorithms include the Elo-based update rules of Klinkenberg, Straatemeier and van der Maas (2011) <doi:10.1016/j.compedu.2011.02.003> and Vermeiren, Kruis, Bolsinova, van der Maas and Hofman (2025) <doi:10.1016/j.caeai.2025.100376>.

Last updated

5.43 score 17 scripts 239 downloads

mixedsubjects - Causal Inference in Experiments with Mixed-Subjects Designs

Implements seven estimators for average treatment effect (ATE) estimation in mixed-subjects designs (MSDs), where human subjects data is augmented with predictions from large language models (LLMs)., Includes Difference-in-Means, GREG, Prediction Powered Inference (PPI), Power Tuned PPI (PPI++), Doubly-Tuned PPI (D-T PPI++), Difference-in-Predictions (DiP), DiP++, and D-T DiP estimators. Provides point estimates, variance estimation via delta-method or bootstrap, and optimal design selection for budget allocation between human observations and LLM predictions.

Last updated

4.78 score 2 scripts 256 downloads