Variable-based probabilistic calibration with binary outcome
Hiroe Seto, Shuji Kitora, Asuka Oyama +3 more
In developing risk prediction models for specific diseases, it is essential to evaluate the calibration performance of the prediction model. Various methods have been proposed to assess the calibration of prediction models, but it has been pointed out that conventional methods based on the predicted probability of the model are insufficient to detect miscalibration. Another problem is that a method for evaluating calibration for continuous variables of interest has not yet been established. We therefore propose two methods to evaluate the calibration of the variable of interest: the variable-based probabilistic calibration plot (VPC-Plot), which is a visual assessment, and the variable-based probabilistic calibration error (VPCE), which is a corresponding evaluation metric. We conducted theoretical and simulation studies to investigate the properties and effectiveness of the proposed…