[R] Calibration score for survival probability

[Frank E Harrell Jr]

Eleni Rapsomaniki wrote:
> Good afternoon!
> 
> I need to evaluate the goodness-of-fit (aka calibration) for survival probability estimates from a Cox model. 
> I tried to use 'calibrate' in the Design package but I'm not sure if it should/would produce what I need (ie a chi-sq type statistic with a table of expected vs observed probabilities). Any other functions I should be aware of?
> 
> Also, has anybody come across an implementation of the statistic described in:
> "A global goodness of fit statistic for Cox regression models" by Parzen & Lpisitz, Biometrics 55, 1999 
> 
> Many thanks in advance
> 
> Eleni Rapsomaniki

Eleni,

The Design package, and its replacement, the rms package, produces 
calibration curves but no chi-square test because we do not have a 
corresponding method for that.  Formal tests are overused in this 
context anyway.  An index such as the maximum or 90th percentile of 
absolute calibration error are often more useful.  I have learned 
however that any statistical method that categorizes continuous 
variables (in this case, the predictions or the covariate space) is 
arbitrary and has many other problems.  The calibrate functions in the 
rms package have a new option to obtain smooth calibration curves 
without grouping, by fitting spline hazard models during validation.

Note that if you have done any model/variable selection you have to 
re-run such model building from scratch for each resample of the data, 
or the calibration plot will be over optimistic.  calibrate() makes this 
automatic if doing backward stepdown variable selection.  Many 
statisticians make the mistake of only "validating" the final selected 
model, which can only be done by one-time data splitting (which requires 
  tens of thousands of observations to perform adequately).

Frank

> 
> Research Associate
> Strangeways Research Laboratory
> Department of Public Health and Primary Care
> 
> University of Cambridge
>  
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 


-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University