Physical Sciences and Mathematics Commons^™

Multi-State Models For Natural History Of Disease, Amy Laird, Rebecca A. Hubbard, Lurdes Y. T. Inoue

UW Biostatistics Working Paper Series

Longitudinal studies are a useful tool for investigating the course of chronic diseases. Many chronic diseases can be characterized by a set of health states. We can improve our understanding of the natural history of the disease by modeling the sequence of visited health states and the duration in each state. However, in most applications, subjects are observed only intermittently. This observation scheme creates a major modeling challenge: the transition times are not known exactly, and in some cases the path through the health states is not known.

In this manuscript we review existing approaches for modeling multi-state longitudinal data. …

Issues Related To Combining Multiple Speciated Pm2.5 Data Sources In Spatio-Temporal Exposure Models For Epidemiology: The Npact Case Study, Sun-Young Kim, Lianne Sheppard, Timothy V. Larson, Joel Kaufman, Sverre Vedal

UW Biostatistics Working Paper Series

Background: Regulatory monitoring data have been the most common exposure data resource in studies of the association between long-term PM_2.5 components and health. However, data collected for regulatory purposes may not be compatible with epidemiological study.

Objectives: We aimed to explore three important features of the PM_2.5 component monitoring data obtained from multiple sources to combine all available data for developing spatio-temporal prediction models in the National Particle Component and Toxicity (NPACT) study.

Methods: The NPACT monitoring data were collected in an extensive monitoring campaign targeting cohort participants. The regulatory monitoring data were obtained from the Chemical Speciation …

Prediction Of Fine Particulate Matter Chemical Components For The Multi-Ethnic Study Of Atherosclerosis Cohort: A Comparison Of Two Modeling Approaches, Sun-Young Kim, Lianne Sheppard, Silas Bergen, Adam A. Szpiro, Paul D. Sampson, Joel Kaufman, Sverre Vedal

UW Biostatistics Working Paper Series

Recent epidemiological cohort studies of the health effects of PM_2.5 have developed exposure estimates from advanced exposure prediction models. Such models represent spatial variability across participant residential locations. However, few cohort studies have developed exposure predictions for PM_2.5 components. We used two exposure modeling approaches to obtain long-term average predicted concentrations for four PM_2.5 components: sulfur, silicon, and elemental and organic carbon (EC and OC). The models were specifically developed for the Multi-Ethnic Study of Atherosclerosis (MESA) cohort as a part of the National Particle Component and Toxicity (NPACT) study. The spatio-temporal model used 2-week average measurements …

Characterizing Expected Benefits Of Biomarkers In Treatment Selection, Ying Huang, Eric Laber, Holly Janes

UW Biostatistics Working Paper Series

Biomarkers associated with the treatment response heterogeneity hold potential for treatment selection. In practice, the decision regarding whether to adopt a treatment selection marker depends on the effect of treatment selection on the rate of targeted disease as well as additional cost associated with the treatment. We propose an expected benefit measure that incorporates both aspects to quantify a biomarker's treatment selection capacity. This measure extends an existing decision-theoretic framework, to account for the fact that optimal treatment absent marker information varies with the cost of treatment. In addition, we establish upper and lower bounds for the performance of a …

Hypothesis Testing For An Extended Cox Model With Time-Varying Coefficients, Takumi Saegusa, Chongzhi Di, Ying Qing Chen

UW Biostatistics Working Paper Series

The log-rank test has been widely used to test a treatment effect under the Cox model for censored time-to-event outcomes, though it may lose power substantially when the model's proportional hazards assumption does not hold. In this paper, we consider an extended Cox model that uses B-splines or smoothing splines to model a time-varying treatment effect and propose score test statistics for the treatment effect. Our proposed new tests combine statistical evidence from both the magnitude and the shape of the time-varying hazard ratio function, and thus are omnibus and powerful against various types of alternatives. In addition, the new …

Net Reclassification Index: A Misleading Measure Of Prediction Improvement, Margaret Sullivan Pepe, Holly Janes, Kathleen F. Kerr, Bruce M. Psaty

UW Biostatistics Working Paper Series

The evaluation of biomarkers to improve risk prediction is a common theme in modern research. Since its introduction in 2008, the net reclassification index (NRI) (Pencina et al. 2008, Pencina et al. 2011) has gained widespread use as a measure of prediction performance with over 1,200 citations as of June 30, 2013. The NRI is considered by some to be more sensitive to clinically important changes in risk than the traditional change in the AUC (Delta AUC) statistic (Hlatky et al. 2009). Recent statistical research has raised questions, however, about the validity of conclusions based on the NRI. (Hilden and …

Net Reclassification Indices For Evaluating Risk Prediction Instruments: A Critical Review, Kathleen F. Kerr, Zheyu Wang, Holly Janes, Robyn Mcclelland, Bruce M. Psaty, Margaret S. Pepe

UW Biostatistics Working Paper Series

Background Net Reclassification Indices (NRI) have recently become popular statistics for measuring the prediction increment of new biomarkers.

Methods In this review, we examine the various types of NRI statistics and their correct interpretations. We evaluate the advantages and disadvantages of the NRI approach. For pre-defined risk categories, we relate NRI to existing measures of the prediction increment. We also consider statistical methodology for constructing confidence intervals for NRI statistics and evaluate the merits of NRI-based hypothesis testing.

Conclusions Investigators using NRI statistics should report them separately for events (cases) and nonevents (controls). When there are two risk categories, the …

The Net Reclassification Index (Nri): A Misleading Measure Of Prediction Improvement With Miscalibrated Or Overfit Models, Margaret Pepe, Jin Fang, Ziding Feng, Thomas Gerds, Jorgen Hilden

UW Biostatistics Working Paper Series

The Net Reclassification Index (NRI) is a very popular measure for evaluating the improvement in prediction performance gained by adding a marker to a set of baseline predictors. However, the statistical properties of this novel measure have not been explored in depth. We demonstrate the alarming result that the NRI statistic calculated on a large test dataset using risk models derived from a training set is likely to be positive even when the new marker has no predictive information. A related theoretical example is provided in which a miscalibrated risk model that includes an uninformative marker is proven to erroneously …

Asymptotic And Finite Sample Behavior Of Net Reclassification Indices, Zheyu Wang

UW Biostatistics Working Paper Series

The Net Reclassification Index (NRI) introduced by Pencina and colleagues [1, 2] is designed to quantify the prediction increment provided by a new biomarker. It has become popular for evaluating and selecting novel markers. The published variance formulae for NRI statistics do not account for the fact that risks are estimated based on risk models fit to data, and thus are not valid in practice when estimated risks are used [3]. Kerr and colleagues [4] showed that the confidence intervals constructed based on a bootstrap estimate of the variance and Normal approximation had the best performance among various methods they …

Statistical Methods For Evaluating And Comparing Biomarkers For Patient Treatment Selection, Holly Janes, Marshall D. Brown, Margaret Pepe, Ying Huang

UW Biostatistics Working Paper Series

Despite the heightened interest in developing biomarkers predicting treatment response that are used to optimize patient treatment decisions, there has been relatively little development of statistical methodology to evaluate these markers. There is currently no unified statistical framework for marker evaluation. This paper proposes a suite of descriptive and inferential methods designed to evaluate individual markers and to compare candidate markers. An R software package has been developed which implements these methods. Their utility is illustrated in the breast cancer treatment context, where candidate markers are evaluated for their ability to identify a subset of women who do not benefit …

An Evaluation Of Inferential Procedures For Adaptive Clinical Trial Designs With Pre-Specified Rules For Modifying The Sample Size, Greg P. Levin, Sarah C. Emerson, Scott S. Emerson

UW Biostatistics Working Paper Series

Many papers have introduced adaptive clinical trial methods that allow modifications to the sample size based on interim estimates of treatment effect. There has been extensive commentary on type I error control and efficiency considerations, but little research on estimation after an adaptive hypothesis test. We evaluate the reliability and precision of different inferential procedures in the presence of an adaptive design with pre-specified rules for modifying the sampling plan. We extend group sequential orderings of the outcome space based on the stage at stopping, likelihood ratio test statistic, and sample mean to the adaptive setting in order to compute …