Physical Sciences and Mathematics Commons^™

Hypothesis Testing For An Extended Cox Model With Time-Varying Coefficients, Takumi Saegusa, Chongzhi Di, Ying Qing Chen

UW Biostatistics Working Paper Series

The log-rank test has been widely used to test a treatment effect under the Cox model for censored time-to-event outcomes, though it may lose power substantially when the model's proportional hazards assumption does not hold. In this paper, we consider an extended Cox model that uses B-splines or smoothing splines to model a time-varying treatment effect and propose score test statistics for the treatment effect. Our proposed new tests combine statistical evidence from both the magnitude and the shape of the time-varying hazard ratio function, and thus are omnibus and powerful against various types of alternatives. In addition, the new …

Net Reclassification Index: A Misleading Measure Of Prediction Improvement, Margaret Sullivan Pepe, Holly Janes, Kathleen F. Kerr, Bruce M. Psaty

UW Biostatistics Working Paper Series

The evaluation of biomarkers to improve risk prediction is a common theme in modern research. Since its introduction in 2008, the net reclassification index (NRI) (Pencina et al. 2008, Pencina et al. 2011) has gained widespread use as a measure of prediction performance with over 1,200 citations as of June 30, 2013. The NRI is considered by some to be more sensitive to clinically important changes in risk than the traditional change in the AUC (Delta AUC) statistic (Hlatky et al. 2009). Recent statistical research has raised questions, however, about the validity of conclusions based on the NRI. (Hilden and …

Net Reclassification Indices For Evaluating Risk Prediction Instruments: A Critical Review, Kathleen F. Kerr, Zheyu Wang, Holly Janes, Robyn Mcclelland, Bruce M. Psaty, Margaret S. Pepe

UW Biostatistics Working Paper Series

Background Net Reclassification Indices (NRI) have recently become popular statistics for measuring the prediction increment of new biomarkers.

Methods In this review, we examine the various types of NRI statistics and their correct interpretations. We evaluate the advantages and disadvantages of the NRI approach. For pre-defined risk categories, we relate NRI to existing measures of the prediction increment. We also consider statistical methodology for constructing confidence intervals for NRI statistics and evaluate the merits of NRI-based hypothesis testing.

Conclusions Investigators using NRI statistics should report them separately for events (cases) and nonevents (controls). When there are two risk categories, the …

The Net Reclassification Index (Nri): A Misleading Measure Of Prediction Improvement With Miscalibrated Or Overfit Models, Margaret Pepe, Jin Fang, Ziding Feng, Thomas Gerds, Jorgen Hilden

UW Biostatistics Working Paper Series

The Net Reclassification Index (NRI) is a very popular measure for evaluating the improvement in prediction performance gained by adding a marker to a set of baseline predictors. However, the statistical properties of this novel measure have not been explored in depth. We demonstrate the alarming result that the NRI statistic calculated on a large test dataset using risk models derived from a training set is likely to be positive even when the new marker has no predictive information. A related theoretical example is provided in which a miscalibrated risk model that includes an uninformative marker is proven to erroneously …

Asymptotic And Finite Sample Behavior Of Net Reclassification Indices, Zheyu Wang

UW Biostatistics Working Paper Series

The Net Reclassification Index (NRI) introduced by Pencina and colleagues [1, 2] is designed to quantify the prediction increment provided by a new biomarker. It has become popular for evaluating and selecting novel markers. The published variance formulae for NRI statistics do not account for the fact that risks are estimated based on risk models fit to data, and thus are not valid in practice when estimated risks are used [3]. Kerr and colleagues [4] showed that the confidence intervals constructed based on a bootstrap estimate of the variance and Normal approximation had the best performance among various methods they …

Statistical Methods For Evaluating And Comparing Biomarkers For Patient Treatment Selection, Holly Janes, Marshall D. Brown, Margaret Pepe, Ying Huang

UW Biostatistics Working Paper Series

Despite the heightened interest in developing biomarkers predicting treatment response that are used to optimize patient treatment decisions, there has been relatively little development of statistical methodology to evaluate these markers. There is currently no unified statistical framework for marker evaluation. This paper proposes a suite of descriptive and inferential methods designed to evaluate individual markers and to compare candidate markers. An R software package has been developed which implements these methods. Their utility is illustrated in the breast cancer treatment context, where candidate markers are evaluated for their ability to identify a subset of women who do not benefit …

An Evaluation Of Inferential Procedures For Adaptive Clinical Trial Designs With Pre-Specified Rules For Modifying The Sample Size, Greg P. Levin, Sarah C. Emerson, Scott S. Emerson

UW Biostatistics Working Paper Series

Many papers have introduced adaptive clinical trial methods that allow modifications to the sample size based on interim estimates of treatment effect. There has been extensive commentary on type I error control and efficiency considerations, but little research on estimation after an adaptive hypothesis test. We evaluate the reliability and precision of different inferential procedures in the presence of an adaptive design with pre-specified rules for modifying the sampling plan. We extend group sequential orderings of the outcome space based on the stage at stopping, likelihood ratio test statistic, and sample mean to the adaptive setting in order to compute …

A Regionalized National Universal Kriging Model Using Partial Least Squares Regression For Estimating Annual Pm2.5 Concentrations In Epidemiology, Paul D. Sampson, Mark Richards, Adam A. Szpiro, Silas Bergen, Lianne Sheppard, Timothy V. Larson, Joel Kaufman

UW Biostatistics Working Paper Series

Many cohort studies in environmental epidemiology require accurate modeling and prediction of fine scale spatial variation in ambient air quality across the U.S. This modeling requires the use of small spatial scale geographic or “land use” regression covariates and some degree of spatial smoothing. Furthermore, the details of the prediction of air quality by land use regression and the spatial variation in ambient air quality not explained by this regression should be allowed to vary across the continent due to the large scale heterogeneity in topography, climate, and sources of air pollution. This paper introduces a regionalized national universal kriging …

A National Model Built With Partial Least Squares And Universal Kriging And Bootstrap-Based Measurement Error Correction Techniques: An Application To The Multi-Ethnic Study Of Atherosclerosis, Silas Bergen, Lianne Sheppard, Paul D. Sampson, Sun-Young Kim, Mark Richards, Sverre Vedal, Joel Kaufman, Adam A. Szpiro

UW Biostatistics Working Paper Series

Studies estimating health effects of long-term air pollution exposure often use a two-stage approach, building exposure models to assign individual-level exposures which are then used in regression analyses. This requires accurate exposure modeling and careful treatment of exposure measurement error. To illustrate the importance of carefully accounting for exposure model characteristics in two-stage air pollution studies, we consider a case study based on data from the Multi-Ethnic Study of Atherosclerosis (MESA). We present national spatial exposure models that use partial least squares and universal kriging to estimate annual average concentrations of four PM2.5 components: elemental carbon (EC), organic carbon (OC), …

Decline In Health For Older Adults: 5-Year Change In 13 Key Measures Of Standardized Health, Paula H. Diehr, Stephen M. Thielke, Anne B. Newman, Calvin H. Hirsch, Russell Tracy

UW Biostatistics Working Paper Series

Introduction

The health of older adults declines over time, but there are many ways of measuring health. We examined whether all measures declined at the same rate, or whether some aspects of health were less sensitive to aging than others.

Methods

We compared the decline in 13 measures of physical, mental, and functional health from the Cardiovascular Health Study: hospitalization, bed days, cognition, extremity strength, feelings about life as a whole, satisfaction with the purpose of life, self-rated health, depression, digit symbol substitution test, grip strength, ADLs, IADLs, and gait speed. Each measure was standardized against self-rated health. We compared …

Methods For Evaluating Prediction Performance Of Biomarkers And Tests, Margaret Pepe, Holly Janes

UW Biostatistics Working Paper Series

This chapter describes and critiques methods for evaluating the performance of markers to predict risk of a current or future clinical outcome. We consider three criteria that are important for evaluating a risk model: calibration, benefit for decision making and accurate classification. We also describe and discuss a variety of summary measures in common use for quantifying predictive information such as the area under the ROC curve and R-squared. The roles and problems with recently proposed risk reclassification approaches are discussed in detail.

Borrowing Information Across Populations In Estimating Positive And Negative Predictive Values, Ying Huang, Youyi Fong, John Wei, Ziding Feng

UW Biostatistics Working Paper Series

A marker's capacity to predict risk of a disease depends on disease prevalence in the target population and its classification accuracy, i.e. its ability to discriminate diseased subjects from non-diseased subjects. The latter is often considered an intrinsic property of the marker; it is independent of disease prevalence and hence more likely to be similar across populations than risk prediction measures. In this paper, we are interested in evaluating the population-specific performance of a risk prediction marker in terms of positive predictive value (PPV) and negative predictive value (NPV) at given thresholds, when samples are available from the target population …

Fitting And Interpreting Continuous-Time Latent Markov Models For Panel Data, Jane M. Lange, Vladimir N. Minin

UW Biostatistics Working Paper Series

Multistate models are used to characterize disease processes within an individual. Clinical studies often observe the disease status of individuals at discrete time points, making exact times of transitions between disease states unknown. Such panel data pose considerable modeling challenges. Assuming the disease process progresses according a standard continuous-time Markov chain (CTMC) yields tractable likelihoods, but the assumption of exponential sojourn time distributions is typically unrealistic. More flexible semi-Markov models permit generic sojourn distributions yet yield intractable likelihoods for panel data in the presence of reversible transitions. One attractive alternative is to assume that the disease process is characterized by …

Transitions Among Health States Using 12 Measures Of Successful Aging: Results From The Cardiovascular Health Study, Stephen Thielke, Paula Diehr

UW Biostatistics Working Paper Series

Introduction

Successful aging has many dimensions, which may manifest differently in men and women and at different ages. We sought to characterize one-year transitions in 12 measures of successful aging among a large cohort of older adults.

Methods

We analyzed twelve different measures of health in the Cardiovascular Health Study: self-rated health, ADLs, IADLs, depression, cognition, timed walk, number of days spent in bed, number of blocks walked, extremity strength, recent hospitalizations, feelings about life as a whole, and life satisfaction. We dichotomized responses for each variable into “healthy” or “sick”, and estimated the prevalence of the healthy state and …

Testing For Improvement In Prediction Model Performance, Margaret S. Pepe Phd, Kathleen F. Kerr, Gary M. Longton, Zheyu Wang

UW Biostatistics Working Paper Series

New methodology has been proposed in recent years for evaluating the improvement in prediction performance gained by adding a new predictor, Y, to a risk model containing a set of baseline predictors, X, for a binary outcome D. We prove theoretically that null hypotheses concerning no improvement in performance are equivalent to the simple null hypothesis that the coefficient for Y is zero in the risk model, P(D = 1|X, Y ). Therefore, testing for improvement in prediction performance is redundant if Y has already been shown to be a risk factor. We investigate properties of tests through simulation studies, …

Some Observations On The Wilcoxon Rank Sum Test, Scott S. Emerson

UW Biostatistics Working Paper Series

This manuscript presents some general comments about the Wilcoxon rank sum test. Even the most casual reader will gather that I am not too impressed with the scientific usefulness of the Wilcoxon test. However, the actual motivation is more to illustrate differences between parametric, semiparametric, and nonparametric (distribution-free) inference, and to use this example to illustrate how many misconceptions have been propagated through a focus on (semi)parametric probability models as the basis for evaluating commonly used statistical analysis models. The document itself arose as a teaching tool for courses aimed at graduate students in biostatistics and statistics, with parts of …

The Importance Of Statistical Theory In Outlier Detection, Sarah C. Emerson, Scott S. Emerson

UW Biostatistics Working Paper Series

We explore the performance of the outlier-sum statistic (Tibshirani and Hastie, Biostatistics 2007 8:2--8), a proposed method for identifying genes for which only a subset of a group of samples or patients exhibits differential expression levels. Our discussion focuses on this method as an example of how inattention to standard statistical theory can lead to approaches that exhibit some serious drawbacks. In contrast to the results presented by those authors, when comparing this method to several variations of the $t$-test, we find that the proposed method offers little benefit even in the most idealized scenarios, and suffers from a number …

When Does Combining Markers Improve Classification Performance And What Are Implications For Practice?, Aasthaa Bansal, Margaret Sullivan Pepe

UW Biostatistics Working Paper Series

When an existing standard marker does not have sufficient classification accuracy on its own, new markers are sought with the goal of yielding a combination with better performance. The primary criterion for selecting new markers is that they have good performance on their own and preferably be uncorrelated with the standard. Most often linear combinations are considered. In this paper we investigate the increment in performance that is possible by combining a novel continuous marker with a moderately performing standard continuous marker under a variety of biologically motivated models for their joint distribution. We find that an uncorrelated continuous marker …

Adaptive Clinical Trial Designs With Pre-Specified Rules For Modifying The Sample Size: Understanding Efficient Types Of Adaptation, Gregory P. Levin, Sarah C. Emerson, Scott S. Emerson

UW Biostatistics Working Paper Series

Methods allowing unplanned adaptations to the sample size based on the interim estimate of treatment effect do not base inference on the minimal sufficient statistic and suffer losses in efficiency when compared to group sequential designs [1, 2, 3]. However, when adaptive sampling plans are completely pre-specified at the design stage of the trial, investigators can proceed with frequentist inference based on the minimal sufficient statistic at the analysis stage. In the context of two general settings where different optimality criteria govern the choice of clinical trial design, we quantify the relative costs and benefits of a variety of fixed …

Bate Curve In Assessment Of Clinical Utility Of Predictive Biomarkers, Xiao-Hua Zhou, Yunbei Ma

UW Biostatistics Working Paper Series

In this paper, for time-to-event data, we propose a new statistical framework for casual inference in evaluating clinical utility of predictive biomarkers and in selecting an optimal treatment for a particular patient. This new casual framework is based on a new concept, called Biomarker Adjusted Treatment Effect (BATE) curve, which can be used to represent the clinical utility of a predictive biomarker and select an optimal treatment for one particular patient. We then propose semi-parametric methods for estimating the BATE curves of biomarkers and establish asymptotic results of the proposed estimators for the BATE curves. We also conduct extensive simulation …

Non-Homogeneous Markov Process Models With Incomplete Observations: Application To A Dementia Disease Study, Xiao-Hua Zhou, Baojiang Chen

UW Biostatistics Working Paper Series

Identifying risk factors for transition rates among normal cognition, mildly cognitive impairment, dementia and death in an Alzheimer's disease study is very important. It is known that transition rates among these states are strongly time dependent. While Markov process models are often used to describe these disease progressions, the literature mainly focuses on time homogeneous processes, and limited tools are available for dealing with non-homogeneity. Further, patients may choose when they want to visit the clinics, which creates informative observations. In this paper, we develop methods to deal with non-homogeneous Markov processes through time scale transformation when observation times are …

Doubly Robust Estimates For Binary Longitudinal Data Analysis With Missing Response And Missing Covariates, Baojiang Chen, Xiao-Hua Zhou

UW Biostatistics Working Paper Series

Longitudinal studies often feature incomplete response and covariate data. Likelihood-based methods such as the EM algorithm give consistent estimators for model parameters when data are missing at random provided that the response model and the missing covariate model are correctly specified; but we do not need to specify the missing data mechanism. An alternative method is the weighted estimating equation which gives consistent estimators if the missing data and response models are correctly specified; but we do not need to specify the distribution of the covariates that have missing values. In this paper we develop a doubly robust estimation method …

Semiparametric Estimation Of The Covariate-Specific Roc Curve In Presence Of Ignorable Verification Bias, Danping Liu, Xiao-Hua Zhou

UW Biostatistics Working Paper Series

Covariate-specific ROC curves are often used to evaluate the classification accuracy of a medical diagnostic test or a biomarker, when the accuracy of the test is associated with certain covariates. In many large-scale screening tests, the gold standard is subject to missingness due to high cost or harmfulness to the patient. In this paper, we propose a semiparametric estimation method for the covariate-specific ROC curves with a partial missing gold standard. A location-scale model is constructed for the test result to model the covariates' effect, but the residual distributions are left unspecified. Thus the baseline and link functions of the …

Evaluating Markers For Treatment Selection Based On Survival Time, Xiao Song, Xiao-Hua Zhou

UW Biostatistics Working Paper Series

For many medical conditions there are several treatment options available to patients. We consider evaluating markers based on a simple treatment selection policy that incorporates information on the patient's marker value exceeding a threshold. Although traditional regression methods may assess the effect of the marker and treatment on outcomes, it is appealing to quantify more directly the potential impact on the population of using the marker to select treatment. A useful tool is the selection impact (SI) curve proposed by Song and Pepe (2004, \textit{Biometrics} \textbf{60}, 874--883) for binary outcomes. However, this approach does not deal with continuous outcomes, nor …

A Flexible Spatio-Temporal Model For Air Pollution: Allowing For Spatio-Temporal Covariates, Johan Lindstrom, Adam A. Szpiro, Paul D. Sampson, Lianne Sheppard, Assaf Oron, Mark Richards, Tim Larson

UW Biostatistics Working Paper Series

Given the increasing interest in the association between exposure to air pollution and adverse health outcomes, the development of models that provide accurate spatio-temporal predictions of air pollution concentrations at small spatial scales is of great importance when assessing potential health effects of air pollution. The methodology presented here has been developed as part of the Multi-Ethnic Study of Atherosclerosis and Air Pollution (MESA Air), a prospective cohort study funded by the US EPA to investigate the relationship between chronic exposure to air pollution and cardiovascular disease. We present a spatio-temporal framework that models and predicts ambient air pollution by …

Oracle And Multiple Robustness Properties Of Survey Calibration Estimator In Missing Response Problem, Kwun Chuen Gary Chan

UW Biostatistics Working Paper Series

In the presence of missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. However, inverse probability weighting is not efficient in general and is not robust against misspecification of the missing probability model. Calibration was developed by survey statisticians for improving efficiency of inverse probability weighting estimators when population totals of auxiliary variables are known and when inclusion probability is known by design. In missing data problem we can calibrate auxiliary variables in the complete case subsample to the full sample. However, the inclusion probability is unknown in general …

Modification And Improvement Of Empirical Likelihood For Missing Response Problem, Kwun Chuen Gary Chan

UW Biostatistics Working Paper Series

An empirical likelihood (EL) estimator was proposed by Qin and Zhang (2007) for a missing response problem under a missing at random assumption. They showed by simulation studies that the finite sample performance of EL estimator is better than some existing estimators. However, the empirical likelihood estimator does not have a uniformly smaller asymptotic variance than other estimators in general. We consider several modifications to the empirical likelihood estimator and show that the proposed estimator dominates the empirical likelihood estimator and several other existing estimators in terms of asymptotic efficiencies. The proposed estimator also attains the minimum asymptotic variance among …

Modification And Improvement Of Empirical Liklihood For Missing Response Problem, Gary Chan

UW Biostatistics Working Paper Series

An empirical likelihood (EL) estimator was proposed by Qin and Zhang (2007) for a missing response problem under a missing at random assumption. They showed by simulation studies that the finite sample performance of EL estimator is better than some existing estimators. However, the empirical likelihood estimator does not have a uniformly smaller asymptotic variance than other estimators in general. We consider several modifications to the empirical likelihood estimator and show that the proposed estimator dominates the empirical likelihood estimator and several other existing estimators in terms of asymptotic efficiencies. The proposed estimator also attains the minimum asymptotic variance among …

Efficient Measurement Error Correction With Spatially Misaligned Data, Adam A. Szpiro, Lianne Sheppard, Thomas Lumley

UW Biostatistics Working Paper Series

Association studies in environmental statistics often involve exposure and outcome data that are misaligned in space. A common strategy is to employ a spatial model such as universal kriging to predict exposures at locations with outcome data and then estimate a regression parameter of interest using the predicted exposures. This results in measurement error because the predicted exposures do not correspond exactly to the true values. We characterize the measurement error by decomposing it into Berkson-like and classical-like components. One correction approach is the parametric bootstrap, which is effective but computationally intensive since it requires solving a nonlinear optimization problem …

On Two-Stage Hypothesis Testing Procedures Via Asymptotically Independent Statistics, James Dai, Charles Kooperberg, Michael L. Leblanc, Ross Prentice

UW Biostatistics Working Paper Series

Kooperberg and LeBlanc (2008) proposed a two-stage testing procedure to screen for significant interactions in genome-wide association (GWA) studies by a soft threshold on marginal associations (MA), though its theoretical properties and generalization have not been elaborated. In this article, we discuss conditions that are required to achieve strong control of the Family-Wise Error Rate (FWER) by such procedures for low or high-dimensional hypothesis testing. We provide proof of asymptotic independence of marginal association statistics and interaction statistics in linear regression, logistic regression, and Cox proportional hazard models in a randomized clinical trial (RCT) with a rare event. In case-control …