Short Course

Morning session:

Afternoon session:


Fundamentals of Multiple Testing and Biotechnology with Applications to Clinical Trials and Personalized Medicine

Jason C. Hsu (The Ohio State University) and
Xinping Cui (University of California Riverside)

Clinical trials have become more complex. And use of biomarkers to target a patient subgroup (personalized medicine) is becoming increasingly common. This course uses Partition Testing, a principle of multiple test construction that can control a variety of error rates, to show applicable statistical concepts remain fundamentally the same in various clinical trials. Multiplicity issues raised in biomarker studies for personalized medicine are also discussed.
  1. To demonstrate multiple test construction, we show that, when there are multiple endpoints in combination with multiple doses, partition testing can directly formulate null hypotheses to respects decision paths, testing exponentially fewer hypotheses than closed testing and gatekeeping.
  2. Before discussing multiple testing in personalized medicine, some basics of molecular biology and next generation sequencing technology will be introduced.
  3. To demonstrate sound multiple testing principle for biomarker studies, we first construct a rigorous test for a predictive biomarker, testing whether a SNP mutation has a dominant, recessive, or additive effect on clinical response (while controlling familywise error rate FWER).
  4. Then, to discuss multiplicity issues in GWAS (genome-wide association studies) for discovery of potential biomarkers, we consider how to adjust for multiplicity across different SNPs and genes. Specifically, we will examine critically the interpretation and implication of controlling per family error rate, false discovery rate (FDR), and local false discovery rate (Fdr).


Adaptive Designs

Martin Posch, Franz König (Medical University of Vienna)

The goal of this course is to give an introduction to the key principles and statistical methodologies of adaptive designs for clinical trials. Adaptive (flexible) designs allow for mid-course design adaptations based on interim data without compromising the overall type I error rate. Examples of design adaptations are the adjustment of sample sizes or the number and timing of interim analyses. These design parameters may be adapted depending on interim estimates of the variance, the treatment effect and safety parameters. An important field of application of the adaptive design methodology are clinical trials with several treatment arms, where promising treatments can be selected at an interim analysis. Using adaptive multiple test procedures the type I error rate can be controlled even if the selection rule, the number of selected treatments or the final sample sizes are not prefixed. Adaptive multiple testing procedures can also be used in adaptive designs with the option of population enrichment. In such designs a sub population may be selected in an interim analysis and further recruitment of patients is restricted to the selected subgroup.

The course provides an overview of the current state of the art in design i and analysis of adaptive clinical trials including the most recent developments. Special emphasis is put on un-blinded sample size adjustment and multiple hypotheses testing with adaptive designs. Furthermore, regulatory issues will be discussed.


Graphical approaches to multiple test problems

Frank Bretz, Ekkehard Glimm, Willi Maurer (Novartis)

Methods for addressing multiplicity are becoming increasingly more important in clinical trials and other applications. In the recent past several multiple test procedures have been developed that allow one to map the relative importance of different study objectives as well as their relation onto an appropriately tailored multiple test procedure, such as fixed-sequence, fallback, and gatekeeping procedures. In this course we focus on graphical approaches that can be applied to common multiple test problems, such as comparing several treatments with a control, assessing the benefit of a new drug for more than one endpoint, and combined non-inferiority and superiority testing. Using graphical approaches, one can easily construct and explore different test strategies and thus tailor the test procedure to the given study objectives. The resulting multiple test procedures are represented by directed, weighted graphs, where each node corresponds to an elementary hypothesis, together with a simple algorithm to generate such graphs while sequentially testing the individual hypotheses. The class of procedures covered in this course include weighted Bonferroni tests, weighted parametric tests accounting for the correlation between the test statistics, and weighted Simes' tests. The approach is illustrated with the visualization of several common gatekeeping strategies. We also present several case studies to illustrate how the approach can be used in clinical practice. In addition, we briefly consider power and sample size calculation to optimize a multiple test procedure for given study objectives. The presented methods will be illustrated using the graphical user interface from the gMCP package in R, which is freely available on CRAN.


Gatekeeping procedures in clinical trials

Alex Dmitrienko (Quintiles)

This half-day course will focus on issues arising in clinical trials with ordered multiple objectives (e.g., primary and secondary endpoints, primary and secondary patient populations, etc.) using gatekeeping procedures. The course will provide a detailed overview of novel statistical approaches developed over the past ten years. There will be a well-balanced coverage of theory and applications, regulatory considerations and software implementation of gatekeeping procedures in SAS and R. Examples from clinical trials will be used throughout the discussion to illustrate the statistical approaches discussed in the course.

Text book: Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). (2009). Multiple Testing Problems in Pharmaceutical Statistics. Chapman and Hall/CRC Press, New York.

Web site: Supplementary material, including the bibliography, software code and useful links, is available on the Multxpert web site: