Revisit of test‐then‐pool methods and some practical considerations

Summary Test‐then‐pool is a simple statistical method that borrows historical information to improve efficiency of the drug development process. The original test‐then‐pool method examines the difference between the historical and current information and then pools the information if there is no significant difference. One drawback of this method is that a nonsignificant difference may not always imply consistency between the historical and current information. As a result, the original test‐then‐pool method is more likely to incorrectly borrow information from the historical control when the current trial has a small sample size. Statistically, it is more natural to use an equivalence test for examining the consistency. This manuscript develops an equivalence‐based test‐then‐pool method for a continuous endpoint, explains the relationship between the two test‐then‐pool methods, explores the choice of an equivalence margin through the overlap probability, and proposes an adjustment to the nominal testing level for controlling type I error under the true consistency scenario. Furthermore, the analytical forms of the type I error and power for the two test‐then‐pool methods are derived, and practical considerations for using them are presented.