Name: Opdyke

Firstname: J.D.

Title: President

Institution: DataMineIt

Street: 23 Naples Road #4

City: Brookline

Zip-Code: 02446

Country: USA

Phone: 203-249-4837

Fax:

Email: jdopdyke@usa.net

Authors: J.D. Opdyke (sole author)

Title: Fast Permutation Tests, Especially for Multiple Comparisons and Even When One Sample is Large, That Efficiently Maximize Power Under Conventional Monte Carlo Sampling and Can Be Used for Permutation-style p-value Adjustments

Abstract: I present a method designed especially for multiple comparisons for quickly performing multiple nonparametric two-sample permutation tests on continuous data, even when one sample is large. I maximize statistical power (within the context of a crude Monte Carlo approach) by “oversampling” – drawing more permutation samples than desired, deleting duplicates, and then selecting the desired number of samples from the remainder. I determine the optimal number of samples to “oversample” based on sampling probability and the runtime of a sampling procedure (PROC PLAN in SAS^®). Implementing “oversampling” with nearly optimal numbers of samples increases start-to-finish runtime typically by only 5%, and always by less than 10%. I benchmark start-to-finish runtime against a) two other SAS procedures (PROC MULTTEST and PROC NPAR1WAY), b) another SAS program written for the same purpose, and c) Cytel’s PROC TWOSAMPL^®, with very favorable results. The code provides an even greater speed premium when used for multiple comparisons under certain null hypotheses, and it can be directly used to perform permutation-style p-value adjustments for multiple comparisons. Its speed even permits the simultaneous use of permutation testing and adjustments (see Westfall, 1993), although for relatively few tests.