Statistical Tests
The following section discusses the statistical tests performed in
the MULTTEST procedure. For continuous data, a ttest for the
mean is available. For discrete variables, available tests are the
CochranArmitage (CA) linear trend test, the FreemanTukey (FT)
double arcsine test, the Peto mortalityprevalence test, and the
Fisher exact test.
Throughout this section, the discrete and continuous variables are
denoted by S_{vgsr} and X_{vgsr}, respectively, where v is the
variable, g is the treatment group, s is the stratum, and r is
the replication. A plus sign (+) subscript denotes summation over
an index.
CochranArmitage Linear Trend Test
The CochranArmitage linear trend test (Cochran 1954;
Armitage 1955; Agresti 1990) is implemented using a
Zscore approximation, an exact permutation
distribution, or a combination of both.
Let m_{vgs} denote the sample size for a binary variable v
within group g and stratum s. The pooled probability
estimate for variable v and stratum s is

p_{vs} = [(S_{v+s+})/(m_{v+s})]
The expected value (under constant withinstratum treatment
probabilities) for variable v, group g, and stratum s is

E_{vgs} = m_{vgs} p_{vs}
The test statistic for variable v has numerator
where t_{g} denotes a trend coefficient (specified by the CONTRAST
statement).
The binomial variance estimate for this statistic is
where
The hypergeometric variance estimate (the default) is
For any strata s with , the contribution to the
variance is taken to be zero.
PROC MULTTEST computes the Zscore statistic
The pvalue for this statistic comes from the standard normal distribution.
Whenever a 0 is computed for the denominator, the pvalue is
set to 1. This pvalue approximates the probability obtained
from the exact permutation distribution, discussed in the
following text.
The Zscore statistic can be continuitycorrected to better
approximate the permutation distribution. With continuity
correction c, the uppertailed pvalue is computed from
For twotailed, noncontinuitycorrected tests, PROC MULTTEST
reports the pvalue as 2 min(p, 1  p), where p is the
uppertailed pvalue. The same formula holds for the
continuitycorrected test, with the exception
that when the noncontinuitycorrected Z
and the continuitycorrected Z have opposite signs, the
twotailed pvalue is 1.
When the PERMUTATION= option is specified and no STRATA variable is
specified, PROC MULTTEST uses a continuity correction selected to
optimally approximate the uppertail probability of permutation
distributions with smaller marginal totals (Westfall and Lin 1988).
Otherwise, the continuity correction is specified using the
CONTINUITY= option in the TEST statement.
The CA Zscore statistic is the HoelWalburg (MantelHaenszel)
statistic reported by Dinse (1985).
When you use the PERMUTATION= option for CA in the TEST
statement, PROC MULTTEST computes the exact permutation
distribution of the trend score
and then compares the observed value of this trend with the
permutation distribution to obtain the pvalue
where X is a random variable from the permutation distribution
and where uppertailed tests are requested. This probability can be
viewed as a binomial probability, where the withinstratum
probabilities are constant and where the probability is conditional
with respect to the marginal totals S_{v+s+}. It also can be
considered a rerandomization probability.
Because the computations can be quite timeconsuming with large
data sets, specifying the PERMUTATION=number option in
the TEST statement limits the situations where PROC MULTTEST
computes the exact permutation distribution. When marginal
total success or total failure frequencies exceed number
for a particular stratum, the permutation distribution is
approximated using a continuitycorrected normal distribution.
You should be cautious in using the PERMUTATION= option in
conjunction with bootstrap resampling because the permutation
distribution is recomputed for each bootstrap sample. This
recomputation is not necessary with permutation resampling.
The permutation distribution is computed in two steps:
 The permutation distributions of the trend scores
are computed within each stratum.
 The distributions are convolved to obtain the distribution of
the total trend.
As long as the total success or failure frequency does not
exceed number for any stratum, the computed distributions
are exact. In other words, if number or
number for all s, then the
permutation trend distribution for variable v is computed
exactly.
In step 1, the distribution of the withinstratum trend
is computed using the multivariate hypergeometric distribution
of the S_{vgs+}, provided number is not exceeded.
This distribution can be written as
The distribution of the withinstratum trend is then computed
by summing these probabilities over appropriate configurations.
For further information on this technique, refer to Bickis and
Krewski (1986) and Westfall and Lin (1988). In step 2, the
exact convolution distribution is obtained for the trend
statistic summed over all strata having totals that meet the
threshold criterion. This distribution is obtained by applying
the fast Fourier transform to the exact withinstratum
distributions. A description of this general method can be
found in Pagano and Tritchler (1983) and Good (1987).
The convolution distribution of the overall trend is then computed
by convolving the exact distribution with the distribution of the
continuitycorrected standard normal approximation. To be more
specific, let S_{1} denote the subset of stratum indices that
satisfy the threshold criterion, and let S_{2} denote the subset of
indices that do not satisfy the criterion. Let T_{v1} denote the
combined trend statistic from the set S_{1}, which has an exact
distribution obtained using Fourier analysis as previously
outlined, and let T_{v1} denote the combined trend statistic from
the set S_{2}. Then the distribution of the overall trend T_{v} =
T_{v1} + T_{v2} is obtained by convolving the analytic distribution
of T_{v1} with the continuitycorrected normal approximation for
T_{v2}. Using the notation from
the "ZScore Approximation" section, this convolution can be written as
where Z is a standard normal random variable, and
In this expression, the summation of s in V_{v} is over
S_{2}, and c is the continuity correction discussed under
the Zscore approximation.
When a twotailed test is requested, the expected trend
is computed, and the twotailed pvalue is reported as
the permutation tail probability for the observed trend
T_{v} plus the permutation tail probability for
2E_{v}  T_{v}, the reflected trend.
FreemanTukey Double Arcsine Test
For this test, the trend scores t_{1}, ... , t_{G}
are centered to the values c_{1}, ... , c_{G}, where
, , and G is the number of groups. The
numerator of this test statistic is
and is weighted by the withinstrata sample size (m_{v+s}) to
ensure comparability with the ordinary CA trend statistic.
The function f(r,n) is the double arcsine
transformation:
The variance estimate is
and the test statistic is
The FreemanTukey transformation and its variance are described by
Freeman and Tukey (1950) and Miller (1978). Since its variance
is not weighted by the pooled probabilities, as is the
CA test, the FT test can be more useful than the CA test for
tests involving only a subset of the groups.
Peto MortalityPrevalence Test
The Peto test is a modified CochranArmitage procedure incorporating
mortality and prevalence information. It represents a special case
in PROC MULTTEST because the data structure requirements are
different, and the resampling methods used for adjusting pvalues
are not valid. The TIME= option variable is required
to specify
"death" times or, more generally, time of occurrence. In addition,
the test variables must assume one of the following three values.
 0 = no occurrence
 1 = incidental occurrence
 2 = fatal occurrence
Use the TIME= option variable to define the mortality strata, and
use the STRATA statement variable to define the prevalence strata.
The Peto test is computed like two CochranArmitage Zscore
approximations, one for prevalence and one for mortality.
In the following notation, the subscript
v represents the variable, g represents the treatment group,
s represents the stratum, and t represents the time.
Recall that a plus sign
(+) in a subscript location denotes summation over that
subscript.
Let S^{P}_{vgs} be the number of incidental occurrences, and let
m^{P}_{vgs} be the total sample size for variable v in group g,
stratum s, excluding fatal tumors.
Let S^{F}_{vgt} be the number of fatal occurrences in time period
t, and let m^{F}_{vgt} be the number alive at the end of time
t1.
The pooled probability estimates are
The expected values are
Define the numerator terms:
where t_{g} denotes a trend coefficient.
Define the denominator variance terms (using the binomial variance)
:
The hypergeometric variances (the default) are calculated by
weighting the withinstrata variances as discussed in
the "ZScore Approximation" section.
The Peto statistic is computed as
where c is a continuity correction. The pvalue is
determined from the standard normal distribution unless
the PERMUTATION=number option is used. When you
use the PERMUTATION= option for PETO in the TEST
statement, PROC MULTTEST computes the ``discrete
approximation'' permutation distribution described by
Mantel (1980) and Soper and Tonkonoh (1993). Specifically, the
permutation distribution of
is computed, assuming that and are independent over all s and t. The
pvalues are exact under this independence assumption. However,
the independence assumption is valid only asymptotically, which is
why these pvalues are called "approximate."
An exact permutation distribution is available only under the
assumption of equal risk of censoring in all treatment groups;
even then, computing this distribution can be cumbersome. Soper and
Tonkonoh (1993) describe situations where the discrete approximation
distribution closely fits the exact permutation distribution.
Fisher Exact Test
The CONTRAST statement in PROC MULTTEST enables you to compute
Fisher exact tests for twogroup comparisons. No stratification
variable is allowed for this test. Note, however, that the FISHER
exact test is a special case of the exact permutation tests
performed by PROC MULTTEST and that these permutation tests allow a
stratification variable. Recall that contrast coefficients can be
1, 0, or 1 for the Fisher test. The frequencies and sample
sizes of the groups scored as 1 are combined, as are the
frequencies and sample sizes of the groups scored as 1. Groups
scored as 0 are excluded. The 1 group is then compared with
the 1 group using the Fisher exact test.
Letting x and m denote the frequency and sample
size of the 1 group, and y and n denote those of the
1 group, the pvalue is calculated as
where X and Y are independent binomially distributed random
variables with sample sizes m and n and common probability
parameters. The hypergeometric distribution is used to determine
the stated probability; Yates (1984) discusses this
technique. PROC MULTTEST computes the twotailed pvalues by
adding probabilities from both tails of the hypergeometric
distribution. The first tail is from the observed x and y,
and the other tail is chosen so that the resulting probability
is as large as possible without exceeding the probability from
the first tail.
tTest for the Mean
For continuous variables, PROC MULTTEST automatically centers
the trend coefficients, as in the FreemanTukey test. These
centered coefficients c_{g} are then used to form a
tstatistic contrasting the withingroup means. Let n_{vgs}
denote the sample
size within group g and stratum s; it
depends on variable v only when there are missing values.
Define
as the sample mean within a groupandstratum combination, and
define
as the pooled sample variance. Assume constant variance for
all groupandstratum combinations. Then the tstatistic
for the mean is
and is weighted by the withinstrata sample size (n_{v+s}) to
ensure comparability with the CA trend and FreemanTukey
statistics.
Let denote the treatment means. Then
under the null hypothesis that
and assuming normality, independence, and homoscedasticity, M_{v}
follows a tdistribution with degrees of freedom.
Whenever a denominator of 0 is computed, the
pvalue is set to 1. When missing data force n_{vgs} = 0,
then the contribution to the denominator of the pooled variance
is 0 and not 1. This is also true for degrees of freedom.
Copyright © 1999 by SAS Institute Inc., Cary, NC, USA. All rights reserved.