Lobachevskii Journal of Mathematics http://ljm.ksu.ru Vol. 18, 2005, 3–20

© S. E. Ahmed, D. Li, A. Rosalsky, and A. Volodin

Abstract. In this paper, the asymptotic probability for the deviations of dependent bootstrap means from the sample mean is obtained without imposing any conditions on the joint distributions associated with the original sequence of random variables from which the dependent bootstrap sample is selected. The mild condition of stochastic domination by a random variable is imposed on the marginal distributions of the random variables in this sequence.

1. Introduction.

It is a great pleasure for us to contribute this paper in honour of Professor Daniar Khamidovich Mushtari on the occasion of his 60th birthday. The main focus of the present investigation is to obtain asymptotic results for the probability of the deviations of dependent bootstrap means from the sample mean.

The work on the validity of bootstrap estimators has received much attention in recent years due to a growing demand for the procedure, both theoretically and practically. As is mentioned in Mikosch (1994), the sample mean is fundamental for parameter estimation in statistics. Therefore, most of the recent literature on the bootstrap is devoted to statistics of this type. This literature is mainly concerned with bootstrap validity; that is, with showing that a statistic and its bootstrap version have the similar asymptotic distributional behaviour.

However, the limiting behaviour of bootstrap statistics is also of interest since it is by no means clear whether the bootstrap version of a consistent estimator is itself consistent. From our point of view, this explains the usefulness and impact on statistical inference of deviations from the sample means for “exogenously generated” bootstrap samples. Furthermore, asymptotic probabilities for the deviations of bootstrap means are a quite useful tool for the study of bootstrap moments. It is important to note that exponential inequalities are of practical use in establishing the strong asymptotic validity of bootstrap means.

We call the reader’s attention to the special issue of the journal Statistical Science (2003) Volume 18, Number 2 devoted to the Silver Anniversary of the Bootstrap, where the wide applications of the bootstrap procedure to diverse areas of statistics are discussed.

We begin with a brief discussion of results in the literature pertaining to a sequence of independent and identically distributed (i.i.d.) random variables and to the classical (independent) bootstrap of the mean. Let $\left\{X,X_n,n\ge 1\right\}$ be a sequence of i.i.d. random variables defined on a probability space $\left(\Omega ,\mathcal{ℱ},P\right)$. For $\omega \in \Omega$ and $n\ge 1$, let $P_n\left(\omega \right)=n^{-1}{\sum }_{i=1}^n{\delta }_{X_i\left(\omega \right)}$ denote the empirical measure and let $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ be i.i.d. random variables with law $P_n\left(\omega \right)$ where $\left\{m\left(n\right),n\ge 1\right\}$ is a sequence of positive integers. In other words, the random variables $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ result by sampling $m\left(n\right)$ times with replacement from the $n$ observations $X_1\left(\omega \right),\cdots ,X_n\left(\omega \right)$ such that for each of the $m\left(n\right)$ selections, each $X_j\left(\omega \right)$ has probability $n^{-1}$ of being chosen.

For each $n\ge 1,\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ is the so-called Efron (1979) independent bootstrap sample from $X_1,\cdots ,X_n$ with bootstrap sample size $m\left(n\right)$. Let ${\overline{X}}_n\left(\omega \right)=\frac{1}{n}{\sum }_{j=1}^n{X}_j\left(\omega \right)$ denote the sample mean of $\left\{X_j\left(\omega \right),1\le j\le n\right\},n\ge 1$.

When $X$ is nondegenerate and $E{X}^2<\infty$, Bickel and Freedman (1981) showed that for almost every $\omega \in \Omega$ the central limit theorem (CLT)

obtains. Here and below and ${\sigma }^2=$ Var $X$. Note that by the Glivenko-Cantelli theorem $P_n\left(\omega \right)$ is close to $L\left(X\right)$ for almost every $\omega \in \Omega$ and all large $n$, and by the classical Lévy CLT

It follows that for almost every $\omega \in \Omega$, the bootstrap statistic

is close in distribution to that of

for all large $n$. This is the basic idea behind the bootstrap. See the pioneering work of Efron (1979) where this nice idea is made explicit and where it is substantiated with several important examples.

A strong law of large numbers (SLLN) was first proved by Athreya (1983) for bootstrap means from the classical bootstrap. Arenal-Gutiérrez, Matrán, and Cuesta-Albertos (1996) analyzed the work of Athreya (1983) and, by taking into account different growth rates for the bootstrap sample size $m\left(n\right)$, they gave new and simple proofs of even more general results. They also provided examples that show that the sizes of resampling required by their results to ensure almost sure (a.s.) convergence are not far from optimal.

An article which is important for this paper is that of Mikosch (1994). He established a series of exponential inequalities (cf. Lemma 6 below) that are an important tool for deriving results on the consistency of the bootstrap mean. Based on these exponential inequalities, he proved an a.s. convergence result for bootstrap means (Theorem 1 below). Next, using the same exponential inequalities, the Baum - Katz / Erd$\ddot{o}$s / Hsu - Robbins / Spitzer type complete convergence result for bootstrap means (Theorem 2 below) and a moment result for the supremum of normed bootstrap sums were established in Li, Rosalsky, and Ahmed (1999).

The following bootstrap counterpart to the Marcinkiewicz-Zygmund SLLN was obtained by Mikosch (1994), Proposition 3.3.

Theorem 1. Let $\left\{X_n,n\ge 1\right\}$ be a sequence of i.i.d. random variables and let $0<\alpha <2$. If $m\left(n\right)\equiv n$ and

then for almost every $\omega \in \Omega$

where $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ is Efron’s independent bootstrap sample from $X_1,\cdots ,X_n$.

We note that  the classical Efron  independent  bootstrap sample $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right),n\ge 1\right\}$ can of course be defined in the same manner even if the original sequence $\left\{X_n,n\ge 1\right\}$ is not comprised of independent or identically distributed random variables. The following result was proved by Li, Rosalsky, and Ahmed (1999), Theorem 2.1 and Remark 2.4.

Theorem 2. Let $\left\{X_n,n\ge 1\right\}$ be a sequence of pairwise i.i.d. random variables and let $0<\alpha <2$. If $m\left(n\right)\equiv n$ and

then for every real number $q$, every $\epsilon >0$, and almost every $\omega \in \Omega$

where $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ is Efron’s independent bootstrap sample from $X_1,\cdots ,X_n$.

Remark 1. Taking $q=0$, it follows from the Borel-Cantelli lemma and Theorem 2 that for almost every $\omega \in \Omega$,

We also refer the reader to the recent expository paper by Cs$\ddot{o}$rgő and Rosalsky (2003) where a detailed and comprehensive survey of limit laws for bootstrap sums is given.

The notion of the dependent bootstrap procedure was introduced by Smith and Taylor (2001a and 2001b) for a sequence of i.i.d. random variables where some important properties were also established. However, the dependent bootstrap procedure can be defined as follows for an arbitrary sequence of random variables. Let $\left\{X_n,n\ge 1\right\}$ be a sequence of random variables (which are not necessarily independent or identically distributed) defined on a probability space $\left(\Omega ,\mathcal{ℱ},P\right)$. Let $\left\{m\left(n\right),n\ge 1\right\}$ and $\left\{k\left(n\right),n\ge 1\right\}$ be two sequences of positive integers such that $m\left(n\right)\le nk\left(n\right)$ for all $n\ge 1$. For $\omega \in \Omega$ and $n\ge 1$, the dependent bootstrap is defined to be the sample of size $m\left(n\right)$, denoted $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$, drawn without replacement from the collection of $nk\left(n\right)$ items made up of $k\left(n\right)$ copies each of the sample observations $X_1\left(\omega \right),\cdots ,X_n\left(\omega \right)$.

This dependent bootstrap procedure is proposed as a procedure to reduce variation of estimators and to obtain better confidence intervals. We refer to Smith and Taylor (2001b) for details and where simulated confidence intervals are obtained to examine possible gains in coverage probabilities and interval lengths.

We may consider the dependent bootstrap procedure as a more general procedure than the classical Efron independent bootstrap. If we take $k\left(n\right)=\infty$ for all $n\ge 1$, then the dependent bootstrap reduces to the classical Efron independent bootstrap. The main results presented in this paper do not require any assumptions on $k\left(n\right)$; they are certainly true for the independent bootstrap as well.

Henceforth we let $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ denote the dependent bootstrap sample from $X_1,\cdots ,X_n$.

The following result from Volodin, Ordóñez Cabrera, and Hu (2005) extends the above cited result of Li, Rosalsky, and Ahmed (1999) to the case of the dependent bootstrap. The content of Remark 1 also pertains to Theorem 3.

Theorem 3. Let $\left\{X_n,n\ge 1\right\}$ be a sequence of identically distributed (not necessary independent) random variables and let $0<\alpha <2$. If

then for every real number $q$, every $\epsilon >0$, and almost every $\omega \in \Omega$

The initial objective of the investigation resulting in the present paper was only to extend the results of Hu, Ordóñez Cabrera, and Volodin (2005) on the SLLN to the dependent bootstrap of the mean. But we are even able to establish in Theorem 4 a more general result than Theorem 3. Notice that Theorem 3 has the moment assumption $E\mid X_1{\mid }^{\alpha }\mid log\mid X_1\mid {\mid }^{\alpha }<\infty$ whereas our result has much more general moment assumption. The no independence condition in Theorem 3 is noteworthy and it also prevails in Theorem 4; the identical distributions assumption is relaxed in Theorem 4 to stochastic domination by a random variable.

The following notion is well known. We recall that a sequence of random variables $\left\{X_n,n\ge 1\right\}$ is stochastically dominated by a random variable $X$ if there exists a constant $C>0$ such that

for all $t\ge 0$ and all $n\ge 1$.

The main focus of this paper is to obtain in Theorem 4 asymptotic results for

as $n\to \infty$ where $\epsilon >0$, $0<\alpha <2$, and $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ is the dependent bootstrap sample from $X_1,\cdots ,X_n$. The sequence $\left\{X_n,n\ge 1\right\}$ is not necessary a sequence of independent random variables but it is assumed to be stochastically dominated.

2. Some general results on the dependent bootstrap

The results from this section are modifications, generalizations, or extensions of the results of Smith and Taylor (2001a and 2001b) for the dependent bootstrap from a sequence of random variables $\left\{X_n,n\ge 1\right\}$ which are not necessarily i.i.d. We note again that Smith and Taylor (2001a and 2001b) consider only the i.i.d. case. The results in this section are of general interest and play a role in establishing the asymptotic results discussed above.

The first proposition gives the joint distribution of the random variables in the dependent bootstrap sample. We need the following notation.

For $\omega \in \Omega ,n\ge 1,$ and a real number $x$, denote

where $I\left(\cdot \right)$ is the indicator function. Hence, $\tau \left(x\right)$ is the random variable that counts the number of observations $X_j\left(\omega \right),1\le j\le n$, that are less than or equal to $x$, while $\mu \left(x\right)$ is the random variable that counts the number of observations $X_j\left(\omega \right),1\le j\le n$, that are strictly greater than $x$. Certainly, $\tau \left(x\right)+\mu \left(x\right)=n$ for every $x$.

For a finite sequence $\left\{x_1,x_2,\cdots ,x_m\right\}$ of real numbers, denote

its nondecreasing rearrangement, that is $x_{\left(1\right)}\le x_{\left(2\right)}\le \cdots \le x_{\left(m\right)}$ and for any $1\le j\le m$ there exists $1\le i\le m$ such that $x_i=x_{\left(j\right)}$.

Proposition 1. For $\omega \in \Omega ,n\ge 1,$ and a sequence $\left\{x_1,x_2,\cdots ,x_{m\left(n\right)}\right\}$ of real numbers:

1) If $k\left(n\right)\tau \left(x_{\left(j\right)}\right)\ge j$ for all $1\le j\le m\left(n\right)$, then

If $k\left(n\right)\tau \left(x_{\left(j\right)}\right)<j$ for at least one $1\le j\le m\left(n\right),$ then the above probability is 0.

2) If $k\left(n\right)\mu \left(x_{\left(i\right)}\right)\ge m\left(n\right)-i$ for all $1\le i\le m\left(n\right)$, then

If $k\left(n\right)\mu \left(x_{\left(i\right)}\right)<m\left(n\right)-i$ for at least one $1\le i\le m\left(n\right),$ then the above probability is 0.

Proof. Let $\pi$ be the reordering of $\left\{1,2,\cdots ,m\left(n\right)\right\}$ such that $\pi \left(j\right)=i$ for $x_i=x_{\left(j\right)}$.

For the proof of the first statement of Proposition 1, note that

if $k\left(n\right)\tau \left(x_{\left(j\right)}\right)\ge j$ for all $1\le j\le m\left(n\right)$. The second part of the first statement is obvious.

For the proof of the second statement of Proposition 1, note that

if $k\left(n\right)\mu \left(x_{\left(i\right)}\right)\ge m\left(n\right)-i$ for all $1\le i\le m\left(n\right)$. The second part of the second statement is obvious.

Of course,   the dependent bootstrap random variables $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ are indeed dependent. They obey the so-called negatively dependent property; this property will be established in Proposition 2. The concept of negatively dependent random variables was introduced by Lehmann (1966) as follows.

Random variables $Y_1,Y_2,\cdots$ are said to be negatively dependent if for each $n\ge 2$, the following two inequalities hold:

and

for every sequence $\left\{y_1,\cdots ,y_n\right\}$ of real numbers.

Proposition 2. For $\omega \in \Omega$ and $n\ge 1$, the dependent bootstrap random variables $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\}$ are negatively dependent and exchangeable.

Proof. Let $\left\{x_1,x_2,\cdots ,x_{m\left(n\right)}\right\}$ be a sequence of real numbers. For the first inequality of the negative dependence property, we note that we only need to consider the case $k\left(n\right)\tau \left(x_{\left(j\right)}\right)\ge j$ for all $1\le j\le m\left(n\right)$. By Proposition 1(1)

For the second inequality of the negative dependence property, we note that we only need to consider the case $k\left(n\right)\mu \left(x_{\left(i\right)}\right)\ge m\left(n\right)-i$ for all $1\le i\le m\left(n\right)$. By Proposition 1(2)

The exchangeability is obvious by Proposition 1.

3. Some technical lemmas

In this section we present six technical results that we will use in establishing the main result of this paper including its corollaries. Some of the lemmas are only generalizations and extensions of well-known results. For expository purposes we outline many of their proofs.

For the simplicity, by the $log$-function in this section we mean the natural logarithm function. The results can be easily generalized to any other logarithm function with base greater than one.

The first lemma is well known and trivial.

Lemma 1. Let $\left\{Y_n,n\ge 1\right\}$ be a sequence of negatively dependent random variables.

1) If $\left\{f_n,n\ge 1\right\}$ is a sequence of real functions all of which are monotone increasing (or all monotone decreasing), then $\left\{f_n\left(Y_n\right),n\ge 1\right\}$ is a sequence of negatively dependent random variables.

2) For every $n\ge 1,E\left(\prod _{j=1}^n{Y}_j\right)\le \prod _{j=1}^{n}E\left(Y_j\right)$ provided the expectations are finite.

Proof. 1) Let $f_n^{-1}$ denote the inverse function of $f_n,n\ge 1$ and assume that all $f_n$ are increasing. Then for any $n\ge 1$ and all real $y_1,\cdots ,y_n$ we have by the definition of negative dependence that

The second inequality also follows from the definition of negative dependence. The case of decreasing functions can be proved in the same manner.

2) Consider the expectation

The next lemma is in effect the special case $a_n\equiv 1$ of Theorem 2 of Adler and Rosalsky (1987) and we omit the proof. The random variables $\left\{Y_n,n\ge 1\right\}$ are not assumed to be independent.

Lemma 2. Let $\phi \left(t\right),t>0$, be a continuous function that is positive, strictly increasing and satisfying the condition $\phi \left(t\right)\to \infty$ as $t\to \infty$. Put $b_n={\phi }^{-1}\left(n\right),n\ge 1$, where ${\phi }^{-1}\left(t\right)$ is the inverse function of $\phi \left(t\right)$. Let $\left\{Y_n,n\ge 1\right\}$ be a sequence of random variables which is stochastically dominated by a random variable $Y$. If

then

The third lemma deals with convergence of maxima of random variables and is a generalization of the Corollary to Theorem 3 of Barnes and Tucker (1977). Again, no assumption of independence is made.

Lemma 3. Let $\psi \left(t\right),t\ge 0$ be an increasing function such that $\psi \left(t\right)\to \infty$ as $t\to \infty$ and let $\left\{b_n,n\ge 1\right\}$ be a sequence of positive numbers such that $b_n={\psi }^{-1}\left(n\right),n\ge 1$, , where ${\psi }^{-1}\left(t\right)$ is the inverse function of $\psi \left(t\right)$. Let $\left\{X_n,n\ge 1\right\}$ be a sequence of positive random variables which is stochastically dominated by a random variable $X$ such that $E\psi \left(CX\right)<\infty$ for all $C>0$. Then

Proof. We recall at the outset that for any random variable $Y$, the conditions $E\mid Y\mid <\infty$ and ${\sum }_{n=1}^{\infty }P\left\{\mid Y\mid \ge n\right\}<\infty$ are equivalent. Hence the assumption

is equivalent to

Then by the stochastic domination hypothesis and the Borel-Cantelli lemma $X_n∕b_n\to 0$ a.s. For arbitrary $n\ge k\ge 2$,

as first $n\to \infty$ and then $k\to \infty .$

Unfortunately, it is not possible to find the inverse to the function $\phi \left(t\right)=t^{1∕\beta }∕{log}^{\gamma }t,t>0,\beta >0$, and $\gamma >0$ in closed form. But the following lemma gives a good “approximation” to the inverse function.

Lemma 4. Let $\phi \left(t\right)=t^{1∕\beta }{log}^{-\gamma ∕\beta }t$ and $\psi \left(t\right)=t^{\beta }{log}^{\gamma }t,t\ge e,\beta >0$, and $0<\gamma <e$. Then for any $\epsilon >0$ and for all sufficiently large $t$

and, consequently, for all sufficiently large $t$

Proof. Note that

and

for $t\ge e^e$ which can be established by the differentiation.

It follows from Lemma 4 that for a positive random variable $Y$, the conditions $E{\phi }^{-1}\left(Y\right)<\infty$ and $E\psi \left(Y\right)<\infty$ are equivalent.

Lemma 5. Let $\beta >1$ and $b_n=n^{\beta }{log}^{-2}n,n\ge 2$, then

Proof. Note that for all large $n$, letting $C_i\left(1\le i\le 4\right)$ denote positive constants,

The exponential inequality presented in the last lemma is the key tool used in establishing in Theorem 4 the asymptotic probability for the deviations of dependent bootstrap means from the sample mean. It is a dependent bootstrap analog of the Mikosch exponential inequality (Mikosch (1994), Lemma 5.1). We mention that this result was proved by Mikosch (1994) under the assumption that $\left\{X_n,n\ge 1\right\}$ is a sequence of i.i.d. random variables, for supremum (not partial sums) of bootstrap random variables, and for the independent bootstrap procedure.

Lemma 6. Let $\left\{a_n,n\ge 1\right\}$ and $\left\{h_n,n\ge 1\right\}$ be two sequences of positive real numbers and let $\left\{X_n,n\ge 1\right\}$ be a sequence of (not necessary independent or identically distributed) random variables. Then for $\omega \in \Omega$ and $n\ge 1$ such that $h_n{M}_n\left(\omega \right)<1$ the following inequality holds for all $\epsilon >0$:

where

and

Proof. By the Markov inequality,

We will estimate only the expectation in the first term of the last expression; the same bound is valid for the second expectation.

Now  by Proposition 2  the dependent  bootstrap  random variables $\left\{{\widehat{X}}_{n,j}^{\left(\omega \right)},1\le j\le m\left(n\right)\right\},n\ge 1$, are negatively dependent and exchangeable. Hence, by Lemma 1(1) the random variables

are negatively dependent and identically distributed.

Therefore