src="More about the theoretical underpinnings of the Bootstrap.files/img163.png" 
width=20 align=bottom border=0>, and we write <BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}Y_n \stackrel{\cal{ L}}{\longrightarrow} Y\end{displaymath} --><IMG 
height=31 
alt="\begin{displaymath}Y_n \stackrel{\cal{ L}}{\longrightarrow} Y\end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img188.png" 
width=75 border=0> </DIV><BR clear=all>
<P></P>This does not mean that <IMG height=35 alt=$Y_n$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img187.png" 
width=24 align=middle border=0> and <IMG height=16 alt=$Y$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img163.png" 
width=20 align=bottom border=0> are arbitrarily close, think of the random 
variables <IMG height=37 alt="$U \sim U(0,1)$" 
src="More about the theoretical underpinnings of the Bootstrap.files/img189.png" 
width=101 align=middle border=0> and <IMG height=35 alt=$1-U$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img190.png" 
width=52 align=middle border=0>. 
<P><FONT color=#ff0000>Convergence in Probability</FONT> <BR><BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}Y_n \stackrel{P}{\longrightarrow} Y \quad\forall \epsilon, P(|Y_n-c| < \epsilon) \longrightarrow 1\end{displaymath} --><IMG 
height=33 
alt="\begin{displaymath}Y_n \stackrel{P}{\longrightarrow} Y \quad&#10;\forall \epsilon, P(\vert Y_n-c\vert < \epsilon) \longrightarrow 1\end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img191.png" 
width=294 border=0> </DIV><BR clear=all>
<P></P>
<P>Note: <BR>If 
<!-- MATH $k_n Y_n \stackrel{\cal{ L}}{\longrightarrow} H$ --><IMG height=50 
alt="$k_n Y_n \stackrel{\cal{ L}}{\longrightarrow} H$" 
src="More about the theoretical underpinnings of the Bootstrap.files/img192.png" 
width=102 align=middle border=0> where <IMG height=16 alt=$H$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img185.png" 
width=22 align=bottom border=0> is a limit distribution and <!-- MATH $k_n\longrightarrow \infty$ --><IMG height=35 
alt="$k_n\longrightarrow \infty$" 
src="More about the theoretical underpinnings of the Bootstrap.files/img193.png" 
width=84 align=middle border=0> then <!-- MATH $Y_n \stackrel{P}{\longrightarrow} 0$ --><IMG height=50 
alt="$Y_n \stackrel{P}{\longrightarrow} 0$" 
src="More about the theoretical underpinnings of the Bootstrap.files/img194.png" 
width=75 align=middle border=0>. 
<H3><A name=SECTION00291200000000000000>Why is the empirical cdf <IMG height=45 
alt=$\hat{F}_n$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img16.png" 
width=26 align=middle border=0> a good estimator of F?</A> </H3>We showed in 
class that for fixed real <IMG height=16 alt=$a$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img195.png" 
width=14 align=bottom border=0> <BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}\sqrt{n}(\hat{F}_n(a)-F(a)) \stackrel{\cal{ L}}{\longrightarrow} \NN (0,F(a)(1-F(a)))\end{displaymath} --><IMG 
height=33 
alt="\begin{displaymath}\sqrt{n}(\hat{F}_n(a)-F(a)) \stackrel{\cal{ L}}{\longrightarrow} \NN (0,F(a)(1-F(a)))\end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img196.png" 
width=345 border=0> </DIV><BR clear=all>
<P></P>Because of the result noted above, this also ensures that <!-- MATH $\hat{F}_n(a) \stackrel{P}{\longrightarrow} F(a)$ --><IMG height=50 
alt="$\hat{F}_n(a) \stackrel{P}{\longrightarrow} F(a)$" 
src="More about the theoretical underpinnings of the Bootstrap.files/img197.png" 
width=131 align=middle border=0>, this is actually true uniformly in <IMG 
height=16 alt=$a$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img195.png" 
width=14 align=bottom border=0> because Kolmogorovs statistic is pivotal <BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}d(\hat{F}_n,F)=sup_{x} |\hat{F}_n(x)-F(x)| =D_n\end{displaymath} --><IMG 
height=33 
alt="\begin{displaymath}d(\hat{F}_n,F)=sup_{x} \vert\hat{F}_n(x)-F(x)\vert =D_n \end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img198.png" 
width=299 border=0> </DIV><BR clear=all>
<P></P>has a distribution that does not depend on <IMG height=16 alt=$F$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img1.png" 
width=19 align=bottom border=0>. 
<P>Definition: <BR>A statsitic is said to be pivotal if its distribution does 
not depend on any unknown parameters. 
<P>Example: Student's <IMG height=16 alt=$t$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img199.png" 
width=11 align=bottom border=0> statistic. 
<H3><A name=SECTION00291300000000000000>Generalized Statistical Functionals</A> 
</H3>When we want to evaluate an estimator, construct confidence intervals, 
etc.. we are usually interested in evaluating quantities that are functions of 
both the unknown distribution <IMG height=16 alt=$F$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img1.png" 
width=19 align=bottom border=0>, the empirical <IMG height=45 alt=$\hat{F}_n$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img16.png" 
width=26 align=middle border=0> and the sample size, here are some examples: 
<OL>
  <LI>The sampling distribution of the error: <BR>
  <P></P>
  <DIV align=center><!-- MATH \begin{displaymath}\lambda_n(F,\hat{F}_n)=P_F(\sqrt{n}(\theta(\hat{F}_n)-\theta(F)))\end{displaymath} --><IMG 
  height=33 
  alt=\begin{displaymath}\lambda_n(F,\hat{F}_n)=P_F(\sqrt{n}(\theta(\hat{F}_n)-\theta(F)))\end{displaymath} 
  src="More about the theoretical underpinnings of the Bootstrap.files/img200.png" 
  width=285 border=0> </DIV><BR clear=all>
  <P></P>
  <LI>The bias: <BR>
  <P></P>
  <DIV align=center><!-- MATH \begin{displaymath}\lambda_n(F,\hat{F}_n)=E_F(\theta(\hat{F}_n))-\theta(F)\end{displaymath} --><IMG 
  height=33 
  alt=\begin{displaymath}\lambda_n(F,\hat{F}_n)=E_F(\theta(\hat{F}_n))-\theta(F)\end{displaymath} 
  src="More about the theoretical underpinnings of the Bootstrap.files/img201.png" 
  width=245 border=0> </DIV><BR clear=all>
  <P></P>
  <LI>The standard error: <BR>
  <P></P>
  <DIV align=center><!-- MATH \begin{displaymath}\lambda_n(F,\hat{F}_n)=\sqrt{E_F(\theta(\hat{F}_n)-\theta(F))^2}\end{displaymath} --><IMG 
  height=36 
  alt="\begin{displaymath}\lambda_n(F,\hat{F}_n)=\sqrt{E_F(\theta(\hat{F}_n)-\theta(F))^2}&#10;\end{displaymath}" 
  src="More about the theoretical underpinnings of the Bootstrap.files/img202.png" 
  width=271 border=0> </DIV><BR clear=all>
  <P></P></LI></OL>For each of these examples, what the bootstrap proposes is to 
replace <IMG height=16 alt=$F$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img1.png" 
width=19 align=bottom border=0> by the empirical <IMG height=45 alt=$\hat{F}_n$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img16.png" 
width=26 align=middle border=0>. 
<P>The bootstrap is said to <FONT color=#ff0000>work</FONT> if <BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}\lambda_n(\hat{F}_n,\hat{F}_n^*)-\lambda_n(F,\hat{F}_n)\stackrel{P}{\longrightarrow} 0\end{displaymath} --><IMG 
height=33 
alt="\begin{displaymath}\lambda_n(\hat{F}_n,\hat{F}_n^*)-\lambda_n(F,\hat{F}_n)\stackrel{P}{\longrightarrow} 0&#10;\end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img203.png" 
width=236 border=0> </DIV><BR clear=all>
<P></P>
<H2><A name=SECTION00292000000000000000>Example and Counterexample</A> </H2>
<H3><A name=SECTION00292100000000000000>Bootstrap of the maximum</A> 
</H3>Suppose we have a random variable <IMG height=16 alt=$X$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img162.png" 
width=22 align=bottom border=0> uniformly distributed on <IMG height=37 
alt=$(0,\theta)$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img204.png" 
width=46 align=middle border=0> where <IMG height=17 alt=$\theta$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img10.png" 
width=14 align=bottom border=0> is the unkown parameter that we wish to estimate 
and whose sampling distribution we would like to know. 
<H4><A name=SECTION00292110000000000000>Theoretical Analysis</A> </H4>We showed 
in class that if we take the largest value of a sample of size <IMG height=16 
alt=$n$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img28.png" 
width=16 align=bottom border=0> to be the estimate of <IMG height=17 
alt=$\theta$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img10.png" 
width=14 align=bottom border=0>, <!-- MATH $\hat{\theta}=X_{(n)}$ --><IMG 
height=45 alt=$\hat{\theta}=X_{(n)}$ 
src="More about the theoretical underpinnings of the Bootstrap.files/img205.png" 
width=74 align=middle border=0>, then <BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}P[\theta-c<X_{(n)}<\theta]=1-P[X_{(n)}<\theta-c]=1-(\frac{\theta-c}{\theta})^n\end{displaymath} --><IMG 
height=46 
alt="\begin{displaymath}&#10;P[\theta-c<X_{(n)}<\theta]=1-P[X_{(n)}<\theta-c]=1-(\frac{\theta-c}{\theta})^n&#10;\end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img206.png" 
width=468 border=0> </DIV><BR clear=all>
<P></P>so that 
<!-- MATH $X_{(n)} \stackrel{P}{\longrightarrow} \theta$ --><IMG height=50 
alt="$X_{(n)} \stackrel{P}{\longrightarrow} \theta$" 
src="More about the theoretical underpinnings of the Bootstrap.files/img207.png" 
width=90 align=middle border=0> 
<P>As for the convergence in law: <BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}P[n(\theta-X_{(n)})\leq x]=(1-\frac{x}{\theta n})^n\end{displaymath} --><IMG 
height=41 
alt="\begin{displaymath}&#10;P[n(\theta-X_{(n)})\leq x]=(1-\frac{x}{\theta n})^n&#10;\end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img208.png" 
width=258 border=0> </DIV><BR clear=all>
<P></P>and <BR>
<P></P>
<DIV align=center><!-- MATH \begin{displaymath}Distribution(X_{(n)})\longrightarrowH(x)=1-e^{-\frac{x}{\theta}},\mbox{ as } n\longrightarrow \infty\end{displaymath} --><IMG 
height=34 
alt="\begin{displaymath}Distribution(X_{(n)})\longrightarrow&#10;H(x)=1-e^{-\frac{x}{\theta}}&#10;,\mbox{ as } n\longrightarrow \infty&#10;\end{displaymath}" 
src="More about the theoretical underpinnings of the Bootstrap.files/img209.png"