Section01_参数估计

总体 XX 的分布中，含有未知参数 θ\theta，如何估计 θ\theta ?

矩估计

总体 XX, 样本 X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n}

  1. 样本矩→P总体矩\text{样本矩} \xrightarrow{P} \text{总体矩}
    1. Xˉ=1n∑i=1nXi→PEX\displaystyle \bar{X} = \frac{1}{n}\sum_{i=1}^{n}X_{i} \xrightarrow{P} EX
    2. A2=1n∑i=1nXi2→PEX2\displaystyle A_{2} = \frac{1}{n}\sum_{i=1}^{n}X_{i}^{2} \xrightarrow{P} EX^{2}
    3. B2=1n∑i=1n(Xi−Xˉ)2=1n∑i=1nXi2−(1n∑i=1nXi)2→PDX\displaystyle B_{2} = \frac{1}{n}\sum_{i=1}^{n}(X_{i}-\bar{X})^{2}= \frac{1}{n} \sum_{i=1}^{n}X^{2}_{i} - \bigg(\frac{1}{n}\sum_{i=1}^{n}X_{i}\bigg)^{2} \xrightarrow{P} DX
  2. 用样本矩去代替相应的总体矩 (此处的 "==" 实质上为 "≈\approx" )
    1. Xˉ=EX\bar{X} = EX
    2. A2=EX2A_{2} = EX^{2}
    3. B2=DXB_{2} = DX
    4. 注 原则上优先使用一阶矩来进行估计

最大似然估计

  1. 似然函数 L(θ)L(\theta) 指样本取到观测值的概率 L(θ)={P{X1=x1,X2=x2,⋯ ,Xn=xn},离散型P{X1∈U(x1),X2∈U(x2),⋯ ,Xn∈U(xn)},连续型 L(\theta) = \begin{cases} \mathbb{P}\{X_{1}= x_{1},X_{2}=x_{2},\cdots,X_{n} = x_{n}\},&\text{离散型} \\ \mathbb{P}\{X_{1}\in U(x_{1}), X_{2}\in U(x_{2}),\cdots, X_{n}\in U(x_{n})\},&\text{连续型} \end{cases}
    • 例1 设总体 X∼(123θ22θ(1−θ)(1−θ)2)X\sim \left( \begin{matrix} 1 & 2 & 3\\ \theta^{2} & 2\theta(1-\theta) & (1-\theta)^{2} \end{matrix} \right)，其中 θ (0<θ<1)\theta\ (0<\theta<1) 为未知参数，已知取得了样本值，X1=1,X2=2,X3=1X_{1} = 1, X_{2}=2, X_{3} = 1，则 θ\theta 的似然函数为 L(θ)=P{X1=x1,X2=x2,X3=x3}=P{X1=1,X2=2,X3=1}=P{X1=1}P{X2=2}P{X3=1}=θ2⋅2θ(1−θ)⋅θ2 \begin{split} & L(\theta) = \mathbb{P}\{X_{1}=x_{1},X_{2}=x_{2},X_{3} = x_{3}\} \\ = & \mathbb{P}\{X_{1}=1,X_{2} = 2, X_{3} = 1\} \\ = & \mathbb{P}\{X_{1} =1\}\mathbb{P}\{X_{2} =2\}\mathbb{P}\{X_{3} =1\} \\ = & \theta^{2}\cdot2\theta(1-\theta)\cdot \theta^{2} \end{split}
    • 例2 设 (X1,⋯ ,Xn)(X_{1},\cdots,X_{n}) 是总体 XX 的一个样本，XX 的概率密度为 f(x)={2xθ2,0<x<θ0,其他f(x) = \begin{cases} \frac{2x}{\theta^{2}}, & 0<x<\theta \\ 0, & \text{其他} \end{cases}，则未知参数的 θ\theta 的似然函数为 L(θ)=P{X1∈U(x1),X2∈U(x2),⋯ ,Xn∈U(xn)}=∏i=1nP{Xi∈U(xi)}=∏i=1nfX(xi)⋅Δxi=2nx1x2⋯xnθ2n×∏i=1nΔxi=2nx1x2⋯xnθ2n \begin{split} & L(\theta) = \mathbb{P}\{X_{1}\in U(x_{1}), X_{2}\in U(x_{2}),\cdots, X_{n}\in U(x_{n})\} \\ =& \prod_{i=1}^{n}\mathbb{P}\{X_{i}\in U(x_{i})\} \\ = & \prod_{i=1}^{n}f_{X}(x_{i})\cdot \Delta x_{i} \\ = & \frac{2^{n}x_{1}x_{2}\cdots x_{n}}{\theta^{2n}}\times \prod_{i=1}^{n}\Delta x_{i}\quad \\ = & \frac{2^{n}x_{1}x_{2}\cdots x_{n}}{\theta^{2n}} \end{split}
      • 注 因为之后需要计算 θ\theta 取何值时，L(θ)L(\theta) 取得最大值，而 ∏i=1nΔxi\prod\limits_{i=1}^{n}\Delta x_{i} 与之无关，故可忽略
  2. 思想 在 θ\theta 取值范围内，找到 θ^\hat{\theta} 使得 L(θ)L(\theta) 最大
  3. 步骤
    1. 写 L(θ)L(\theta)
    2. 找出 L(θ)L(\theta) 的最大值 dln⁡L(θ)dθ=0\color{#D0104C} \frac{d \ln L(\theta)}{d\theta} = 0
    3. 两种情况
      1. 若驻点唯一为 θ0\theta_{0}，则 θ^L=θ0\hat{\theta}_{L} = \theta_{0}
      2. 无驻点，观察其单调性和取值范围，找出 θ^L\hat{\theta}_{L}

例题

  1. 设 (X1,⋯ ,Xn)(X_{1},\cdots,X_{n}) 是总体 XX 的一个样本，XX 的概率密度为 f(x)={2xθ2,0<x<θ0,其他f(x) = \begin{cases} \frac{2x}{\theta^{2}}, & 0<x<\theta \\ 0, & \text{其他} \end{cases}，则未知参数的 θ\theta 求 θ\theta 的矩估计量和最大似然估计量 矩估计∵Xˉ=EX=∫0θx⋅2xθ2⋅dx=2x33θ2∣0θ∴23θ=Xˉ⇒θ^M=32Xˉ最大似然估计L(θ)=P{X1∈U(x1),X2∈U(x2),⋯ ,Xn∈U(xn)}=2x1θ22x2θ2⋯2xnθ2=2nx1x2⋯xnθ2n∴ln⁡L(θ)=nln⁡2+∑i=1nln⁡xi−2nln⁡θ∴dln⁡L(θ)dθ=−2nθ∵0<x<θ∴dln⁡L(θ)dθ<0, 即 L(θ)单调递减∴θ^L=max⁡1≤i≤n{Xi}时, L(θ)去最大值∴θ^L=max⁡1≤i≤n{Xi} \begin{array}{ll} & \text{矩估计} \\ \because & \displaystyle \bar{X} = EX = \int_{0}^{\theta}x\cdot \frac{2x}{\theta^{2}}\cdot dx = \left.\frac{2x^{3}}{3\theta^{2}}\right\vert^{\theta}_{0} \\ \therefore & \frac{2}{3}\theta = \bar{X} \Rightarrow \hat{\theta}_{M} = \frac{3}{2}\bar{X}\\\\ & \text{最大似然估计} \\ & L(\theta) = \mathbb{P}\{X_{1}\in U(x_{1}), X_{2}\in U(x_{2}),\cdots, X_{n}\in U(x_{n})\} \\ & = \frac{2 x_{1}}{\theta^{2}}\frac{2 x_{2}}{\theta^{2}}\cdots \frac{2 x_{n}}{\theta^{2}} = \frac{2^{n}x_{1}x_{2}\cdots x_{n}}{\theta^{2n}} \\ \therefore & \ln L(\theta) = n\ln 2 + \sum\limits_{i=1}^{n}\ln x_{i} - 2n\ln \theta\\ \therefore & \frac{d\ln L(\theta)}{d\theta} = -\frac{2n}{\theta} \\ \because & 0< x < \theta \\ \therefore & \frac{d\ln L(\theta)}{d\theta} < 0 \text{, 即 } L(\theta) \text{单调递减}\\ \therefore & \hat{\theta}_{L} = \max\limits_{1\le i \le n}\{X_{i}\} \text{时, } L(\theta) \text{去最大值} \\ \therefore & \hat{\theta}_{L} = \max\limits_{1\le i \le n}\{X_{i}\}\\ \end{array}
    • ==注==
      1. θ^=max⁡{x1,x2,⋯ ,xn}\hat{\theta} = \max\{x_{1},x_{2},\cdots,x_{n}\} 估计值
      2. θ^=max⁡{X1,X2,⋯ ,Xn}\hat{\theta} = \max\{X_{1},X_{2},\cdots,X_{n}\} 估计量
  2. 设总体 XX 的密度为 f(x;θ)={θxθ−1,0<x<10,其他f(x;\theta) = \begin{cases} \theta x^{\theta-1}, & 0<x<1 \\ 0, & \text{其他} \end{cases}，θ>0\theta>0为未知参数，(X1,X2,⋯ ,Xn)(X_{1},X_{2},\cdots,X_{n})为来自总体XX的一个简单随机样本，分别求 θ\theta 的矩估计量 θ^M\hat{\theta}_{M} 和最大似然估计量 θ^L\hat{\theta}_{L} 矩估计∵Xˉ=EX=∫01θxθ⋅dx=θθ+1xθ+1∣01=θθ+1∴θ^M=Xˉ1−Xˉ最大似然估计L(θ)=P{X1∈U(x1),X2∈U(x2),⋯ ,Xn∈U(xn)}=θn×(x1x2⋯xn)θ−1∴ln⁡L(θ)=nln⁡θ+(θ−1)∑i=1nln⁡xi∴dln⁡L(θ)dθ=nθ+∑i=1nln⁡xi∴θ=−n∑i=1nln⁡xi时，L(θ)最大∴θ^L=−n∑i=1nln⁡Xi \begin{array}{ll} & \text{矩估计} \\ \because & \displaystyle \bar{X} = EX = \int_{0}^{1}\theta x^{\theta}\cdot dx \\ & = \frac{\theta}{\theta+1}x^{\theta+1}\vert_{0}^{1} = \frac{\theta}{\theta+1} \\ \therefore & \hat{\theta}_{M} = \frac{\bar{X}}{1-\bar{X}} \\\\ & \text{最大似然估计} \\ & L(\theta) = \mathbb{P}\{X_{1}\in U(x_{1}), X_{2}\in U(x_{2}),\cdots, X_{n}\in U(x_{n})\} \\ & = \theta^{n}\times (x_{1}x_{2}\cdots x_{n})^{\theta-1} \\ \therefore & \ln L(\theta) = n\ln \theta + (\theta-1)\sum\limits_{i=1}^{n}\ln x_{i} \\ \therefore & \frac{d \ln L(\theta)}{d\theta} = \frac{n}{\theta} + \sum\limits_{i=1}^{n}\ln x_{i} \\ \therefore & \theta = \frac{-n}{\sum\limits_{i=1}^{n}\ln x_{i}}\text{时，} L(\theta) \text{最大} \\ \therefore & \hat{\theta}_{L} = -\frac{n}{\sum\limits_{i=1}^{n}\ln X_{i}} \\ \end{array}
  3. 设总体 X∼(123θ22θ(1−θ)(1−θ)2)X\sim \left( \begin{matrix} 1 & 2 & 3\\ \theta^{2} & 2\theta(1-\theta) & (1-\theta)^{2} \end{matrix} \right)，其中 θ (0<θ<1)\theta\ (0<\theta<1) 为未知参数，已知取得了样本值，X1=1,X2=2,X3=1X_{1} = 1, X_{2}=2, X_{3} = 1，求 θ\theta 的矩估计值和最大似然估计量 矩估计∵Xˉ=EX=θ2+4θ(1−θ)+3(1−θ)2=43∴θ^M=56最大似然估计L(θ)=P{X1=x1,X2=x2,X3=x3}=θ2⋅2θ(1−θ)⋅θ2=2θ5(1−θ)∴ln⁡L(θ)=ln⁡2+5ln⁡θ+ln⁡(1−θ)∴dln⁡L(θ)dθ=5θ−11−θ=5−6θθ(1−θ)∵θ=56时, L(θ)取最大值∴θ^L=56 \begin{array}{ll} & \text{矩估计} \\ \because & \bar{X} = EX = \theta^{2} + 4\theta(1-\theta) + 3(1-\theta)^{2} = \frac{4}{3}\\ \therefore & \hat{\theta}_{M} = \frac{5}{6} \\\\ & \text{最大似然估计} \\ & L(\theta) = \mathbb{P}\{X_{1} = x_{1}, X_{2} = x_{2},X_{3}=x_{3}\} \\ & = \theta^{2}\cdot 2\theta(1-\theta)\cdot \theta^{2} = 2\theta^{5}(1-\theta) \\ \therefore & \ln L(\theta) = \ln 2 + 5\ln \theta + \ln (1-\theta)\\ \therefore & \frac{d\ln L(\theta)}{d\theta} = \frac{5}{\theta} - \frac{1}{1-\theta}= \frac{5-6\theta}{\theta(1-\theta)}\\ \because & \theta = \frac{5}{6} \text{时, } L(\theta) \text{取最大值}\\ \therefore & \hat{\theta}_{L} = \frac{5}{6} \\ \end{array}
  4. 设 X1,X2,⋯ ,XnX_{1},X_{2},\cdots,X_{n} 为来自均值为 θ\theta 的指数分布总体的简单随机样本，Y1,Y2,⋯ ,YmY_{1},Y_{2},\cdots,Y_{m} 为来自均值为 2θ2\theta 的指数分布的简单随机样本，且两样本相互独立，其中 θ (θ>0)\theta\ (\theta>0) 为未知参数，利用样本 X1,X2,⋯ ,Xn,Y1,Y2,⋯ ,YmX_{1},X_{2},\cdots,X_{n},Y_{1},Y_{2},\cdots,Y_{m} 求 θ\theta 的最大似然估计量 θ^\hat{\theta} L(θ)=P{X1∈U(x1),⋯ ,Xn∈U(xn),Y1∈U(y1),⋯ ,Ym∈U(ym)}∵Xi∼E(1θ),Yj∼E(12θ)∴fX(x)={1θe−1θx,x>00,x≤0fY(y)={12θe−12θy,y>00,y≤0∴L(θ)=12mθn+me−x1+⋯+xnθ−y1+⋯+ym2θ \begin{array}{ll} & L(\theta) = \mathbb{P}\{X_{1}\in U(x_{1}),\cdots,X_{n}\in U(x_{n}),Y_{1}\in U(y_{1}), \cdots , Y_{m}\in U(y_{m})\} \\ \because & X_{i} \sim E(\frac{1}{\theta}) , Y_{j}\sim E(\frac{1}{2\theta}) \\ \therefore & f_{X}(x) = \begin{cases} \frac{1}{\theta}e^{-\frac{1}{\theta}x}, & x>0 \\ 0, & x\le 0 \end{cases}\quad f_{Y}(y) = \begin{cases} \frac{1}{2\theta}e^{-\frac{1}{2\theta}y}, & y>0 \\ 0, & y\le 0 \end{cases}\\ \therefore & L(\theta) = \frac{1}{2^{m}\theta^{n+m}}e^{-\frac{x_{1}+\cdots+x_{n}}{\theta}-\frac{y_{1}+\cdots+y_{m}}{2\theta}} \\ \end{array}

无偏估计

  • 设 θ^\hat{\theta} 是 θ\theta 的估计量，若 Eθ^=θE \hat{\theta} = \theta，称 θ^\hat{\theta} 为 θ\theta 的无偏估计量

results matching ""

    No results matching ""