01: Bayesian Linear Model

Consider the following model,

$$ \begin{aligned} \mathbf y \mid \boldsymbol\beta &\sim \text{MvNormal}(\mathbf X \boldsymbol \beta, \boldsymbol\Sigma) \\ \boldsymbol \beta & \sim \text{MvNormal}(\mathbf m, \mathbf S) \end{aligned} $$

Using section 14 from BDA3, we can write the above equivalently as:

$$ \begin{aligned} \begin{pmatrix} \boldsymbol \Sigma ^{-1/2} \mathbf y \\ \mathbf m \end{pmatrix} \Bigg| \boldsymbol \beta &\sim \text{MvNormal}\left( \begin{bmatrix} \boldsymbol\Sigma^{-1/2} \mathbf X \\ \mathbf I \end{bmatrix} \boldsymbol \beta, \begin{bmatrix} \mathbf I & \mathbf 0 \\ \mathbf 0 & \mathbf S \end{bmatrix} \right). \end{aligned} $$

Thus, we obtain the following posterior for $\boldsymbol\beta$:

$$ \begin{aligned} \boldsymbol \beta \mid \mathbf y &\sim \text{MvNormal}(\mathbf m^\star, \mathbf S^\star), \text{ where} \\ \mathbf S^\star &= \left(\mathbf X^T \boldsymbol\Sigma^{-1} \mathbf X + \mathbf S^{-1}\right)^{-1} \\ \mathbf m^\star &= \mathbf S^\star \left(\mathbf X^T \boldsymbol\Sigma^{-1} \mathbf y + \mathbf S^{-1} \mathbf m\right) \\ \end{aligned} $$