new issues in lecture notebooks

[bertdv]

The following issues have low priority but should be addressed for next year.

Prob theory

• Move https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Notational-Conventions to Gaussian lecture

• Rephrase key concept: "All valid probabilistic relations can be derived from just two fundamental principles: the sum rule and the product rule. These two rules form the foundation of probability theory, from which more complex constructs such as conditional probabilities, Bayes’ theorem, and marginalization naturally follow." (above https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Examples)

• add solution to exercise https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Moments-of-the-PDF

• We should not ask students to code in Julia before the Julia lecture. So, https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Working-with-Distributions-in-code is inappropriate, as is the showing of code in coding examples before the Julia lecture.

• The example of the Gaussian product of Gaussian distributions is inappropriate here, since the Gaussian distribution has not yet been treated. Postpone to the Gaussian lecture. https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#PDF-for-the-Product-of-Two-Variables

• Inappropriate exercise formulation: "... from the elementary sum rule
  and the sum and product rules." at https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#The-Sum-Rule-(**)

BML

• In BML class https://bmlip.github.io/course/lectures/Bayesian%20Machine%20Learning.html#2.-Parameter-Estimation, $x_n$ are not independent, but conditionally independent, given $\theta$.

• the data $D$ is hard-coded and does not adapt to the slider at the beginning of the lecture https://bmlip.github.io/course/lectures/Bayesian%20Machine%20Learning.html#Challenge-Revisited:-Predicting-a-Coin-Toss

• Let's not show too much Julia code yet before the Julia class.

• "Convergence to 0" is not an exercise. https://bmlip.github.io/course/lectures/Bayesian%20Machine%20Learning.html#Exercise:-Convergence-to-0

FFG

• In FFG class, add why large models are necessarily sparse!

Gaussians

• at the beginning of the class, add a link to the cheatsheet, which contains lots of Gaussian materials.

• add an animation for Bayes rule with Gaussian prior and Gaussian likelihood. The idea is to show that precisions add and precision-weighted means add. Let the user play with precisions in prior and likelihood.

• Update the estimation of $\theta$ from 1 measurement into a story of measuring your weight by # coconuts.

• Jaynes ref does not work consider http://www-biba.inrialpes.fr/Jaynes/prob.html instead

• Solution see later https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Solution must contain a forward link.

• remove refs to Roweis https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Linear-Transformations-of-Gaussian-Variables SRG-4a and SRG-8

• in proof of MLE for mean of Gaussian, https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Maximum-likelihood-estimate-of-mean, add a note to the cheatsheet in particular the Matrix algebra.

• The mini on Distributions in Julia is misplaced here https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Gaussian-Distributions-in-Julia . These are not gaussian distributions.

• add (3) model eval by evidence $p(D)$ and (4) application $p(x_\bullet|D)$ to learning about a Gaussian variable (after https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Inference)

Discrete data

• Move exercise on evidence for die toss to the main text.

Regression

• build model $p(y,x,w) = p(w) \prod_n p(y_n|x_n,w) \underbrace{p(x_n)}_{\delta(x_n - \hat{x}_n)}$

Classification

• The decision boundaries are generally (hyper-)parabolic surfaces. Dat moet denk ik ellipsoids zijn, want parabolen zijn “defecte” ellipsen (missende kwadratische term).

  • Beste Bert,
    Dank je wel voor je leuke en snelle reactie! Ik zat nog verder te denken: het is nog iets subtieler: de niveaukrommen x’ Sigma^{-1} x = constant zijn ellipsoids in de R^n, dat is wat in de tekst wordt beschreven denk ik. En dat is waarschijnlijk het eenvoudigst om te noemen in de tekst: elke klasse heeft ellipsoids om zich heen.

• Both generative and discriminative classification can be worked out from a model specification to lead to $$p(y_{\dot,k}=1 | x_\dot,D) \sim \mathrm{Cat}(\sigma(\beta_k^T x_\dot))$$

• query "Can you work out Bayesian logistic regression with Laplace approximation for the posterior on the weights" in chatGPT gives a nice calculation with shorter formulas (in matrix form) than currently in lecture notes.

Dynamic systems

• the first animation of prediction of position is unclear. I would expect $$p(z_{10})$$ based on priors alone to have a lot of noise.

• [ ]