title
ML Articles Reading Notes

ML Articles Reading Notes*

Notes in chronological order: archive

Fundamentals

• Shalev-Shwartz 2014, Understanding Machine Learning: from Theory to Algorithms
  • Chapter 3: PAC learning, notes
  • Chapter 4: Learning via Uniform Convergence, notes
  • Chapter 5: No Free Lunch and Error Decomposition, notes
  • Chapter 6: VC Dimension, notes
  • Chapter 7: Nonuniform Learnability and SRM, notes
• Cucker Zhou 2007, Learning Theory: an approximation viewpoint
  • Chapter 1: The Framework of Learning, notes
  • Chapter 2: Basic Hypothesis Spaces, notes
• Murphy 2012, Machine Learning: a probabilistic approach
  • Chapter 3: Generative Models for Discrete Data, notes
  • Chapter 10: Directed Graphical Models, notes
• Ng Jordan, On Discriminative vs. Generative classifiers: a comparison of logistic regression and naive Bayes, notes

Information Theory

• Csiszár 2004: Information Theory and Statistics, a tutorial
  • Chapter 1: Preliminaries, notes
  • Chapter 2: Large deviations, hypothesis testing, notes

Statistics

• Minsker 2015: Geometric median and robust estimation in Banach spaces, notes

Generalization Theory

• Belkin 2018: To Understand Deep Learning We Need to Understand Kernel Learning, notes
• Tishby 2000: The Information Bottleneck Method, notes
• Tishby 2015: Deep Learning and the Information Bottleneck Principle, notes

Transfer Learning

• Pan Yang 2010: A Survey on Transfer Learning, notes
• Daume Marcu 2006: Domain Adaptation for Statistical Classifiers, notes
• Zadrozny 2004: Learning and Evaluating Classifiers under Sample Selection Bias, notes

Unsupervised Learning

• Yu Mineyev Varshney 2018: A Group-Theoretic Approach to Abstraction, notes

Optimization

• Jain Kar 2017, Non-convex Optimization for Machine Learning
  • Chapter 1: Introduction, notes
  • Chapter 2: Mathematical Tools, notes
  • Chapter 4: Alternating Minimization, notes
  • Chapter 5: EM Algorithm, notes
• Salimans 2017: Evolution Strategies as a Scalable Alternative to Reinforcement Learning, notes
• Lehman 2017: ES Is More Than Just a Traditional Finite-Difference Approximator, notes
• Zhang 2017: On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent, notes

*The phrasing in these notes are sometimes copied directly from the texts. Other times, the notes diverge quite a bit.