This course explores a selected area relevant to statistical machine learning in depth, and will be taught by an SML staff member of internationally recognised standing and research interest in that area. Based on current SML staffing, this will be one of:

• kernel methods
• graphical models
• reinforcement learning
• convex analysis
• optimisation
• bioinformatics
• minimal description length principle
• topics in information theory
• decision theory

At the end of the course students should be able to:

• write down definitions of key concepts in convex analysis, including convexity of sets and functions, subgradients, and the convex dual

• derive basic results about convex functions such as Jensen’s inequality

• understand how Bregman divergences are constructed from convex functions and derive some of their properties

• write down a formal optimization problem from a high-level description and determine whether the problem is convex

• recognize standard convex optimization problems such as linear programs and quadratic programs

• derive the standard (dual) quadratic program for support vector machines and understand the extension to max-margin methods for structured prediction

• implement and analyse gradient descent algorithms such as stochastic gradient descent and mirror descent

Assessment will be in the form of fortnightly assignments and an open-book final examination.

• Knowledge of machine learning at the level of COMP4670 Introduction to SML
• Familiarity with linear algerba (including norms, inner products, determinants, eigenvalues, eigenvectors, and signular value decomposition)
• Familiarity with multivariate differential calculus (e.g., derivative of a vector-valued function)
• Exposure to mathematical proofs

COMP4670 Introduction to Statistical Machine Learning or permission from the course convenor.

n/a

Main text:

• Stephen Boyd and Lieven Vandenberghe, "Convex Optimization"

Reference texts:

• Hiriart-Urruty and Lemaréchal, “Fundamentals of Convex Analysis”
• Bertsekas, Nedic and Ozdaglar, “Convex Analysis and Optimization”
• Bertsekas, “Nonlinear Programming”