Introduction to Statistical Machine Learning 2010 (ANU COMP6467/4670)
Course Coordinators
Presenter
Tutor
Time and Place
- Lectures
- Tuesday 4pm - 5.30pm, Building 34 , CHEM T2
- Wednesday 4pm - 5.30pm, Building 48 , FSTY 103
- Tutorials
- For more detailed information, please refer to the schedule at the end of this page.
News
- 06-06-2010: Complete Set of Slides available
- 22-04-2010: Oral examination dates: 16/17 June 2010
- Visit to the Machine Learning group in NICTA-Canberra (7 London Circuit)
Friday, 21 May, 11 am - 1.30 pm
Time Presenter Title 11.00 Wray Buntine "Bayesian Methods in Document Analysis" 11.30 Hanna Suominen Machine learning for health and wellness: supporting decision making and improving performance 12.00 Lunch 12.30 Peter Sunehag "Reinforcement Learning - The Exploration-Exploitation Dilemma 13.00 Matthew Robards "Reinforcement Learning for Large Spaces" - 04-05-2010: Assignment 2 is posted.
- Visit to the Machine Learning group in NICTA-Canberra (7 London Circuit)
Friday, 23 April, 11 am - 2.00 pm
Time Presenter Title 11.00 Edwin Bonilla "Gaussian Processes" 11.30 Marcus Hutter "Universal Artificial Intelligence" 12.00 Julian McAuley "Learning with Structured Data" 12.30 Lunch 13.00 Mark Reid "Probability Estimation and Elicitation" 13.30 Novi Quadrianto "Learning in Non-Standard Settings" - 22-03-2010: Assignment 1 is posted.
- 24-02-2010: Please register for the lab session on Wednesday (2-4 pm) or Thursday (1-3 pm).
- 24-02-2010: Added Python and Linear Algebra resources at the end of this page.
- 19-02-2010: Please note the change of lecture theaters.
- 20-01-2010: The course book Christopher M. Bishop: Pattern Recognition and Machine, Springer is available for a great bargain from amazon.com.
- 04-01-2010: Setting up the web site
Enrollment
Prerequisites: some background in elementary statistics and probabilities, numerical algorithms, and programming experience.
If you fulfil official requirements, please send an email to Christfried Webers (firstname.lastname@nicta.com.au) that he can support your enrolment.
Assessment
- 2 written assignments (20% each)
- oral examination (60%)
Assessment Late Policy
- For every beginning day after the deadline of an assignment, the mark will be reduced by 20%.
Important dates and special announcements
- Assignment 1 using Fisher Iris Data , due: April 26, 23:59 (released: March 22)
- Assignment 2 , due: May 27, 23:59 (released: May 4)
- Final examination : 16/17 June 2010
Contact hours for students
After the lecture and per email
Textbook
Required:
We also recommend (ordered by priority):
- Hastie, Tibshirani, and Friedman: The Elements of Statistical Learning, Springer
- MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press
- Hutter, Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability, Springer
Overview
Statistical Machine Learning plays a key role in science and technology. Some examples of applications using Statistical Machine Learning techniques are e.g.
- e-mail spam filtering,
- web page ranking,
- handwritten ZIP code recognition,
- identification of risk factors for cancer,
- object recognition in computer vision, and,
- autonomous robot navigation.
Some of the basic questions raised are
- What is a good model for the available data?
- How computationally effective can the parameters of the model be fitted to the available data?
- How does a model perform on future data?
This course provides a broad but thorough introduction to the methods and practice of statistical machine learning. Topics covered will include Bayesian inference and maximum likelihood modeling; regression, classification, density estimation, clustering, principal and independent component analysis; parametric, semi-parametric, and non-parametric models; basis functions, neural networks, kernel methods, and graphical models; deterministic and stochastic optimisation; overfitting, regularisation, and validation.
The course employs the Elefant - Machine Learning Toolbox developed at NICTA and written in the programming language Python. The graphical user interface of Elefant and a number of predefined building blocks (e.g. data readers and writers, algorithms) allow to easily setup and store a Machine Learning experiment. Elefant is Open Source and available for Linux, Mac OS X, and Windows. It will be provided on all computers used in the tutorials and labs. (Please watch a movie demonstrating Elefant in action.)
Schedule
(to be adapted and refined throughout the course)| week | Lecture | Lecture | Tutorial/Lab | Tue 4 - 5.30 pm | Wed 4 - 5.30 pm | Wed 2-4 (N113) or Thu 1-3(N112) | CHEM T2, Building 34 | FSTY 103, Building 48 | CSIT, Building 108 | 22/2 - 26/2 | Overview (Slides) | Introduction (Slides) | none | 1/3 - 5/3 | Linear Algebra (Slides) | Probability (Slides) |
Python, Linear Algebra, Optimisation (For Matlab users, please see "From Matlab to Python/Numpy") |
8/3 - 12/3 | Linear Regression 1 (Slides) | Linear Regression 2 (Slides) | Tutorial 2 | 15/3 - 19/3 | Linear Classification 1 (Slides) | Linear Classification 2 (Slides) | Tutorial 3 using Fisher Iris Data (see Iris flower data set) (Code) | 22/3 | Assignment 1 using Fisher Iris Data | 22/3 - 26/3 | Neural Networks 1 (Slides) | Neural Networks 2 (Slides) |
Finish previous tutorials (Possibly start Tutorial 4) |
29/3 - 2/4 | Kernel Methods (Slides) | Sparse Kernel Machines (Slides) | Tutorial 4 ( Sample code) | 5/4 - 9/4 | none | none | none | 12/4 - 16/4 | none | none | none | 19/4 - 23/4 | Graphical Models 1 (Slides) | Graphical Models 2 (Slides) | Tutorial 5 (Elefant manual) | 26/4 - 30/4 | Graphical Models 3 (Slides) | Mixture Models and EM (Slides) | Tutorial 6 ( BayesNoisy ) | 4/5 | Assignment 2 using Fisher Iris Data | 3/5 - 7/5 | Mixture Models and EM (Slides) | Approximate Inference (Slides) | Tutorial 7 ( Labels, Images ) | 10/5 - 14/5 | Sampling (Slides) | Principal Component Analysis (Slides) | Tutorial 8 | 17/5 - 21/5 | Sequential Data 1 (Slides) | Sequential Data (Slides) | Tutorial 9 ( PNG images ) | 24/5 - 28/5 | Combining Models (Slides) | Selected Topics (Slides) | Tutorial 10 ( Sample code ) | 31/5 - 4/6 | Discussion/Summary (Slides) | Exam preparations | none | 16+17/6 | Oral Examination (All Slides) |
|---|
Resources
Python crash course from Wikipedia.
Linear Algebra ("Paul's Online Math Notes").
For excellent supplemental slides, see Andrew Moore's Tutorials.
Last modified 2010-02-24 17:12