This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Here you can find the lecture slides for "Data Analysis and Machine Learning 4". This is an applied course that I wrote for 4th year electronic engineering students at the University of Edinburgh which I delivered during the second semester of the 2022-23 academic year.
The course is split into four parts:
- Data analysis (Lectures 1-3):
- An introduction to machine learning and ethical considerations (Lecture 4)
- Linear models (Lecture 5-7)
- Non-parametric and non-linear models (Lecture 8-10)
Each lecture (apart from Lecture 4) is accompanied by a Juypter notebook. These are available on Github. We also had a lab session each week.
There are quite a few things I'm thinking about changing for next year (SVMs are probably out, gradient boosted decision trees are probably in, there is some maths that needs fixing) so please view the material as a work-in-progress!
Lecture slides
Lecture 1: Introduction, data modalities, variable types
This lecture contains an overview of the course structure and assessments. We then examine different modalities of data and consider the different types of variables that make up data.
Lecture 2: Summarising and visualising data
In this lecture, we consider how to summarise datasets to convey information, and whether its actually meaningful. We also explore different ways to visualise data.
Lecture 3: Preprocessing, PCA, clustering
In this lecture we consider how to represent a tabular dataset in matrix form and revise some linear algebra. We then look at principal component analysis (PCA) and how it can be used for dimensionality reduction. Finally, we examine the K-means clustering algorithm.
Lecture 4: Machine Learning and Ethics
In this lecture we introduce supervised learning and give examples of problems that can be solved with machine learning. We then consider ethical conundrums that can arise when deploying machine learning in society.
In this lecture, we introduce regression, and show how we can solve this problem using linear models. We emphasise the importance of a test set in machine learning, and show how hyperparameters can be tuned on a validation set. Finally, we cover how gradient descent can be used to minimise convex loss functions that are not everywhere differentiable.
Lecture 6: Linear Classification
In this lecture, we introduce the problem of classification. We then consider linear classifiers and how their parameters can be learnt using (i) the perceptron algorithm and (ii) logistic regression.
Lecture 7: Support Vector Machines
In this lecture, we consider support vector machines: linear classifers that maximise the margin between classes. We show how the maximum margin criterion can be relaxed for data that isn't linearly separable, and how we can use the kernel trick to project our data into high dimensional spaces so it can be separated.
Lecture 8: Non-parametric models
In this lecture, we consider the non-parametric k-nearest neighbours algorithm for classification. We then look at decision trees, and how to learn them. We note their propensity to overfit, and find out how to form a random forest classifier using boostrap aggregation.
Lecture 9: Deep Neural Networks
In this lecture we introduce deep learning as feature learning, and examine a multilayer perceptron (MLP) model and how its weights can be learned using backpropagation combined with SGD.
Lecture 10: Convolutional Neural Networks
In the last lecture of this course, we look at images and consider how important their spatial structure is. We then examine the convolution operation and how it is used in convolutional neural networks (ConvNets).
Lecture 1: Introduction, data modalities, variable types
This lecture contains an overview of the course structure and assessments. We then examine different modalities of data and consider the different types of variables that make up data.
Lecture 2: Summarising and visualising data
In this lecture, we consider how to summarise datasets to convey information, and whether its actually meaningful. We also explore different ways to visualise data.
Lecture 3: Preprocessing, PCA, clustering
In this lecture we consider how to represent a tabular dataset in matrix form and revise some linear algebra. We then look at principal component analysis (PCA) and how it can be used for dimensionality reduction. Finally, we examine the K-means clustering algorithm.
Lecture 4: Machine Learning and Ethics
In this lecture we introduce supervised learning and give examples of problems that can be solved with machine learning. We then consider ethical conundrums that can arise when deploying machine learning in society.
In this lecture, we introduce regression, and show how we can solve this problem using linear models. We emphasise the importance of a test set in machine learning, and show how hyperparameters can be tuned on a validation set. Finally, we cover how gradient descent can be used to minimise convex loss functions that are not everywhere differentiable.
Lecture 6: Linear Classification
In this lecture, we introduce the problem of classification. We then consider linear classifiers and how their parameters can be learnt using (i) the perceptron algorithm and (ii) logistic regression.
Lecture 7: Support Vector Machines
In this lecture, we consider support vector machines: linear classifers that maximise the margin between classes. We show how the maximum margin criterion can be relaxed for data that isn't linearly separable, and how we can use the kernel trick to project our data into high dimensional spaces so it can be separated.
Lecture 8: Non-parametric models
In this lecture, we consider the non-parametric k-nearest neighbours algorithm for classification. We then look at decision trees, and how to learn them. We note their propensity to overfit, and find out how to form a random forest classifier using boostrap aggregation.
Lecture 9: Deep Neural Networks
In this lecture we introduce deep learning as feature learning, and examine a multilayer perceptron (MLP) model and how its weights can be learned using backpropagation combined with SGD.
Lecture 10: Convolutional Neural Networks
In the last lecture of this course, we look at images and consider how important their spatial structure is. We then examine the convolution operation and how it is used in convolutional neural networks (ConvNets).
Thanks to all the people who helped me make this course. Special thanks to Joe Mellor (who also designed the logo), Amos Storkey, Justin Engelmann, Tom Lee, and Henry Gouk for many a discussion. My T&Ds (Yalei Fu, Hollan Haule, Yihong Zhou) were also amazing.