CS-233 · Introduction to Machine Learning
Instructions
- This exam has 16 multiple-choice questions (Part I, 60 points) and 2 open questions (Part II, 30 points).
- Single-choice questions (SCQ) have exactly one correct answer. Multiple-choice questions (MCQ) have one or more.
- One two-sided A4 cheat sheet is allowed. No electronic device is permitted.
- Answer on this sheet, in the space provided.
Part I · Multiple and single choice 60 points
Question 1 · SCQ, 3 points · Optimisation
You minimise a convex function f whose gradient is L-Lipschitz, with L = 4, using gradient descent xt+1 = xt − η ∇f(xt). With your current step size the iterates diverge. Which step size guarantees convergence?
- Aη = 0.25
- Bη = 0.6
- Cη = 4, the Lipschitz constant itself
- Dηt = t, growing with the iterations
Question 2 · MCQ, 4.5 points · Linear models
Compare linear regression (squared loss) and binary logistic regression (cross-entropy loss), both on D features with a bias term. Which statements are correct?
- ABoth models have D + 1 parameters.
- BThe cross-entropy loss of logistic regression is convex in its parameters.
- CLeast squares has a closed-form solution, while logistic regression needs an iterative solver.
- DThe output of logistic regression can be any real number.
Question 3 · SCQ, 3 points · k-nearest neighbours
On a fixed training set of N points, you increase k from 1 to N. What happens to the classifier?
- AIts bias decreases and its variance increases.
- BIts bias increases and its variance decreases.
- CBoth its bias and its variance decrease.
- DIts training error stays at zero for every k.
Question 4 · MCQ, 4.5 points · Regularisation
Ridge regression minimises ‖y − Xw‖² + λ‖w‖² with λ > 0. Which statements are correct?
- AThe solution is w = (X⊤X + λI)−1 X⊤y.
- BX⊤X + λI is invertible for any data matrix X.
- CA larger λ always lowers the test error.
- DAs λ → ∞, the weights w tend to 0.
Part II · Open questions 30 points
Question 17 · 15 points · Principal component analysis
Let X ∈ ℝN×D be centred data with covariance Σ = X⊤X / N.
- (5 points) Show that the first principal direction, which maximises w⊤Σw subject to ‖w‖ = 1, is an eigenvector of Σ.
- (5 points) The eigenvalues of Σ are 6, 3, 0.6 and 0.4. What fraction of the variance do the first two components keep?
- (5 points) Why must PCA be fitted on the training set only? Say what goes wrong otherwise.
Cet aperçu montre 5 des 18 questions. Dans Cortex, le sujet complet est livré avec son corrigé, et chaque question renvoie à la slide ou à l’ancien final dont elle vient.