1.What is a decision boundary in two-class classification problems? In a linear classifier with a given (w, b), what is the formula for the decision boundary?
2. What is maximum likelihood estimation? Show that average is the maximum likelihood
estimation of the mean for Gaussian data
3. Describe several loss functions you know, and discuss their positive/negative characteristics
4. What are the differences between L2 and L1 regularization? For a given linear decision (w, b),
calculate the value of the L2 and L1 penalty
5. Describe majorization-minimization and gradient descent
6. Describe the proximal gradient descent method: what is the gradient step? What is the proximal
step? For a set Q={x : ||x||<=1} in 2D space, what is the result of ProxQ(y) where
a) y=[0.5,0.2]; b) y=[2,0]
7. What is a set function? What is the difference between a modular, and a submodular set
function? On a two-element set, show an example of a set function that is not submodular, and
an example of a function that is submodular.
8. Choose one of the submodular set functions used in regularization in supervised learning, and
describe its formula, formula for the convex extension, and the interpretation/possible
application of the set function.
9. Describe the architecture of a feed-forward neural network, and how dropout is used to limit
overtraining
10. Describe the convolutional layer in neural networks. How is it related to regularization?
11. Describe how batch normalization is performed and in which way it helps with training multilayer
neural networks with ReLU activation function.
12. What is the significance of the Representer Theorem for learning using kernels (e. g. Gaussians).