Topic 20 Conceptual Ideas

20.1 Exercises

1. Mathematically, what makes up a layer in a deep learning model, and what is the goal of having many layers? Somewhere in your answer, you should talk about nonlinearity.

2. Fitting deep learning models
   1. In fitting a deep learning model, the goal is to estimate what quantities?
   2. How does the loss function play a role in fitting a deep learning model?
   3. At a very big-picture, conceptual level, why is calculus needed in the fitting process?

3. How do the number of hidden layers, hidden units within each layer, and density of connections affect model complexity? Discuss in terms of overfitting and the bias-variance tradeoff.

4. Explain 2 general strategies for fighting overfitting in deep learning models.

5. Give an overview of the convolution operation in convolutional neural networks, and explain why this is a useful way to handle image data.

6. Explain why convolution layers are not densely connected.