Deep Learning
Welcome to Deep Learning!
Over the past few years, Deep Learning has become a popular area, with deep neural network methods obtaining state-of-the-art results on applications in computer vision (Self-Driving Cars), natural language processing (Google Translate), and reinforcement learning (AlphaGo). These technologies are having transformative effects on our society, including some undesirable ones (e.g. deep fakes).
This course is there to give students a practical understanding of how Deep Learning works, how to implement neural networks, and how to apply them ethically. We introduce students to the core concepts of deep neural networks and survey the techniques used to model complex processes within the contexts of computer vision and natural language processing.
Throughout the course, we emphasize and require students to think critically about potential ethical pitfalls that can result from mis-application of these powerful models. The course is taught using the Tensorflow deep learning framework.
Lectures
[Week 1-4] | Deep Learning Basics | ||
[1-2] 1/25 | Welcome to Deep Learning | Recording | Slides |
[1-3] 1/27 | Supervised Learning - Classification/Regression, Training/Validation/Testing | Recording | Slides |
[2-1] 1/30 | Perceptron and MNIST | Recording | Slides |
[2-2] 2/1 | Perceptron (continued) and Loss Functions | Recording | Slides |
[2-3] 2/3 | Optimization and Backpropagation | Recording | Slides |
[3-1] 2/6 | Backpropagation (continued) | Recording | Slides |
[3-2] 2/8 | Autodiff | Recording | Slides |
[3-3] 2/10 | Matrix representation of NNs + GPUs + Intro to Tensorflow | Recording | Slides |
[4-1] 2/13 | Multi-layer NNs and Activation Functions | Recording | Slides |
[4-2] 2/15 | The Lifecycle of a Machine Learning Project | Recording | Slides |
[Week 4-6] | CNNs | ||
[4-3] 2/17 | Multi-layer NNs (contd.) + Intro to CNNs | Recording | Slides |
[5-2] 2/22 | CNNs (contd.) | Recording | Slides |
[5-3] 2/24 | Multi-layer CNNs | Recording | Slides |
[6-1] 2/27 | Overfitting and regularization | Recording | Slides |
[Week 6-9] | Language Models | ||
[6-2] 3/1 | Language models and word embeddings | Recording | Slides |
[6-3] 3/3 | Feedforward language models | Recording | Slides |
[7-1] 3/6 | Recurrent neural networks | Recording | Slides |
[7-2] 3/8 | LSTMs and GRUs | Recording | Slides |
[7-3] 3/10 | Machine translation | Recording | Slides |
[8-1] 3/13 | Attention | Recording | Slides |
[8-2] 3/15 | Transformers | Recording | Slides |
[8-3] 3/17 | Transformers (continued) and scaling deep learning systems | Recording | Slides |
[9-1] 3/20 | Scaling deep learning systems continued. | Recording | Slides |
[Week 9-10] | Interpretation | ||
[9-2] 3/22 | Interpretation of Neural Networks | Recording | Slides |
Assignments
Assignment | Out | Due | |
---|---|---|---|
[HW0] | Setup | Wednesday 1/25 | Friday 2/3 |
[HW0] | Math Review | Wednesday 1/25 | Friday 2/3 |
[HW1] | Linear Regression: Conceptual | Wednesday 2/1 | Friday 2/10 |
[HW1] | Linear Regression: Programming | Wednesday 2/1 | Monday 2/13 |
[HW2] | Beras: Conceptual | Wednesday 2/15 | Friday 2/24 |
[HW2] | Beras: Programming | Wednesday 2/15 | Monday 2/27 |
[HW3] | CNNs: Conceptual | Wednesday 3/1 | Monday 3/6 |
[HW3] | CNNs: Programming | Wednesday 3/1 | Friday 3/10 |
[HW4] | LMs: Conceptual | Wednesday 3/13 | Monday 3/20 |
[HW4] | LMs: Programming | Wednesday 3/13 | Friday 3/24 |
Final Project
See the handout for full detailsDeliverable | Date/Due |
---|---|
Forming teams | Wednesday 2/22 6:00pm ET |
Project Check-in 1 | Week beginning 3/6 |
Project Proposal | Friday 3/17 6:00pm ET |
Project Check-in 2 | Week beginning 4/10 |
Project Check-in 3 | Week beginning 4/24 |
Final Check-in (Optional) | Week beginning 5/1 |
Deep Learning Day | Monday 5/8 |
Final Projects Due | Friday 5/12 6:00pm ET |
Labs
Lab | From | Until | |
---|---|---|---|
[Lab0] | Introduction to NumPy | Wednesday 1/25 | Tuesday 1/31 |
[Lab1] | Introduction to Machine Learning | Monday 1/30 | Sunday 2/05 |
[Lab2] | Optimizers | Monday 2/06 | Sunday 2/12 |
[Lab3] | Tensorflow | Monday 2/13 | Sunday 2/19 |
[Lab4] | CNNs | Monday 2/27 | Sunday 3/5 |
[Lab5] | Debiasing | Monday 3/13 | Sunday 3/19 |
Hours
Resources
Staff
Professor

HTAs

brew upgrade
just to feel something




UTAs



















STAs


Grad TA
