Welcome to
Deep Learning

Welcome to Deep Learning!

Over the past few years, Deep Learning has become a popular area, with deep neural network methods obtaining state-of-the-art results on applications in computer vision (Self-Driving Cars), natural language processing (Google Translate), and reinforcement learning (AlphaGo). These technologies are having transformative effects on our society, including some undesirable ones (e.g. deep fakes).

This course is there to give students a practical understanding of how Deep Learning works, how to implement neural networks, and how to apply them ethically. We introduce students to the core concepts of deep neural networks and survey the techniques used to model complex processes within the contexts of computer vision and natural language processing.

Throughout the course, we emphasize and require students to think critically about potential ethical pitfalls that can result from mis-application of these powerful models. The course is taught using the Tensorflow deep learning framework.

ProfessorRitambhara Singh
TimeMWF 12:00-12:50pm

Lectures

Monday, Wednesday, and Friday at 12:00-12:50pm in Salomon Center 001 Course offered in-person with recordings made available for reviewing.

[Week 1-4] Deep Learning Basics
[1-2] 1/25 Welcome to Deep Learning Recording Slides
[1-3] 1/27 Supervised Learning - Classification/Regression, Training/Validation/Testing Recording Slides
[2-1] 1/30 Perceptron and MNIST Recording Slides
[2-2] 2/1 Perceptron (continued) and Loss Functions Recording Slides
[2-3] 2/3 Optimization and Backpropagation Recording Slides
[3-1] 2/6 Backpropagation (continued) Recording Slides
[3-2] 2/8 Autodiff Recording Slides
[3-3] 2/10 Matrix representation of NNs + GPUs + Intro to Tensorflow Recording Slides
[4-1] 2/13 Multi-layer NNs and Activation Functions Recording Slides
[4-2] 2/15 The Lifecycle of a Machine Learning Project Recording Slides
[Week 4-6] CNNs
[4-3] 2/17 Multi-layer NNs (contd.) + Intro to CNNs Recording Slides
[5-2] 2/22 CNNs (contd.) Recording Slides
[5-3] 2/24 Multi-layer CNNs Recording Slides
[6-1] 2/27 Overfitting and regularization Recording Slides
[Week 6-9] Language Models
[6-2] 3/1 Language models and word embeddings Recording Slides
[6-3] 3/3 Feedforward language models Recording Slides
[7-1] 3/6 Recurrent neural networks Recording Slides
[7-2] 3/8 LSTMs and GRUs Recording Slides
[7-3] 3/10 Machine translation Recording Slides
[8-1] 3/13 Attention Recording Slides
[8-2] 3/15 Transformers Recording Slides
[8-3] 3/17 Transformers (continued) and scaling deep learning systems Recording Slides
[9-1] 3/20 Scaling deep learning systems continued. Recording Slides
[Week 9] Interpretation
[9-2] 3/22 Interpretation of Neural Networks Recording Slides
[Week 10-11] Probabilistic Models
[10-1] 4/3 Unsupervised learning, Autoencoders Recording Slides
[10-2] 4/5 Variational Autoencoders Recording Slides
[10-3] 4/7 VAEs contd. and Generative adversarial networks Recording Slides
[11-1] 4/10 VAE and GANs contd + Deepfakes Recording Slides
[11-2] 4/12 Diffusion (guest lecture by Calvin Luo) Recording Slides
[11-3] 4/14 Diffusion continued (guest lecture by Calvin Luo) Recording Slides
[Week 12-13] Reinforcement Learning
[12-1] 4/17 Introduction to reinforcement learning Recording Slides
[12-2] 4/19 Value Iteration Recording Slides
[12-3] 4/21 Deep Q learning Recording Slides
[13-1] 4/24 Policy gradient methods Recording Slides
[13-2] 4/26 Actor-critic methods Recording Slides
[Week 13] GNNs
[13-3] 4/28 Graph neural networks Recording Slides

Assignments

Assignments will be released at noon and due at 6:00pm U.S. Eastern Time.

Assignment Out Due
[HW0] Setup Wednesday 1/25 Friday 2/3
[HW0] Math Review Wednesday 1/25 Friday 2/3
[HW1] Linear Regression: Conceptual Wednesday 2/1 Friday 2/10
[HW1] Linear Regression: Programming Wednesday 2/1 Monday 2/13
[HW2] Beras: Conceptual Wednesday 2/15 Friday 2/24
[HW2] Beras: Programming Wednesday 2/15 Monday 2/27
[HW3] CNNs: Conceptual Wednesday 3/1 Monday 3/6
[HW3] CNNs: Programming Wednesday 3/1 Friday 3/10
[HW4] LMs: Conceptual Wednesday 3/13 Monday 3/20
[HW4] LMs: Programming Wednesday 3/13 Friday 3/24
[HW5] Image Captioning: Conceptual Monday 4/3 Monday 4/10
[HW5] Image Captioning: Programming Monday 4/3 Friday 4/14
[HW6] Variational Autoencoders: Conceptual Friday 4/14 Friday 4/21
[HW6] Variational Autoencoders: Programming Friday 4/14 Friday 4/28

Final Project

See the handout for full details

Deliverable Date/Due
Forming teams Wednesday 2/22 6:00pm ET
Project Check-in 1 Week beginning 3/6
Project Proposal Friday 3/17 6:00pm ET
Project Check-in 2 Week beginning 4/10
Project Check-in 3 Week beginning 4/24
Final Check-in (Optional) Week beginning 5/1
Deep Learning Day Monday 5/8
Final Projects Due Friday 5/12 6:00pm ET

Labs

See the Resources section for information on opening labs and using Google Colaboratory.

Lab From Until
[Lab0] Introduction to NumPy Wednesday 1/25 Tuesday 1/31
[Lab1] Introduction to Machine Learning Monday 1/30 Sunday 2/05
[Lab2] Optimizers Monday 2/06 Sunday 2/12
[Lab3] Tensorflow Monday 2/13 Sunday 2/19
[Lab4] CNNs Monday 2/27 Sunday 3/5
[Lab5] Debiasing Monday 3/13 Sunday 3/19
[Lab6] LIME Monday 4/3 Sunday 4/9
[Lab7] Autoencoders Monday 4/10 Sunday 4/16
[Lab8] GANs Monday 4/17 Sunday 4/23
[Lab9] Reinforcement Learning Monday 4/24 Sunday 4/30

Hours

Resources

Staff

Professor

Ritambhara Singh
Ritambhara Singh
she/her • rsingh47
Dory, from Finding Nemo, is my favorite animated movie character ("Just keep swimming").

HTAs

Dylan Hu
Dylan Hu
he/him • dhu24
Runs brew upgrade just to feel something
Nitya Thakkar
Nitya Thakkar
she/her • nthakka3
I am notoriously terrible at remembering names so you'll have to be really memorable for me to remember yours :)
Raymond Dai
Raymond Dai
he/him • rdai4
Will be replaced by ChatGPT in late March.
Vadim Kudlay
Vadim Kudlay
he/him • vkudlay
:D
Robert Scheidegger
Robert Scheidegger
he/him • rscheide
When I was in elementary school, I was fascinated by sea animals and would spend many of my mornings reading about them.

UTAs

Bumjin Joo
Bumjin Joo
he/him • bjoo2
Custom keebs are pretty cool.
Dave Lubawski
Dave Lubawski
he/him • dlubawsk
Hi I’m Dave and my fun fact is I once fought 4 chickens and lost
Jitpuwapat (Earth) Mokkamakkul
Jitpuwapat (Earth) Mokkamakkul
he/him • jmokkama
My first name, Jitpuwapat, was given to me by the late king of Thailand
Eric Han
Eric Han
he/him • ehan31
My last name means Korean! (I'm Chinese though, I swear)
Evan Lu
Evan Lu
he/him • elu14
sassy lost child
Henry Sowerby
Henry Sowerby
he/him • hsowerby
I spent the weekend with a bunch of highly agentic people and now i've got a bunch of thoughts about agency.
Iris Cheng
Iris Cheng
she/her • icheng3
I make really good dumplings.
Joe Dodson
Joe Dodson
he/him • jdodson4
Jun Suk Ha
Jun Suk Ha
he/him • jha38
Karan Kashyap
Karan Kashyap
he/him • kkashyap
I caught a shark the first time I went fishing
Logan Bauman
Logan Bauman
he/him • lbauman
I love food and sports so come talk to me about either! You can also just give me food.
Michael Lu
Michael Lu
he/him • mlu54
That first bite of Chinatown always hit different.
Nange Li
Nange Li
she/her • nli32
I only order bubble tea with 0% sugar.
Preeti Nagalamadaka
Preeti Nagalamadaka
she/her • pnagala1
Did you know sea otters hold hands when they sleep so they don't drift away from each other?
Ray Del Vecchio
Ray Del Vecchio
he/him • rdelvecc
I have a math tattoo somewhere on me!
Ray Wang
Ray Wang
he/him • xwang356
I accidentally became a dance major in undergrad
Shirley Loayza Sanchez
Shirley Loayza Sanchez
she/her • sloayzas
I have taken pictures of +100 rose varieties!
Taishi Nishizawa
Taishi Nishizawa
he/him • tnishiza
i like cheese
Will Guo
Will Guo
he/him • wguo25
I like spinach. A lot.

STAs

Brendan Ho
Brendan Ho
he/him • bho15
I prefer leftovers cold right out of the fridge, excited to meet you all!
Faizaan Vidhani
Faizaan Vidhani
he/him • fvidhani
I like rooting for Dallas sports teams and spending time at the OMAC.

Grad TA

Xianghao Xu
Xianghao Xu
he/him • xxu43