Deep Learning

This site is dedicated to the simplest video tutorials on Deep Learning. My aim is to prepare a free & interactive video book on Deep Learning. I have used my knowledge and experience to prepare these tutorials. All feedback and suggestions are welcome (email me at nirajrkumar@gmail.com or nirajrkumar@yahoo.com).

As, scientific development is an endless process, so I will keep updating it. Clicking on the link will drive you to the YouTube page for related content. Or You can use the link: https://www.youtube.com/c/DrNirajRKumar

1. Bias Variance Tradeoffs.

Content:

• Basics of Bias and Variance
• Bias and variance using bulls-eye diagram.
• Bias Variance Tradeoff and Algorithm Design Strategies

Video Link:

• Bias Variance Tradeoff Part-1 (Direct Link: https://www.youtube.com/watch?v=UxSCxX6KNQQ&t=315s )

2. Gradient Descent Algorithms.

Content:

• Gradient Descent
• Batch Gradient Descent
• Stochastic Gradient Descent
• Mini Batch Gradient Descent

Video Link:

• Gradient Descent & Batch Gradient Descent (Direct Link: https://www.youtube.com/watch?v=E73o0tKKxrs&t=33s )

• Stochastic Gradient Descent & Mini Batch Gradient Descent ( Direct Link: https://www.youtube.com/watch?v=FpgSyASgu6A&t=32s)

3. Vanishing and Exploding Gradient Problems.

Content:

• Key Reasons for Vanishing and Exploding Gradient
• Symptoms to Identify
• How to Solve the problem of Vanishing and Exploding Gradients (some key approaches)
• Weight Initialization
• Gradient Clipping
• Batch Normalizations

Video Link:

• Vanishing and Exploding Gradient Problems Part-1 (Direct Link: https://www.youtube.com/watch?v=UCMfZcuF8ME&t=668s)

• Vanishing and Exploding Gradient Problems Part-2 (Direct Link: https://www.youtube.com/watch?v=W4Hq2-3Jxt4&t=438s)

Vanishing and Exploding Gradient Problems.

4. Momentum, RMSProp, and ADAM Optimizers.

Content:

• Momentum Optimizers
• RMSProp Optimizers
• ADAM Optimizers

Video Link:

• Momentum and RMSProp Optimizers (Direct Link: https://www.youtube.com/watch?v=PioWzG9yRII)

• ADAM(Adaptive Moment Estimation) Optimizers. (Direct Link: https://www.youtube.com/watch?v=PcJz1Si0_ic&t=665s )

5. Deep Learning Cost Functions.

Content:

• Entropy and Cross-Entropy
• Binary Cross Entropy Loss
• Categorical Cross Entropy Loss

Video Link:

• Entropy and Cross-Entropy (Direct Link: https://www.youtube.com/watch?v=7YQBh0gFZ00 )

• Binary Cross Entropy Loss (Direct Link: https://www.youtube.com/watch?v=098Q3Nmce4A )

• Categorical Cross Entropy Loss (Direct Link: https://www.youtube.com/watch?v=a5WiHmNNJAY )

6. Regularization in Deep Learning.

Content:

• L1-Regularization
• L2-Regularization
• L1 Regularization in Deep Learning and Sparsity
• L2 Regularization in Deep Learning and Weight Decay

Video Link:

• L1-Regularization (Direct Link: https://www.youtube.com/watch?v=eUIZjUpYbwU)

• L2-Regularization (Direct Link: https://www.youtube.com/watch?v=NQJ0JXgreh8)

7. Saddle Points Problem in Deep Learning.

Content:

• Saddle Points - Introduction
• Impact of Saddle Points in Learning
• Second Partial Derivative Test
• Eigen Value & Hessian Matrix based Test.
• How to Escape from Saddle points

Video Link:

• Saddle Points Problem in Deep Learning (Direct Link:https://www.youtube.com/watch?v=jRLEyS6ID90&t=120s)

8. CNN (Convolutional Neural Networks).

Content:

• Part-1: Basics of CNN,
• Convolution Operation,
• Pooling Operation, and Calculating I/O size,
• Part-2: Basic of Padding in Convolutional Neural Network,
• The Padding I/O Size Computation,
• Some Insight about Padding,
• Part-3: Basics of Stride in Convolutional Neural Network,
• Convolution and Pooling Operation with Higher Stride Values.,
• Some Insight about Strides

Video Link:

• Part-1: Basics of CNN, Convolution, Pooling (Direct Link: https://www.youtube.com/watch?v=yRVzvWllx-g)

• Part-2: Basic of Padding in Convolutional Neural Network (Direct Link: https://www.youtube.com/watch?v=LCH82VqX_iE)

• Part-3: Basics of Stride in Convolutional Neural Network (Direct Link: https://www.youtube.com/watch?v=iKAjmELMCh4)

9. Deep Learning using Deep Neural Network.

Content:

• Basics of Deep Learning and Deep Neural Networks,
• Forward Propagation and Total Error Computation
• Back Propagation in Deep Neural Networks

Video Link:

• Part-1: Basics of Deep Learning and Deep Neural Networks (Direct Link: https://www.youtube.com/watch?v=Lmhp8ZQ2n-Y&t=461s)

• Part-2: Forward Propagation and Total Error Computation (Direct Link: https://www.youtube.com/watch?v=nqERS3xVSGI)

• Part-3: Back Propagation in Deep Neural Networks (Direct Link: https://www.youtube.com/watch?v=ylFODd8UTio&t=169s)

10. RNN (Recurrent Neural Network).

Content:

• Basics of Recurrent Neural Network (RNN),
• Architectural differences between RNN and DNN (Deep Neural Network).
• Training RNN - Through a very simple example.

• Explains the use of "Back-propagation Through Time (BPTT)", to train the RNN.

Video Link:

• RNN - Part-1 (Direct Link: https://www.youtube.com/watch?v=SiBLzJxSzpo)

• RNN Part-2 (Direct Link: https://www.youtube.com/watch?v=_ugj7u96_Zk)

11. LSTM (Long Short Term Memory).

Content:

• Basics of LSTM (Long Short Term Memory),
• Vanishing Gradient Problem in RNN
• Architectural differences between RNN and LSTM.
• Feed Forward Propagation

• Explains the Forward Propagation and "Back-propagation Through Time (BPTT)", to train the LSTM.

• Derivation for the "Back-propagation Through Time (BPTT)", to train the LSTM.

Video Link:

• Long Short-Term Memory (LSTM) Part-1 (Direct Link: https://www.youtube.com/watch?v=g5ka8WdNpDk)

• Long Short Term Memory (LSTM) part-2 (Direct Link: https://www.youtube.com/watch?v=7lzmyDKRfbg)

• Long Short Term Memory (LSTM) part-3 (Direct Link: https://www.youtube.com/watch?v=KGOBB3wUbdc)

• RNN Part-2 (Direct Link: https://www.youtube.com/watch?v=_ugj7u96_Zk)

12. Restricted Boltzmann Machine (RBM).

Content:

• Basic Overview of RBM and
• Applications of RBM

• Constructive divergence,
• Gibbs Sampling,
• Using Gibbs Sampling for training.

• Continue Training process for RBM.

Video Link:

• Restricted Boltzmann Machine - Part-1 (Direct Link: https://www.youtube.com/watch?v=BqfvL3NbY_o)

• Restricted Boltzmann Machine - Part-2 (Direct Link: https://www.youtube.com/watch?v=83xXiZ12tPk&t=260s)

• Restricted Boltzmann Machine - Part-3 (Direct Link: https://www.youtube.com/watch?v=G6rSUTCqYYs&t=4s)

13. Deep Learning using Deep Belief Network.

Content:

• Part-1
• Deep Belief Network Basics and
• Working of the DBN Greedy Training through an example.
• Part-2:
• Working of the DBN Greedy Training (continue from part-1)
• Fine tuning of Deep Belief Networks,
• Classification using Deep Belief Networks

Video Link:

• Deep Learning using Deep Belief Network Part-1 (Direct Link: https://www.youtube.com/watch?v=WKet0_mEBXg)

• Deep Learning using Deep Belief Network Part-2 (Direct Link: https://www.youtube.com/watch?v=CzoNuCNeCC0)

14. Attention-Based Models.

Content:

• Self-Attention
• Hierarchical Attention

Video Link:

• Attention Model Simplified (Direct Link: https://www.youtube.com/watch?v=6l1fv0dSIg4&t=19s)

• Self Attention Made Easy (Direct Link: https://www.youtube.com/watch?v=abWYwL819JQ)

• Hierarchical Attention Networks Simplified (Direct Link: https://www.youtube.com/watch?v=QUjmiA2VMQ4&t=14s)