Advanced Deep Learning

This site is dedicated to the simplest video tutorials on Advance Topics of Deep Learning. I have used my knowledge and experience to prepare these tutorials. All feedback and suggestions are welcome (email me at nirajrkumar@gmail.com or nirajrkumar@yahoo.com).

As, scientific development is an endless process, so I will keep updating it. Clicking on the link will drive you to the YouTube page for related content. Or You can use the link: https://www.youtube.com/c/DrNirajRKumar

1. Basics of Transformer Architecture and Transformer-to-RNN/T2RNN.

Content: 

• Part-1
• Explains the Transformer Architecture in Details
• Positional Encoding
• Multi-Head Attention
• Part-2.
• Explains the Advancements in Transformer Architecture (T2RNN)

Video Link: 

• Part-1: Transformer Architecture in Details  (Direct Link: https://www.youtube.com/watch?v=JMeYGYANEqU)

• Part-2: Transformer to RNN (T2RNN)  (Direct Link: https://www.youtube.com/watch?v=UHgy2faOD_M&t=417s)

2. XLNet.

Content:

• Part-1 
• BERT Vs XLNet, 
• Overview of XLNet, 
• Autoregressive Language Modeling
• Part-2
• Permutation Language Modeling for XLNet, 
• Merits and Demerits of Permutation Language Modeling
• Part-3
• Masked Attention for XLNet, 
• Two Stream Self Attention for XLNet, 
• Final Working Overview of XLNet

Video Link: 

• XLNet Made Easy Part-1 (Direct Link: https://www.youtube.com/watch?v=1yPT-aAD_a0&t=104s)

• XLNet Made Easy Part-2 (Direct Link: https://www.youtube.com/watch?v=HnlVO5n3mtY&t=2s)

• XLNet Made Easy Part-3 (Direct Link: https://www.youtube.com/watch?v=o7zbeGb2nZQ&t=29s)

3. BERT (Bidirectional Encoder Representations from Transformers).

Content: 

• Part-1
• Important Points Related to BERT, 
• BERT Embedding Layer Architecture,
• Part-2
• Bi-directional Transformers inside BERT, 
• Bidirectional Self-Attention, 
• Multi-Headed Attention
• Part-3
• Role of Layer Normalization in BERT, 
• Role of Residual Connections in BERT, 
• Overall Functioning

Video Link: 

• BERT PART-1 (Bidirectional Encoder Representations from Transformers)  (Direct Link: https://www.youtube.com/watch?v=YkE4bkSBIOw)

• BERT - Part-2 (Bidirectional Encoder Representations from Transformers)  (Direct Link: https://www.youtube.com/watch?v=BOZRkYoXW9s)

• BERT - Part-3 (Bidirectional Encoder Representations from Transformers)  (Direct Link: https://www.youtube.com/watch?v=xgxmJFqqpbU)

4. Scalability of the Transformer Architecture.

Content: 

• Part-1 Contains.
• Paper: “Transformer Quality in Linear Time”
• Gated Linear Unit
• Gated Attention Unit
• Mixed Chunk Attention
• Relative Position Bias
• Squared RELU.

Video Link: 

• Deep-Learning: How to improve the Scalability of The Transformer Architecture Part-1  (Direct Link: https://www.youtube.com/watch?v=eQgwHrtCI_s&t=1766s)

5. Transfer Learning.

Content: 

• Part-1:
• Overview of Transfer Learning
• Different types of Transfer Learning
• Part-2:
• Multi-Task Learning with sample code.

Video Link: 

• Transfer Learning Part-1  (Direct Link: https://www.youtube.com/watch?v=IS22-AoinGQ)

• Transfer Learning Part-2 (Multi-Task Learning). (Direct Link: https://www.youtube.com/watch?v=lFKbZ66KjPs)

6. Multivariate Time Series Forecasting Using Deep Learning.

Content: 

• Part-1:
• Different Types of Multivariate Time Series Forecasting Strategies.
• Multivariate Multi-Step Multi-Output Time series Forecasting
  • Strategy to prepare dataset.
  • How to write code?
• Part-2:
• Multivariate Single-Step Multi-Output Time series Forecasting
  • Strategy to prepare dataset.
  • How to write code?
• Strategy for the Future Enhancements.

Video Link: 

• Multivariate Time Series Forecasting Using Deep Learning [Part-1]  (Direct Link: https://youtu.be/xaQpLz6QkVQ)

• Multivariate Time Series Forecasting Using Deep Learning [Part-2]. (Direct Link: https://youtu.be/DLzaG4SW4pM)

7. Deep Clustering (A Self-Supervised Deep Learning Algorithm).

Content: 

• Part-1:
• Basics of Self-Supervised Algorithm
• Basics of Deep Clustering
• Part-2:
• Details of Deep Clustering
• Details of Cost Functions used in the Deep Clustering Algorithms.

Video Link: 

• Deep Clustering- Part-1 (A Self-Supervised Deep Learning Algorithm)  (Direct Link: https://youtu.be/j9KmEpaLers)

• Deep Clustering- Part-2 (A Self-Supervised Deep Learning Algorithm). (Direct Link: https://youtu.be/Ca0r0ZbeHxM)

8. Forced/Guided Learning in Deep Learning.

Content: 

• Part-1:
• Teacher Forcing
• Exposure Bias 
• Part-2:
• Scheduled Sampling
• Exposure Bias.

Video Link: 

• Forced/Guided Learning in Deep Learning Part-1  (Direct Link: https://youtu.be/FsidD3Tb1as)

• Forced/Guided Learning in Deep Learning Part-2. (Direct Link: https://youtu.be/Gz_TxwqppKg)

9. Internal Covariate Shift.

Content: 

• Part-1:
• Basics of Internal Covariate Shift
• Basics of Network Whitening
• Requirement of Normalization Techniques – e.g. Batch Normalization.
• Part-2:
• Batch Normalization
• Differentiability of ‘Batch Normalization’
• Discussion on Merits and Demerits of ‘Batch Normalization’

Video Link: 

• Internal Covariate Shift – Part-1 (with Batch Normalization)  (Direct Link: https://www.youtube.com/watch?v=VSM9ZXXS0BQ)

• Internal Covariate Shift and Batch Normalization– Part-2. (Direct Link: https://www.youtube.com/watch?v=nbDHgsyhkio)