Advanced Deep Learning

This site is dedicated to the simplest video tutorials on Advance Topics of Deep Learning. I have used my knowledge and experience to prepare these tutorials. All feedback and suggestions are welcome (email me at nirajrkumar@gmail.com or nirajrkumar@yahoo.com).

As, scientific development is an endless process, so I will keep updating it. Clicking on the link will drive you to the YouTube page for related content. Or You can use the link: https://www.youtube.com/c/DrNirajRKumar

Content: 

  • Part-1
  • Explains the Transformer Architecture in Details
  • Positional Encoding
  • Multi-Head Attention
  • Part-2.
  • Explains the Advancements in Transformer Architecture (T2RNN)

Video Link: 

Content:

  • Part-1 
  • BERT Vs XLNet, 
  • Overview of XLNet, 
  • Autoregressive Language Modeling
  • Part-2
  • Permutation Language Modeling for XLNet, 
  • Merits and Demerits of Permutation Language Modeling
  • Part-3
  • Masked Attention for XLNet, 
  • Two Stream Self Attention for XLNet, 
  • Final Working Overview of XLNet

Video Link: 

Content: 

  • Part-1
  • Important Points Related to BERT, 
  • BERT Embedding Layer Architecture,
  • Part-2
  • Bi-directional Transformers inside BERT, 
  • Bidirectional Self-Attention, 
  • Multi-Headed Attention
  • Part-3
  • Role of Layer Normalization in BERT, 
  • Role of Residual Connections in BERT, 
  • Overall Functioning

Video Link: 

Content: 

  • Part-1 Contains.
  • Paper: “Transformer Quality in Linear Time”
  • Gated Linear Unit
  • Gated Attention Unit
  • Mixed Chunk Attention
  • Relative Position Bias
  • Squared RELU.

Video Link: 

Content: 

  • Part-1:
  • Overview of Transfer Learning
  • Different types of Transfer Learning
  • Part-2:
  • Multi-Task Learning with sample code.

Video Link: 

6. Multivariate Time Series Forecasting Using Deep Learning.

Content: 

  • Part-1:
  • Different Types of Multivariate Time Series Forecasting Strategies.
  • Multivariate Multi-Step Multi-Output Time series Forecasting
    • Strategy to prepare dataset.
    • How to write code?
  • Part-2:
  • Multivariate Single-Step Multi-Output Time series Forecasting
    • Strategy to prepare dataset.
    • How to write code?
  • Strategy for the Future Enhancements.

Video Link: 

7. Deep Clustering (A Self-Supervised Deep Learning Algorithm).

Content: 

  • Part-1:
  • Basics of Self-Supervised Algorithm
  • Basics of Deep Clustering
  • Part-2:
  • Details of Deep Clustering
  • Details of Cost Functions used in the Deep Clustering Algorithms.

Video Link: 

8. Forced/Guided Learning in Deep Learning.

Content: 

  • Part-1:
  • Teacher Forcing
  • Exposure Bias 
  • Part-2:
  • Scheduled Sampling
  • Exposure Bias.

Video Link: 

9. Internal Covariate Shift.

Content: 

  • Part-1:
  • Basics of Internal Covariate Shift
  • Basics of Network Whitening
  • Requirement of Normalization Techniques – e.g. Batch Normalization.
  • Part-2:
  • Batch Normalization
  • Differentiability of ‘Batch Normalization’
  • Discussion on Merits and Demerits of ‘Batch Normalization’

Video Link: