Pytorch

• Prediction Problems
  • Regression
• Classification Problems
  • The Softmax
  • Cross-Entropy Loss
• Attention Mechanism
  • Attention Pooling
  • Large-Scale Pretraining with Transformers
• Activation Functions
• Data Loading Pipeline
  • torch.utlis.data Package
  • Map-style Dataset Pipeline
  • Iterable-style Dataset Pipeline
• Recurrent Neural Networks
• Transformer
  • Word Embedding - nn.Embedding
  • Positional Encoding - Custom nn.Module
  • Positional Embedding - Word embedding + Postional Encoding
  • Self-Attention
  • Multi-Head Attention - nn.MultiheadAttention
  • Masked Multi-Head Attention(Decoder) Causal-Attention
  • Encoder-Decoder Attention(Decoder) Cross-Attention
  • Residual Connection
  • Layer Normalization
  • Positionwise Feed-Forward Network
  • Targets Shifted Right