Syllabus Deep Learning - (AD3501) UNIT I DEEP NETWORKS BASICS Linear Algebra : Scalars-Vectors-Matrices and tensors; Probability Distributions-Gradient-based Optimization-Machine Learning Basics : Capacity-Overfitting and underfitting-Hyperparameters and validation sets-Estimators-Bias and variance-Stochastic gradient descent-Challenges motivating deep learning; Deep Networks : Deep feedforward networks; Regularization - Optimization. (Chapter - 1) UNIT II CONVOLUTIONAL NEURAL NETWORKS Convolution Operation-Sparse Interactions-Parameter Sharing-Equivariance-Pooling-Convolution Variants : Strided-Tiled-Transposed and dilated convolutions ; CNN Learning : Nonlinearity Functions-Loss Functions-Regularization-Optimizers-Gradient Computation. (Chapter - 2) UNIT III RECURRENT NEURAL NETWORKS Unfolding Graphs-RNN Design Patterns : Acceptor-Encoder-Transducer; Gradient Computation- Sequence Modeling Conditioned on Contexts-Bidirectional RNN-Sequence to Sequence RNN-Deep Recurrent Networks-Recursive Neural Networks-Long Term Dependencies ; Leaky Units : Skip connections and dropouts ; Gated Architecture : LSTM. (Chapter - 3) UNIT IV MODEL EVALUATION Performance metrics-Baseline Models-Hyperparameters : Manual Hyperparameter-Automatic Hyperparameter-Grid search-Random search-Debugging strategies. (Chapter - 4) UNIT V AUTOENCODERS AND GENERATIVE MODELS Autoencoders : Undercomplete autoencoders-Regularized autoencoders-Stochastic encoders and decoders-Learning with autoencoders ; Deep Generative Models : Variational autoencoders-Generative adversarial networks. (Chapter - 5)
