Syllabus Deep Learning - (410251) Credit Examination Scheme : 03 In-Sem (Paper) : 30 Marks End-Sem (Paper) : 70 Marks Unit I Foundations of Deep learning What is machine learning and deep learning ?,Supervised and Unsupervised Learning, bias variance tradeoff, hyper parameters, under/over fitting regularization, Limitations of machine learning, History of deep learning, Advantage and challenges of deep learning. Learning representations from data, Understanding how deep learning works in three figures, Common Architectural Principles of Deep Network, Architecture Design, Applications of Deep learning, Introduction and use of popular industry tools such as TensorFLow, Keras, PyTorch, Caffe, Shogun. (Chapter - 1) Unit II Deep Neural Networks (DNNs) Introduction to Neural Networks : The Biological Neuron, The Perceptron, Multilayer Feed-Forward Networks, Training Neural Networks : Backpropagation and Forward propagation Activation Functions : Linear, Sigmoid, Tannh, Hard Tanh, Softmax, Rectified Linear, Loss Functions : Loss Function Notation, Loss Functions for Regression, Loss Functions for Classification, Loss Functions for Reconstruction, Hyperparameters : Learning Rate, Regularization, Momentum, Sparsity, Deep Feedforward Networks - Example of Ex OR, Hidden Units, cost functions, error backpropagation, Gradient-Based Learning, Implementing Gradient Descent, vanishing and Exploding gradient descent, Sentiment Analysis, Deep Learning with Pytorch, Jupyter, colab. (Chapter - 2) Unit III Convolution Neural Network (CNN) Introduction, CNN architecture overview, The Basic Structure of a Convolutional Network- Padding, Strides, Typical Settings, the ReLU layer, Pooling, Fully Connected Layers, The Interleaving between Layers, Local Response Normalization, Training a Convolutional Network. (Chapter - 3) Unit IV Recurrent Neural Networks (RNN) Recurrent and Recursive Nets : Unfolding Computational Graphs, Recurrent Neural Networks, Bidirectional RNNs, Encoder-Decoder Sequence-to-Sequence Architectures, Deep Recurrent Networks, Recursive Neural Networks, The Challenge of Long-Term Dependencies, Echo State Networks, Leaky Units and Other Strategies for Multiple Time Scales, The Long Short-Term Memory and Other Gated RNNs, Optimization for Long-Term Dependencies, Explicit Memory. Practical Methodology : Performance Metrics, Default Baseline Models, Determining Whether to Gather More Data, Selecting Hyper parameters. (Chapter - 4) Unit V Deep Generative Models Introduction to deep generative model, Boltzmann Machine, Deep Belief Networks, Generative adversarial network (GAN), discriminator network, generator network, types of GAN, Applications of GAN networks. (Chapter - 5) Unit VI Reinforcement Learning Introduction of deep reinforcement learning, Markov Decision Process, basic framework of reinforcement learning, challenges of reinforcement learning, Dynamic programming algorithms for reinforcement learning, Q Learning and Deep Q-Networks, Deep Q recurrent networks, Simple reinforcement learning for Tic-Tac-Toe. (Chapter - 6)
