UNIT - I Artificial Neural Networks : Introduction, Basic models of ANN, Important terminologies, Supervised Learning Networks, Perceptron Networks, Adaptive Linear Neuron, Backpropagation Network. Associative Memory Networks. Training Algorithms for pattern association, BAM and Hopfield Networks. (Chapter - 1) UNIT - II Unsupervised Learning Network : Introduction, Fixed Weight Competitive Nets, Maxnet, Hamming Network, Kohonen Self-Organizing Feature Maps, Learning Vector Quantization, Counter Propagation Networks, Adaptive Resonance Theory Networks. Special Networks - Introduction to various networks. (Chapter - 2) UNIT - III Introduction to Deep Learning : Historical Trends in Deep learning, Deep Feed - forward networks, Gradient-Based learning, Hidden Units, Architecture Design, Back-Propagation and Other Differentiation Algorithms. (Chapter - 3) UNIT - IV Regularization for Deep Learning : Parameter norm Penalties, Norm Penalties as Constrained Optimization, Regularization and Under-Constrained Problems, Dataset Augmentation, Noise Robustness, Semi-Supervised learning, Multi-task learning, Early Stopping, Parameter Typing and Parameter Sharing, Sparse Representations, Bagging and other Ensemble Methods, Dropout, Adversarial Training, Tangent Distance, Tangent Prop and Manifold, Tangent Classifier. (Chapter - 4) UNIT - V Optimization for Train Deep Models : Challenges in Neural Network Optimization, Basic Algorithms, Parameter Initialization Strategies, Algorithms with Adaptive Learning Rates, Approximate Second-Order Methods, Optimization Strategies and Meta-Algorithms. Applications : Large-Scale Deep Learning, Computer Vision, Speech Recognition, Natural Language Processing. (Chapter - 5)
