Computer Science 697L - ST-Deep Learning

Deep learning has been termed one of the leading scientific breakthroughs in recent years (MIT Technology Review). It has attracted significant industrial investment with large groups at Google, Facebook, Baidu, IBM, Microsoft etc. working on a plethora of commercial applications of deep learning. It has resulted in state of the art performance in a variety of areas, including computer vision, natural language processing, reinforcement learning, and speech recognition. This course is intended to provide students with an in-depth introduction to both theory and practice of deep learning. Topics: historical overview of neural networks and relevant ideas from brain modeling in neuroscience; classic architectures, including perceptron and backpropagation; deep learning models, including restricted Boltzmann machines, autoencoders, and convolutional neural networks. Stochastic gradient methods for training deep learning models, including Adagrad, Nesterov's method etc.; Overview of software packages for building deep learning models, including Caffe, Pylearn, Torch/Lua, and Theano. Applications of deep learning to computer vision, natural language processing, reinforcement learning, robotics, and speech recognition; Recent insights from statistical physics and high-dimensional non-convex optimization to explain deep learning. Workload: weekly reading of articles from the literature on deep learning. Final group project exploring some application of deep learning. Class presentation and participation.

Open to graduate students in Computer Science, Engineering, and Natural Sciences. PREREQUISITES: BASIC FAMILIARITY WITH AI/COGNITIVE SCIENCE/MACHINE LEARNING. BASIC KNOWLEDGE OF GRADIENT-BASED OPTIMIZATION. CALCULUS, LINEAR ALGEBRA, ALGORITHMIC ANALYSIS, PROBABILITY/STATISTICS, PROGRAMMING IN PYTHON OR EQUIVALENT HIGH LEVEL LANGUAGE. 10 SEATS HELD FOR INCOMING STUDENT REGISTRATION. STUDENTS NEEDING SPECIAL PERMISSION MUST REQUEST OVERRIDES VIA THE ON-LINE FORM: https://www.cs.umass.edu/overrides.