Computer Science 690S - AI Alignment

This course will focus on modern machine learning approaches to learn from human demonstrations, preferences, feedback, and other multimodal signals, with the goal of aligning agent goals and behaviors with human values and desires. For the purposes of both safety and practicality, it is increasingly important for AI systems to be well-aligned with human users as their capabilities improve and they are deployed more frequently in real-world settings. While the standard ML paradigm assumes that learning objectives are directly provided as part of the problem specification, emerging research in alignment suggests that it is often infeasible to do so accurately, requiring such objectives to be inferred from human data. This course will provide the basic tools to address these important issues, covering topics such as behavioral cloning, inverse reinforcement learning, preference elicitation, active learning, learning from feedback, value alignment, bounded rationality, and best practices for human studies. We will examine applications including robotics, large language models, and self-driving cars.

Open to graduate Computer Science students only. NO FORMAL PREREQUISITES, BUT IT IS STRONGLY RECOMMENDED TO HAVE STRONG PROGRAMMING SKILLS, LINEAR ALGEBRA, PROBABILITY AND STATISTICS, MULTIVARIATE CALCULUS, AND GRADUATE-LEVEL MACHINE LEARNING. SEATS SAVED FOR INCOMING GRADUATE STUDENT REGISTRATION. STUDENTS NEEDING SPECIAL PERMISSION MUST REQUEST OVERRIDES VIA THE ON-LINE FORM: https://www.cics.umass.edu/academics/course-overrides.