Optimal control solution techniques for systems with known and unknown dynamics. Dynamic programming, HamiltonJacobi reachability, and direct and indirect methods for trajectory optimization. Introduction to model predictive control. Modelbased reinforcement learning, and connections between modern reinforcement learning in continuous spaces and fundamental optimal control ideas.
Prof. Marco Pavone 

Lectures meet on Mondays and Wednesdays from 1:30 PM to 2:50 PM in Herrin T175.
Professor Pavone's office hours are Mondays 35pm in Durand 261.
CA office hours are Tuesdays 1:30pm3pm (Durand 353) and Fridays 121:30pm (Durand 270)
The class syllabus can be found here.
Subject to change. Lecture notes are available here. We will try to have the lecture notes updated before the class.
Week  Topic  Lecture Slides 

1 
Course overview, unconstrained nonlinear optimization
Constrained nonlinear optimization Recitation: Linear dynamical systems HW1 out 
Lecture 1
Lecture 2 
2 
Dynamic programming, discrete LQR
Iterative LQR, DDP, and LQG HW2 out, HW1 due 
Lecture 3
Lecture 4 
3 
HamiltonJacobiBellman and HamiltonJacobiIsaacs equations
Reachability analysis HW3 out, HW2 due 
Lecture 5
Lecture 6 
4 
Calculus of variations
Indirect methods for optimal control HW4 out, HW3 due 
Lecture 7
Lecture 8 
5 
Pontryagin's maximum principle
Numerical aspects of indirect optimal control Midterm (May 2, evening) HW5 out, HW4 due 
Lecture 9
Lecture 10 
6 
Direct methods for optimal control
Direct collocation and sequential quadratic programming Recitation: Linear, quadratic, convex, and mixedinteger linear programming HW6 out, HW5 due 
Lecture 11
Lecture 12 Recitation 
7 
Introduction to model predictive control
Feasibility and stability of MPC HW7 out, HW6 due 
Lecture 13
Lecture 14 
8 
Adaptive optimal control, dual control, adaptive LQR
Modelbased reinforcement learning: linear methods HW8 out, HW7 due 
Lecture 15
Lecture 16 
9 
Nonlinear regression fundamentals

Lecture 17 
10 
Modelbased reinforcement learning: nonlinear methods
Intro to modelfree RL, connections to modelbased RL HW8 due 
Lecture 18
Lecture 19 