CS 294-277, Robots That Learn (Fall 2024)

Logistics

UC Berkeley Course Number 34334

Time: Mondays 3-5PM

Location: Room 1203 @ Berkeley Way West

Instructor: Jitendra Malik

TA: Toru Lin

~ Welcome from Jitendra ~

Robotics has been late to the deep learning revolution compared to computer vision and natural language processing, mainly because “big data” is not so readily available. However significant advances have been made in the last few years, and the purpose of this class is to present a coherent framework for studying these advances.

My goal is to build machines that can emulate the remarkable capabilities of humans and other animals at motor control, defined as connecting perception to action in the physical world. Hence there will be a distinct preference for legs over wheels, multi-finger hands over parallel jaw grippers, rich visual and tactile perception over minimal sensing, humanoid robots in the home over specialist robots on the factory floor. The course itself will have three main parts (1) biological motor control basics for inspiration (2) main paradigms for robot motor skill acquisition (3) case studies of locomotion, navigation and manipulation.

Prerequisites

This is a graduate level class, and prerequisite knowledge at the level of Deep Learning (Bishop & Bishop), and Reinforcement Learning (Sutton & Barto) will be assumed.

We have now closed class enrollment. Here is a list of self-evaluation questions for interested students to see whether they are ready for the class.

Course Format

Each session consists of a two 1-hour lectures with a 10 minute break in between, i.e. 3:10-4 PM and 4:10-5PM. We will use notations A/B to denote the sub-sessions: e.g. Lecture 1A, 1B, 2A, 2B, …

Schedule

The following schedule is a tentative assignment and will evolve in real time. Weekly materials will be added.

Please see below (“Reading Materials”) for link to reading assignment submission form.

Coursework

Weekly Reading Assignment: For every weekly reading, each student should come up with 2 multiple choice questions, and supply with answers. We will send out a Google form for submission each week.

Lecture Scribing: For each lecture, two student scribes will organize lecture notes in LaTeX. The students can decide to submit a single note together or individually; grades will be assigned based on note quality. The lecture notes should be ready by the same Friday. Sign-up sheet here.

Note: since the lecture content does not necessarily align with lecture title, each scribe is only required to cover the time slot they have signed up for.

Midterm: During the first hour of our 11/18 class, we will have a mid-term exam based on questions sourced from the weekly reading assignments so far. A total of about 30 multiple-choice questions will be given. One 8.5”x11”, double-sided cheat sheet will be permitted.

Final Project: The goal of the final project is to explore and push the boundaries of robot learning, choosing from topics presented in this course. Here are a few examples of possible project formats: proposal and evaluation of new algorithms, investigation of a robotic application, benchmarking a range of existing methods, etc. Ideally, the project covers interesting new ground and could be the foundation for a future conference paper submission or product. The project can be done in groups of 1-4 people. Note that our expectations will scale linearly with the number of people in the group.

❗Final Project Logistics

Project Proposal: To keep track of your final project progress, please submit a 1 page project proposal using this template by EOD 11/19 (the day after midterm). Please submit through a Google doc shared with the course instructor and TA, so we can give feedback and suggestions. Note that this proposal will not be graded.

Project Presentation: We will have the final project presentations on 12/9 during our regular class time. This will serve as the most important basis for grading.

Project Report: The final project report will be due by EOD 12/13. Please use CoRL 2024 format for the project report, with a maximum of 4 pages. We would like you to focus on the problem setting, why it matters and what’s interesting/novel about it, your approach, your results, analysis of results, limitations, and future directions. Cite and briefly survey prior work as appropriate but don’t re-write prior work when not directly relevant to understanding your approach. References will not be counted against the 4 pages.

❗Assignment Deadlines

Background Materials

Reading materials

Lecture 1

Lecture 1B

(It is not expected to read the Uchida-Delp book, but we will cover a couple of chapters from it.)

Lecture 2

[Reading Assignment Submission Form]

Lecture 2A

In advance of lecture 2A, students should try to familiarize themselves with how 3D rotations and translations are represented. We would like students to learn about “exponential coordinates” - how a rotation matrix is the exponential of a skew-symmetric matrix corresponding to the axis of rotation, and when rotation is accompanied by translation, we use the exponential of a twist. This formalism results in an elegant way to specify the forward kinematics of a robot using the product of matrix exponentials. The Li-Murray-Sastry textbook and the Lynch-Park textbook are good sources. You can find lectures on YouTube for Lynch-Park. I recommend the ones corresponding to Chapter 3. [link]

Lecture 2B

Lecture 3

[Reading Assignment Submission Form]

Lecture 3A

Lecture 3B

Lecture 4

[Reading Assignment Submission Form]

Lecture 4A

Lecture 4B

Lecture 5

[Reading Assignment Submission Form]

Lecture 6

[Reading Assignment Submission Form]

Lecture 7

[Reading Assignment Submission Form]

Lecture 8

[Reading Assignment Submission Form]

Lecture 9

[Reading Assignment Submission Form]

Lecture 10

[Reading Assignment Submission Form]

Lecture 11

[Reading Assignment Submission Form]

Lecture 12

[Reading Assignment Submission Form]