MCE 564 - Robotic Perception
Graduate course, AYBÜ, Mechanical Engineering, 2024
This course introduces perception techniques for autonomous mobile robots, with a focus on how to process data from sensors like IMUs, cameras, and lidars for tasks such as orientation estimation, mapping, and navigation.
Course Objectives
The goal is to equip students with:
- An understanding of state estimation and perception for robotics.
- Practical experience in working with real-world sensor data.
- Programming skills to implement core algorithms in sensor fusion, optical flow, and point cloud processing.
Learning Outcomes
By the end of this course, students will be able to:
- Represent robot state using orientation and position models.
- Process IMU, camera, and lidar data for localization and mapping.
- Implement visual-inertial pose estimation and point cloud registration.
- Understand and apply projective geometry and image-based motion estimation.
Weekly Course Plan
| Week(s) | Subject |
|---|---|
| 1–2 | Representations of position and orientation (axis-angle, SO(3), quaternions, exponential map, Euler angles), rigid body and homogeneous transformations, angular velocity |
| 3 | IMUs: calibration, characterization, estimation |
| 4 | Camera model, lens distortion, intrinsic calibration |
| 5–7 | Projective geometry, transformations, objects in projective space, homography, PnP problem, projective pose estimation |
| 8–9 | Visual features (edges, corners), feature matching, epipolar geometry, RANSAC family, optical flow and velocity estimation |
| 10–12 | Lidar and point cloud processing, data structures, scan matching, registration |
| 13–14 | Sensor fusion techniques: odometry using lidar, IMU, and visual sensors; environmental assumptions (2D, 2.5D, 3D) |
Reference Books
- Ma, Y., Soatto, S., Kosecka, J., & Sastry, S. S. An Invitation to 3D Vision: From Images to Geometric Models. Springer, 2012.
- Hartley, R., & Zisserman, A. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.
- Botsch, M., et al. Polygon Mesh Processing. CRC Press, 2010.