Difference between revisions of "Cognitive Robotics Lectures and Labs"
From David Vernon's Wiki
| Line 27: | Line 27: | ||
| OpenCV | | OpenCV | ||
| Kragic and Vincze (2010). Szeliski (2010), Sections 1.1, 1.2, 2.3, 3.2, 4.2. Vernon (1991), Sections 2.1, 2.1 | | Kragic and Vincze (2010). Szeliski (2010), Sections 1.1, 1.2, 2.3, 3.2, 4.2. Vernon (1991), Sections 2.1, 2.1 | ||
| − | | | + | |Image acquisition and image processing using OpenCV |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 2 | | 2 | ||
| Line 36: | Line 36: | ||
| OpenCV | | OpenCV | ||
| Szeliski (2010), Sections 3.1.2, 3.3.4, 4.3.2. Vernon (1991), Section 3.1, 3.2, 3.3, 4.2.1, 4.2.2, 5.3, 6.4. | | Szeliski (2010), Sections 3.1.2, 3.3.4, 4.3.2. Vernon (1991), Section 3.1, 3.2, 3.3, 4.2.1, 4.2.2, 5.3, 6.4. | ||
| − | | | + | | Hough transforms and colour segmentation using OpenCV |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 2 | | 2 | ||
| Line 45: | Line 45: | ||
| OpenCV, Vienna University of Technology BLORT Library | | OpenCV, Vienna University of Technology BLORT Library | ||
| Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1. | | Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1. | ||
| − | | | + | | Face detection and object recognition using OpenCV |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 3 | | 3 | ||
| Line 54: | Line 54: | ||
|OpenCV | |OpenCV | ||
|Szeliski (2010), Sections 2.1, 11.1, 11.2, 11.3. Vernon (1991), Section 8.6, 9.4.2. | |Szeliski (2010), Sections 2.1, 11.1, 11.2, 11.3. Vernon (1991), Section 8.6, 9.4.2. | ||
| − | | | + | |Camera calibration |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 3 | | 3 | ||
| Line 63: | Line 63: | ||
|Vienna University of Technology RGB-D Segmentation Library and V4R Library | |Vienna University of Technology RGB-D Segmentation Library and V4R Library | ||
|Szeliski (2010), Sections 12.4. Point Cloud Library tutorial. | |Szeliski (2010), Sections 12.4. Point Cloud Library tutorial. | ||
| − | | | + | |Analysis of point cloud data from RGB-D camera |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 4 | | 4 | ||
| Line 70: | Line 70: | ||
|Differential drive locomotion. Forward and inverse kinematics. Holonomic and non-holonomic constraints. Cozmo mobile robot. | |Differential drive locomotion. Forward and inverse kinematics. Holonomic and non-holonomic constraints. Cozmo mobile robot. | ||
|Anki Cozmo mobile robot | |Anki Cozmo mobile robot | ||
| − | |Anki Cozmo SDK | + | |Anki Cozmo SDK, OpenCV |
|Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | |Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | ||
| − | | | + | |Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection) |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 4 | | 4 | ||
| Line 79: | Line 79: | ||
|Map representation. Probabilistic map-based localization. Landmark-based localization. | |Map representation. Probabilistic map-based localization. Landmark-based localization. | ||
|Anki Cozmo mobile robot | |Anki Cozmo mobile robot | ||
| − | |Anki Cozmo SDK | + | |Anki Cozmo SDK, OpenCV |
|Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | |Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | ||
| − | | | + | |Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection) |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 5 | | 5 | ||
| Line 88: | Line 88: | ||
|SLAM: simultaneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. | |SLAM: simultaneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. | ||
|Anki Cozmo mobile robot | |Anki Cozmo mobile robot | ||
| − | |Anki Cozmo SDK | + | |Anki Cozmo SDK, OpenCV |
|Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | |Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | ||
| − | | | + | |Cozmo locomotion (e.g. program Cozmo to follow a cube at a fixed distance; when it stops moving, pick it up) |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 5 | | 5 | ||
| 10 | | 10 | ||
|Mobile robots IV | |Mobile robots IV | ||
| − | | Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. | + | |Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. |
| − | | | + | | |
| − | | | + | | |
| − | | | + | | |
| − | | | + | |Cozmo navigation |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 6 | | 6 | ||
| Line 108: | Line 108: | ||
|Arduino sketch programs for Lynxmotion | |Arduino sketch programs for Lynxmotion | ||
|Paul (1981), Chapters 1 & 2. | |Paul (1981), Chapters 1 & 2. | ||
| − | | | + | |Move end-effector along various paths in joint space |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 6 | | 6 | ||
| Line 117: | Line 117: | ||
|Arduino sketch programs for Lynxmotion | |Arduino sketch programs for Lynxmotion | ||
|Paul (1981), Chapter 3. | |Paul (1981), Chapter 3. | ||
| − | | | + | |Move end-effector along various paths in Cartesian frame of reference |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 7 | | 7 | ||
| Line 126: | Line 126: | ||
|Arduino sketch programs for Lynxmotion | |Arduino sketch programs for Lynxmotion | ||
|Vernon (1991), Sections 8.1-8.4. | |Vernon (1991), Sections 8.1-8.4. | ||
| − | | | + | |Compute the pose of a light cube |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 7 | | 7 | ||
| Line 135: | Line 135: | ||
|Arduino sketch programs for Lynxmotion | |Arduino sketch programs for Lynxmotion | ||
|Vernon (1991), Sections 8.1-8.4 | |Vernon (1991), Sections 8.1-8.4 | ||
| − | | | + | |Implement a program to move light cube from one position/pose to another position/pose |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| 8 | | 8 | ||
Revision as of 07:48, 21 December 2016
| Week | Lecture | Topic | Material covered | Required hardware | Required software | Pre-class reading | Homework exercises |
|---|---|---|---|---|---|---|---|
| 1 | 1 | Cognitive robotics | Introduction to AI and cognition in robotics. Industrial requirements. Artificial cognitive systems. Cognitivist, emergent, and hybrid paradigms in cognitive science. Autonomy. | None | None | Vernon (2014), Chapters 1, 2, and 4. | Installation of software tools. |
| 1 | 2 | Robot vision I | Optics, sensors, and image formation. Image acquisition. Image filtering. Edge detection. | USB camera | OpenCV | Kragic and Vincze (2010). Szeliski (2010), Sections 1.1, 1.2, 2.3, 3.2, 4.2. Vernon (1991), Sections 2.1, 2.1 | Image acquisition and image processing using OpenCV |
| 2 | 3 | Robot vision II | Segmentation. Hough transform: line, circle, and generalized transform; extension to codeword features. Colour-based segmentation. | USB camera | OpenCV | Szeliski (2010), Sections 3.1.2, 3.3.4, 4.3.2. Vernon (1991), Section 3.1, 3.2, 3.3, 4.2.1, 4.2.2, 5.3, 6.4. | Hough transforms and colour segmentation using OpenCV |
| 2 | 4 | Robot vision III | Object recognition. Interest point operators. Gradient orientation histogram - SIFT descriptor. Colour histogram intersection. Haar features, boosting, face detection. | USB camera | OpenCV, Vienna University of Technology BLORT Library | Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1. | Face detection and object recognition using OpenCV |
| 3 | 5 | Robot vision IV | Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. Stereo vision. Epipolar geometry. Structured light & RGB-D cameras. | USB camera | OpenCV | Szeliski (2010), Sections 2.1, 11.1, 11.2, 11.3. Vernon (1991), Section 8.6, 9.4.2. | Camera calibration |
| 3 | 6 | Robot vision V | Visual attention. Plane pop-out. RANSAC. Differential geometry. Surface normals and Gaussian sphere. Point clouds. 3D descriptors. | Kinect RGBD sensor | Vienna University of Technology RGB-D Segmentation Library and V4R Library | Szeliski (2010), Sections 12.4. Point Cloud Library tutorial. | Analysis of point cloud data from RGB-D camera |
| 4 | 7 | Mobile robots I | Differential drive locomotion. Forward and inverse kinematics. Holonomic and non-holonomic constraints. Cozmo mobile robot. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection) |
| 4 | 8 | Mobile robots II | Map representation. Probabilistic map-based localization. Landmark-based localization. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection) |
| 5 | 9 | Mobile robots III | SLAM: simultaneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo locomotion (e.g. program Cozmo to follow a cube at a fixed distance; when it stops moving, pick it up) |
| 5 | 10 | Mobile robots IV | Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. | Cozmo navigation | |||
| 6 | 11 | Robot arms I | Homogeneous transformations. Frame-based pose specification. Denavit-Hartenberg specifications. Robot kinematics. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Paul (1981), Chapters 1 & 2. | Move end-effector along various paths in joint space |
| 6 | 12 | Robot arms II | Analytic inverse kinematics. Iterative approaches. Kinematic structure learning. Kinematics structure correspondences. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Paul (1981), Chapter 3. | Move end-effector along various paths in Cartesian frame of reference |
| 7 | 13 | Robot arms III | Robot manipulation. Frame-based task specification. Vision-based pose estimation. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Vernon (1991), Sections 8.1-8.4. | Compute the pose of a light cube |
| 7 | 14 | Robot arms IV | Language-based programming. Programming by demonstration. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Vernon (1991), Sections 8.1-8.4 | Implement a program to move light cube from one position/pose to another position/pose |
| 8 | 15 | ||||||
| 8 | 16 | ||||||
| 9 | 17 | Cognitive architectures I | Role and requirements; cognitive architecture schemas; example cognitive architectures including Soar, ACT-R, Clarion, LIDA, and ISAC. | Vernon (2014) Chapter 3. Chella et al. (2013). Scheutz et al. (2013). Vernon et al. (2016). | Group discussion on which cognitive architectures are suitable for cognitive robotics | ||
| 9 | 18 | Cognitive architectures II | CRAM: Cognitive Robot Abstract Machine.CRAM Plan Language (CPL). KnowRob knowledge processing and reasoning | CRAM | Beetz et al. (2010) | Exercises on CRAM test programs | |
| 10 | 19 | Learning and development I | Supervised, unsupervised, and reinforcement learning. Hebbian learning. | MaxHebb library | Harmon and Harmon (1997) | Exercise on Hebbian learning | |
| 10 | 20 | Learning and development II | Predictive sequence learning (PSL). | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Sun and Giles (2001). Billing et al. (2011, 2016). | Exercises on PSL test programs |
| 11 | 21 | Learning and development III | Cognitive development in humans and robots. | Anki Cozmo mobile robot | Anki Cozmo SDK | Lungarella et al. (2003). Asada et al. (2009). Cangelosi and Schlesinger (2015), Chapters 1 & 2. | Exercises on PSL test programs |
| 11 | 22 | Learning and development IV | Value systems for developmental and cognitive robots. | Merrick (2016). Vernon et al. (2016). | Group discussion on cognitive development in robotics | ||
| 12 | 23 | Memory and Prospection | Declarative vs. procedural memory. Semantic memory. Episodic memory | Anki Cozmo mobile robot | Anki Cozmo SDK, CINDY library, OpenCV | Vernon (2014), Chapter 7. | Implement episodic memory on Cozmo |
| 12 | 24 | Internal simulation I | Forward and inverse models, internal simulation hypothesis, internal simulation with PSL | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Vernon (2014), Chapter 8. Billing et al. (2016). | Exercises on PSL test programs |
| 13 | 25 | Internal simulation II | HAMMER cognitive architecture | Boost, Imperial College London HAMMER library | Demiris and Khadhouri (2006). Sarabia et al. (2011). | Exercise on HAMMER tutorial using the ICL library | |
| 13 | 26 | Visual attention | Visual attention. Spatial attention vs. selective attention. Saliency functions. Selective Tuning. Overt attention. Inhibition of return. Habituation. Top-down attention. | Anki Cozmo mobile robot | Anki Cozmo SDK, CINDY library | Borji and Itti (2013). | Implement visual attention on Cozmo |
| 14 | 27 | Social interaction I | Joint action. Joint attention. Shared intention. Shared goals. Perspective taking. Theory of mind. | Orabec Astra RGBD sensor | Ubuntu 14.04, ROS, Imperial College London Perspective Taking library | Vernon (2014), Chapter 9. | Exercise on perspective taking using the ICL library |
| 14 | 28 | Social interaction II | Action and intention recognition. Learning from demonstration. Humanoid robotics. | Orabec Astra RGBD sensor | PSL library | Billard et al. (2008). Argall (2009). | Exercise on learning from demonstration using the PSL library |
Back to Cognitive Robotics
