Difference between revisions of "Cognitive Robotics Lectures and Labs"

From David Vernon's Wiki
Jump to: navigation, search
(Created page with "{| class="wikitable" |- ! Week ! Lecture ! Topic ! Material covered ! Required hardware ! Required software ! Pre-class reading ! Lab exercises |- | 1 | 1 | | | | | |- | 1 | 2...")
 
Line 12: Line 12:
 
| 1
 
| 1
 
| 1
 
| 1
−
|
+
| Cognitive robotics
−
|
+
| Introduction to AI and cognition in robotics. Industrial requirements. Artificial cognitive systems. Cognitivist, emergent, and hybrid paradigms in cognitive science. Autonomy.
−
|
+
| None
−
|
+
| None
−
|
+
| Vernon (2014), Chapters 1, 2, and 4.
 +
| Installation of software tools.
 
|-
 
|-
 
| 1
 
| 1
−
| 2
+
| 2  
−
|
+
| Robot Vision I
−
|
+
| Optics, sensors, and image formation. Image acquisition. Image filtering. Edge detection.
−
|
+
| Orabec Astra RGBD sensor
−
|
+
| OpenCV
−
|
+
| Kragic and Vincze (2010). Szeliski (2010), Sections 1.1, 1.2, 2.3, 3.2, 4.2. Vernon (1991), Sections 2.1, 2.1
 +
|Exercises on image acquisition and image processing using OpenCV
 
|-
 
|-
 
| 2
 
| 2
 
| 3
 
| 3
 +
|
 
|
 
|
 
|
 
|
Line 36: Line 39:
 
| 2
 
| 2
 
| 4
 
| 4
 +
|
 
|
 
|
 
|
 
|
Line 44: Line 48:
 
| 3
 
| 3
 
| 5
 
| 5
 +
|
 
|
 
|
 
|
 
|
Line 52: Line 57:
 
| 3
 
| 3
 
| 6
 
| 6
 +
|
 
|
 
|
 
|
 
|
Line 60: Line 66:
 
| 4
 
| 4
 
| 7
 
| 7
 +
|
 
|
 
|
 
|
 
|
Line 68: Line 75:
 
| 4
 
| 4
 
| 8
 
| 8
 +
|
 
|
 
|
 
|
 
|
Line 76: Line 84:
 
| 5
 
| 5
 
| 9
 
| 9
 +
|
 
|
 
|
 
|
 
|
Line 84: Line 93:
 
| 5
 
| 5
 
| 10
 
| 10
 +
|
 
|
 
|
 
|
 
|
Line 92: Line 102:
 
| 6
 
| 6
 
| 11
 
| 11
 +
|
 
|
 
|
 
|
 
|
Line 100: Line 111:
 
| 6
 
| 6
 
| 12
 
| 12
 +
|
 
|
 
|
 
|
 
|
Line 108: Line 120:
 
| 7
 
| 7
 
| 13
 
| 13
 +
|
 
|
 
|
 
|
 
|
Line 116: Line 129:
 
| 7
 
| 7
 
| 14
 
| 14
 +
|
 
|
 
|
 
|
 
|
Line 124: Line 138:
 
| 8
 
| 8
 
| 15
 
| 15
 +
|
 
|
 
|
 
|
 
|
Line 132: Line 147:
 
| 8
 
| 8
 
| 16
 
| 16
 +
|
 
|
 
|
 
|
 
|
Line 140: Line 156:
 
| 9
 
| 9
 
| 17
 
| 17
 +
|
 
|
 
|
 
|
 
|
Line 148: Line 165:
 
| 9
 
| 9
 
| 18
 
| 18
 +
|
 
|
 
|
 
|
 
|
Line 156: Line 174:
 
| 10
 
| 10
 
| 19
 
| 19
 +
|
 
|
 
|
 
|
 
|
Line 164: Line 183:
 
| 10
 
| 10
 
| 20
 
| 20
 +
|
 
|
 
|
 
|
 
|
Line 172: Line 192:
 
| 11
 
| 11
 
| 21
 
| 21
 +
|
 
|
 
|
 
|
 
|
Line 180: Line 201:
 
| 11
 
| 11
 
| 22
 
| 22
 +
|
 
|
 
|
 
|
 
|
Line 188: Line 210:
 
| 12
 
| 12
 
| 23
 
| 23
 +
|
 
|
 
|
 
|
 
|
Line 196: Line 219:
 
| 12
 
| 12
 
| 24
 
| 24
 +
|
 
|
 
|
 
|
 
|
Line 204: Line 228:
 
| 13
 
| 13
 
| 25
 
| 25
 +
|
 
|
 
|
 
|
 
|
Line 212: Line 237:
 
| 13
 
| 13
 
| 26
 
| 26
 +
|
 
|
 
|
 
|
 
|
Line 220: Line 246:
 
| 14
 
| 14
 
| 27
 
| 27
 +
|
 
|
 
|
 
|
 
|
Line 235: Line 262:
 
|
 
|
 
|}
 
|}
 +
 +
 +
2 3 Robot vision II Segmentation. Hough transform: line, circle, and generalized transform; extension to codeword features. Colour-based segmentation. Orabec Astra RGBD sensor OpenCV Szeliski (2010), Sections 3.1.2, 3.3.4, 4.3.2. Vernon (1991), Section 3.1, 3.2, 3.3, 4.2.1, 4.2.2, 5.3, 6.4. Exercises on Hough transforms and colour segmentation using OpenCV
 +
2 4 Robot vision III Object recognition. Interest point operators. Gradient orientation histogram - SIFT descriptor. Colour histogram intersection. Haar features, boosting, face detection. Orabec Astra RGBD sensor "OpenCV
 +
TU Wien BLORT Library" Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1. Exercises on face detection and object recognition using OpenCV
 +
3 5 Robot vision IV Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. Stereo vision. Epipolar geometry. Structured light & RGB-D cameras. Orabec Astra RGBD sensor OpenCV Szeliski (2010), Sections 2.1, 11.1, 11.2, 11.3. Vernon (1991), Section 8.6, 9.4.2. Exercise on camera calibration
 +
3 6 Robot vision V Visual attention. Plane pop-out. RANSAC. Differential geometry. Surface normals and Gaussian sphere. Point clouds. 3D descriptors. Orabec Astra RGBD sensor TU Wien RGB-D Segmentation Library and V4R Library Szeliski (2010), Sections 12.4. Point Cloud Library tutorial. Exercises analysing point cloud data from RGB-D camera
 +
4 7 Mobile robots I Differential drive locomotion. Forward and inverse kinematics. Holonomic and non-holonomic constraints.  Cozmo mobile robot. Anki Cozmo mobile robot Anki Cozmo SDK "Python tutorial. Cozmo SDK API.
 +
OpenCV Python tutorial." Exercises on Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection)
 +
4 8 Mobile robots II Map representation. Probabilistic map-based localization. Landmark-based localization. Anki Cozmo mobile robot Anki Cozmo SDK "Python tutorial. Cozmo SDK API.
 +
OpenCV Python tutorial." Exercises on Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection)
 +
5 9 Moble robots III SLAM: simulataneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. Anki Cozmo mobile robot Anki Cozmo SDK "Python tutorial. Cozmo SDK API.
 +
OpenCV Python tutorial." Exercises on Cozmo locomotion (e.g. program Cozmo to follow a cube at a fixed distance; wnen it stops moving, pick it up and bring it home)
 +
5 10 Moble robots IV Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. Orabec Astra RGBD sensor Ubuntu 14.04, ROS, Gazebo, Java 7 "ROS tutorials. Protege4Pizzas10Minutes tutorial.
 +
Manchester OWL tutorial." ROS Turtlebot view planning simulation
 +
6 11 Robot arms I Homogeneous transformations. Frame-based pose specification. Denavit-Hartenberg specifications. Robot kinematics. "Lynxmotion 5DoF arm
 +
" Arduino sketch programs for Lynxmotion Paul (1981), Chapters 1 & 2. Exercises to move end-effector along various paths in joint space
 +
6 12 Robot arms II Analytic inverse kinematics. Iterative approaches. Kinematic structure learning.  Kinematics structure correspondences. "Lynxmotion 5DoF arm
 +
Arduino interface" Arduino sketch programs for Lynxmotion Paul (1981), Chapter 3. "Exercises to move end-effector along various paths in Cartesian frame of reference
 +
"
 +
7 13 Robot arms III Robot manipulation. Frame-based task specification. Vision-based pose estimation. "Lynxmotion 5DoF arm
 +
Arduino interface" Arduino sketch programs for Lynxmotion Vernon (1991), Sections 8.1-8.4. Exercise to comput the pose of a light cube
 +
7 14 Robot arms IV Language-based programming. Programming by demonstration. "Lynxmotion 5DoF arm
 +
Arduino interface" Arduino sketch programs for Lynxmotion Vernon (1991), Sections 8.1-8.4 Exercises to implement a program to move light cube from one position/pose to another position/pose
 +
8 15 Constraint-based reasoning for robotics I Constraint satisfaction problems (CSP). Meta-Constraints and Meta-CSP reasoning. Ubuntu 14.04, ROS, Gazebo, Java 7, Meta-CSP Russell and Norvig (2010), Chapter 6. Dechter (2003), Chapter 1. Exercises on Meta-CSP exercises
 +
16 Constraint-based reasoning for robotics II Planning and navigation with multiple mobile robots Ubuntu 14.04, ROS, Gazebo, Java 7, Meta-CSP Mansouri and Pecora (2014) Exercises on Meta-CSP exercises
 +
9 17 Cognitive architectures I Role and requirements; cognitive architecture schemas; example cognitive architectures including Soar, ACT-R, Clarion, LIDA, and ISAC. None Vernon (2014) Chapter 3. Chella et al. (2013). Scheutz et al. (2013). Vernon et al. (2016). Focus group discussion on which cognitive architectures are suitable for cognitive robotics
 +
9 18 Cognitive architectures II CRAM: Cognitive Robot Abstract Machine.CRAM Plan Language (CPL). KnowRob knowledge processing and reasoning Beetz et al. (2010) Exercises on CRAM test programs
 +
10 19 Learning and development I Supervised, unsupervised, and reinforcement learning. Hebbian learning. MaxHebb library "Harmon and Harmon (1997)
 +
" Exercise on Hebbian learning
 +
10 20 Learning and development II Predictive sequence learning (PSL). Anki Cozmo mobile robot "Anki Cozmo SDK
 +
PSL library" "Sun and Giles (2001). Billing et al. (2011, 2016).
 +
" Exercises on PSL test programs
 +
11 21 Learning and development III Cognitive development in humans and robots. Anki Cozmo mobile robot Anki Cozmo SDK Vernon (2014), Chapters 6 & 9.  Lungarella et al. (2003). Asada et al. (2009). Cangelosi and Schlesinger (2015), Chapters 1 & 2. Exercises on PSL test programs
 +
11 22 Learning and development IV Value systems for developmental and cognitive robots. None Merrick (2016). Vernon et al. (2016). Focus group discussion on cognitive development in robotics\
 +
12 23 Memory and Prospection Declarative vs. procedural memory. Semantic memory. Episodic memory Anki Cozmo mobile robot "Anki Cozmo SDK
 +
CINDY library
 +
OpenCV" Vernon (2014), Chapter 7. Implement episodic memory on Cozmo
 +
12 24 Internal simulation I Forward and inverse models, internal simulation hypothesis, internal simulation with PSL "Anki Cozmo SDK
 +
PSL library" Vernon (2014), Chapter 8. Billing et al. (2016). Exercises on PSL test programs
 +
13 25 Internal simulation II HAMMER cognitive architecture "Boost
 +
Imperial College London HAMMER library" Demiris and Khadhouri (2006). Sarabia et al. (2011). Exercise on HAMMER tutorial using the ICL library
 +
12 26 Visual attention Visual attention. Spatial attention vs. selective attention. Saliency functions. Selective Tuning.  Overt attention. Inhibition of return. Habituation. Top-down attention. Anki Cozmo mobile robot CINDY library Borji and Itti (2013). Implement visual attention on Cozmo
 +
14 27 Social interaction I Joint action. Joint attention. Shared intention. Shared goals. Perspective taking. Theory of mind. Orabec Astra RGBD sensor Ubuntu 14.04, ROS, Imperial College London Perspective Taking library Vernon (2014), Chapter 9. Exercise on perspective taking using the ICL library
 +
14 28 Social interaction II Action and intention recognition. Learning from demonstration. Humanoid robotics. Orabec Astra RGBD sensor PSL library Billard et al. (2008). Argall (2009). Exercise on learning from demonstration usign the PSL library
  
 
Back to [[Cognitive Robotics]]
 
Back to [[Cognitive Robotics]]

Revision as of 15:22, 6 November 2016

Week Lecture Topic Material covered Required hardware Required software Pre-class reading Lab exercises
1 1 Cognitive robotics Introduction to AI and cognition in robotics. Industrial requirements. Artificial cognitive systems. Cognitivist, emergent, and hybrid paradigms in cognitive science. Autonomy. None None Vernon (2014), Chapters 1, 2, and 4. Installation of software tools.
1 2 Robot Vision I Optics, sensors, and image formation. Image acquisition. Image filtering. Edge detection. Orabec Astra RGBD sensor OpenCV Kragic and Vincze (2010). Szeliski (2010), Sections 1.1, 1.2, 2.3, 3.2, 4.2. Vernon (1991), Sections 2.1, 2.1 Exercises on image acquisition and image processing using OpenCV
2 3
2 4
3 5
3 6
4 7
4 8
5 9
5 10
6 11
6 12
7 13
7 14
8 15
8 16
9 17
9 18
10 19
10 20
11 21
11 22
12 23
12 24
13 25
13 26
14 27
14 28


2 3 Robot vision II Segmentation. Hough transform: line, circle, and generalized transform; extension to codeword features. Colour-based segmentation. Orabec Astra RGBD sensor OpenCV Szeliski (2010), Sections 3.1.2, 3.3.4, 4.3.2. Vernon (1991), Section 3.1, 3.2, 3.3, 4.2.1, 4.2.2, 5.3, 6.4. Exercises on Hough transforms and colour segmentation using OpenCV 2 4 Robot vision III Object recognition. Interest point operators. Gradient orientation histogram - SIFT descriptor. Colour histogram intersection. Haar features, boosting, face detection. Orabec Astra RGBD sensor "OpenCV TU Wien BLORT Library" Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1. Exercises on face detection and object recognition using OpenCV 3 5 Robot vision IV Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. Stereo vision. Epipolar geometry. Structured light & RGB-D cameras. Orabec Astra RGBD sensor OpenCV Szeliski (2010), Sections 2.1, 11.1, 11.2, 11.3. Vernon (1991), Section 8.6, 9.4.2. Exercise on camera calibration 3 6 Robot vision V Visual attention. Plane pop-out. RANSAC. Differential geometry. Surface normals and Gaussian sphere. Point clouds. 3D descriptors. Orabec Astra RGBD sensor TU Wien RGB-D Segmentation Library and V4R Library Szeliski (2010), Sections 12.4. Point Cloud Library tutorial. Exercises analysing point cloud data from RGB-D camera 4 7 Mobile robots I Differential drive locomotion. Forward and inverse kinematics. Holonomic and non-holonomic constraints. Cozmo mobile robot. Anki Cozmo mobile robot Anki Cozmo SDK "Python tutorial. Cozmo SDK API. OpenCV Python tutorial." Exercises on Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection) 4 8 Mobile robots II Map representation. Probabilistic map-based localization. Landmark-based localization. Anki Cozmo mobile robot Anki Cozmo SDK "Python tutorial. Cozmo SDK API. OpenCV Python tutorial." Exercises on Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection) 5 9 Moble robots III SLAM: simulataneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. Anki Cozmo mobile robot Anki Cozmo SDK "Python tutorial. Cozmo SDK API. OpenCV Python tutorial." Exercises on Cozmo locomotion (e.g. program Cozmo to follow a cube at a fixed distance; wnen it stops moving, pick it up and bring it home) 5 10 Moble robots IV Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. Orabec Astra RGBD sensor Ubuntu 14.04, ROS, Gazebo, Java 7 "ROS tutorials. Protege4Pizzas10Minutes tutorial. Manchester OWL tutorial." ROS Turtlebot view planning simulation 6 11 Robot arms I Homogeneous transformations. Frame-based pose specification. Denavit-Hartenberg specifications. Robot kinematics. "Lynxmotion 5DoF arm " Arduino sketch programs for Lynxmotion Paul (1981), Chapters 1 & 2. Exercises to move end-effector along various paths in joint space 6 12 Robot arms II Analytic inverse kinematics. Iterative approaches. Kinematic structure learning. Kinematics structure correspondences. "Lynxmotion 5DoF arm Arduino interface" Arduino sketch programs for Lynxmotion Paul (1981), Chapter 3. "Exercises to move end-effector along various paths in Cartesian frame of reference " 7 13 Robot arms III Robot manipulation. Frame-based task specification. Vision-based pose estimation. "Lynxmotion 5DoF arm Arduino interface" Arduino sketch programs for Lynxmotion Vernon (1991), Sections 8.1-8.4. Exercise to comput the pose of a light cube 7 14 Robot arms IV Language-based programming. Programming by demonstration. "Lynxmotion 5DoF arm Arduino interface" Arduino sketch programs for Lynxmotion Vernon (1991), Sections 8.1-8.4 Exercises to implement a program to move light cube from one position/pose to another position/pose 8 15 Constraint-based reasoning for robotics I Constraint satisfaction problems (CSP). Meta-Constraints and Meta-CSP reasoning. Ubuntu 14.04, ROS, Gazebo, Java 7, Meta-CSP Russell and Norvig (2010), Chapter 6. Dechter (2003), Chapter 1. Exercises on Meta-CSP exercises 16 Constraint-based reasoning for robotics II Planning and navigation with multiple mobile robots Ubuntu 14.04, ROS, Gazebo, Java 7, Meta-CSP Mansouri and Pecora (2014) Exercises on Meta-CSP exercises 9 17 Cognitive architectures I Role and requirements; cognitive architecture schemas; example cognitive architectures including Soar, ACT-R, Clarion, LIDA, and ISAC. None Vernon (2014) Chapter 3. Chella et al. (2013). Scheutz et al. (2013). Vernon et al. (2016). Focus group discussion on which cognitive architectures are suitable for cognitive robotics 9 18 Cognitive architectures II CRAM: Cognitive Robot Abstract Machine.CRAM Plan Language (CPL). KnowRob knowledge processing and reasoning Beetz et al. (2010) Exercises on CRAM test programs 10 19 Learning and development I Supervised, unsupervised, and reinforcement learning. Hebbian learning. MaxHebb library "Harmon and Harmon (1997) " Exercise on Hebbian learning 10 20 Learning and development II Predictive sequence learning (PSL). Anki Cozmo mobile robot "Anki Cozmo SDK PSL library" "Sun and Giles (2001). Billing et al. (2011, 2016). " Exercises on PSL test programs 11 21 Learning and development III Cognitive development in humans and robots. Anki Cozmo mobile robot Anki Cozmo SDK Vernon (2014), Chapters 6 & 9. Lungarella et al. (2003). Asada et al. (2009). Cangelosi and Schlesinger (2015), Chapters 1 & 2. Exercises on PSL test programs 11 22 Learning and development IV Value systems for developmental and cognitive robots. None Merrick (2016). Vernon et al. (2016). Focus group discussion on cognitive development in robotics\ 12 23 Memory and Prospection Declarative vs. procedural memory. Semantic memory. Episodic memory Anki Cozmo mobile robot "Anki Cozmo SDK CINDY library OpenCV" Vernon (2014), Chapter 7. Implement episodic memory on Cozmo 12 24 Internal simulation I Forward and inverse models, internal simulation hypothesis, internal simulation with PSL "Anki Cozmo SDK PSL library" Vernon (2014), Chapter 8. Billing et al. (2016). Exercises on PSL test programs 13 25 Internal simulation II HAMMER cognitive architecture "Boost Imperial College London HAMMER library" Demiris and Khadhouri (2006). Sarabia et al. (2011). Exercise on HAMMER tutorial using the ICL library 12 26 Visual attention Visual attention. Spatial attention vs. selective attention. Saliency functions. Selective Tuning. Overt attention. Inhibition of return. Habituation. Top-down attention. Anki Cozmo mobile robot CINDY library Borji and Itti (2013). Implement visual attention on Cozmo 14 27 Social interaction I Joint action. Joint attention. Shared intention. Shared goals. Perspective taking. Theory of mind. Orabec Astra RGBD sensor Ubuntu 14.04, ROS, Imperial College London Perspective Taking library Vernon (2014), Chapter 9. Exercise on perspective taking using the ICL library 14 28 Social interaction II Action and intention recognition. Learning from demonstration. Humanoid robotics. Orabec Astra RGBD sensor PSL library Billard et al. (2008). Argall (2009). Exercise on learning from demonstration usign the PSL library

Back to Cognitive Robotics

Retrieved from "http://www.vernon.eu/w/index.php?title=Cognitive_Robotics_Lectures_and_Labs&oldid=943"