Difference between revisions of "Cognitive Robotics"
(→Recommended reading:) |
|||
(48 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | __NOTOC__ | + | <!-- __NOTOC__ --> |
− | <span style="color:#AB0000"> | + | <div style="float:right;">__TOC__</div> |
+ | <span style="color:#AB0000" ><span style="font-size:10px">|</span></span><span style="color:#000000" ><span style="font-size:10px"><span style="text-decoration:underline">CARNEGIE MELLON UNIVERSITY AFRICA</span></span></span><span style="color:#AB0000" ><span style="font-size:10px">|</span></span> | ||
− | <span style="color:#AB0000"> | + | <span style="color:#AB0000" ><span style="font-size:18px">04-801/F3 <BR></span></span> |
+ | <span style="color:#AB0000" ><span style="font-size:18px">Cognitive Robotics</span></span> | ||
+ | <!-- <span style="color:#AB0000">Course discipline: TBD</span> --> | ||
<span style="color:#AB0000">Elective</span> | <span style="color:#AB0000">Elective</span> | ||
− | <span style="color:#AB0000">Units: | + | <span style="color:#AB0000">Units: 6 (per Mini Course)</span> |
<span style="color:#AB0000">Lecture/Lab/Rep hours/week: 3 hours lecture/week</span> | <span style="color:#AB0000">Lecture/Lab/Rep hours/week: 3 hours lecture/week</span> | ||
Line 14: | Line 17: | ||
<span style="color:#AB0000">Pre-requisites: Programming skills</span> | <span style="color:#AB0000">Pre-requisites: Programming skills</span> | ||
− | ==<span style="color:#AB0000">Course description | + | ==<span style="color:#AB0000">Course description</span> == |
Cognitive robotics is an emerging discipline that draws on robotics, artificial intelligence, and cognitive science. It often exploits models based on biological cognition. | Cognitive robotics is an emerging discipline that draws on robotics, artificial intelligence, and cognitive science. It often exploits models based on biological cognition. | ||
Line 30: | Line 33: | ||
This course introduces the key elements of cognitive robotics, touching on all of these issues. In doing so, it emphasizes both theory and practice and makes extensive use of physical robots, both a mobile robots and a manipulator arm, as well as different sensor technologies. | This course introduces the key elements of cognitive robotics, touching on all of these issues. In doing so, it emphasizes both theory and practice and makes extensive use of physical robots, both a mobile robots and a manipulator arm, as well as different sensor technologies. | ||
− | ==<span style="color:#AB0000">Learning objectives | + | ==<span style="color:#AB0000">Learning objectives</span> == |
The primary goal of this course is provide students with an intensive treatment of a cross-section of the key elements of robotics, robot vision, AI, and cognitive science. Students will learn about the fundamentals of 2D and 3D visual sensing, focussing on the some essential techniques for mobile robots and robot arms. They will then learn about the kinematics and inverse kinematics of mobile robots, addressing locomotion, mapping, and path planning, as well as robot arm kinematics and inverse kinematics, task specification, and object manipulation. Based on these foundations, students will progress quickly to cover the topics that gives cognitive robotics its special focus, including cognitive architectures, learning and development, memory, attention, prospection by internal simulation, learning from demonstration, and social interaction. | The primary goal of this course is provide students with an intensive treatment of a cross-section of the key elements of robotics, robot vision, AI, and cognitive science. Students will learn about the fundamentals of 2D and 3D visual sensing, focussing on the some essential techniques for mobile robots and robot arms. They will then learn about the kinematics and inverse kinematics of mobile robots, addressing locomotion, mapping, and path planning, as well as robot arm kinematics and inverse kinematics, task specification, and object manipulation. Based on these foundations, students will progress quickly to cover the topics that gives cognitive robotics its special focus, including cognitive architectures, learning and development, memory, attention, prospection by internal simulation, learning from demonstration, and social interaction. | ||
− | ==<span style="color:#AB0000">Outcomes | + | ==<span style="color:#AB0000">Outcomes</span> == |
After completing this course, students should be able to: | After completing this course, students should be able to: | ||
* Apply their knowledge of machine vision and robot kinematics to create computer programs that control mobile robots and robot arms, enabling the robots to recognize and manipulate objects and navigate their environments. | * Apply their knowledge of machine vision and robot kinematics to create computer programs that control mobile robots and robot arms, enabling the robots to recognize and manipulate objects and navigate their environments. | ||
Line 39: | Line 42: | ||
* Create computer programs that realize limited instances of each of these models. | * Create computer programs that realize limited instances of each of these models. | ||
− | ==<span style="color:#AB0000">Content details | + | ==<span style="color:#AB0000">Content details</span> == |
− | + | Refer to the '''[[Cognitive Robotics Lecture Schedule|Lecture Schedule]]''' for information on course delivery, including lectures, assignments, and exercises. | |
+ | |||
The course will cover the following topics: | The course will cover the following topics: | ||
− | * Introduction | + | |
+ | '''Mini Course 1''' | ||
+ | * Introduction: artificial cognitive systems and robotics. | ||
* Robot vision. | * Robot vision. | ||
− | |||
* Mobile robots. | * Mobile robots. | ||
− | * Robot | + | * Robot manipulators. |
+ | |||
+ | '''Mini Course 2''' | ||
* Cognitive architectures. | * Cognitive architectures. | ||
* Learning and development. | * Learning and development. | ||
Line 57: | Line 64: | ||
''The detailed content for each of these topics follows.'' | ''The detailed content for each of these topics follows.'' | ||
− | <span style="color:#AB0000"> | + | |
− | * | + | <span style="color:#AB0000">Introduction: artificial cognitive systems and robotics</span><BR> |
− | * Industrial requirements. | + | * Motivation. |
+ | * Goals of the course. | ||
+ | * Syllabus and lecture schedule. | ||
+ | * Course operation. | ||
+ | * Industrial requirements for cognitive robots. | ||
* Artificial cognitive systems. | * Artificial cognitive systems. | ||
* Cognitivist, emergent, and hybrid paradigms in cognitive science. | * Cognitivist, emergent, and hybrid paradigms in cognitive science. | ||
* Autonomy. | * Autonomy. | ||
+ | * AI and cognition in robotics. | ||
<span style="color:#AB0000">Robot vision </span><BR> | <span style="color:#AB0000">Robot vision </span><BR> | ||
+ | * Computer vision. | ||
* Optics, sensors, and image formation. | * Optics, sensors, and image formation. | ||
* Image acquisition. | * Image acquisition. | ||
* Image filtering. | * Image filtering. | ||
− | + | * Segmentation and edge detection | |
− | * Segmentation | + | |
* Hough transform: line, circle, and generalized transform; extension to codeword features. | * Hough transform: line, circle, and generalized transform; extension to codeword features. | ||
* Colour-based segmentation. | * Colour-based segmentation. | ||
− | * Object recognition. | + | <!-- * Object recognition. --> |
− | * Interest point operators. | + | <!-- * Interest point operators. --> |
− | * Gradient orientation histogram - SIFT descriptor. | + | <!-- * Gradient orientation histogram - SIFT descriptor. --> |
− | * Colour histogram intersection. | + | <!-- * Colour histogram intersection. --> |
− | * Haar features, boosting, face detection. | + | <!-- * Haar features, boosting, face detection.--> |
* Homogeneous coordinates and transformations. | * Homogeneous coordinates and transformations. | ||
* Perspective transformation. | * Perspective transformation. | ||
* Camera model and inverse perspective transformation. | * Camera model and inverse perspective transformation. | ||
− | * Stereo vision. | + | * Stereo vision. |
− | * Epipolar geometry | + | * Epipolar geometry. |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
<span style="color:#AB0000">Mobile robots</span><BR> | <span style="color:#AB0000">Mobile robots</span><BR> | ||
Line 105: | Line 100: | ||
* Holonomic and non-holonomic constraints. | * Holonomic and non-holonomic constraints. | ||
* Cozmo mobile robot. | * Cozmo mobile robot. | ||
+ | * Relative and absolute position estimation. | ||
+ | * Odometry. | ||
* Map representation. | * Map representation. | ||
− | * Probabilistic map-based localization. | + | <!-- * Probabilistic map-based localization. --> |
− | * Landmark-based localization. | + | <!-- * Landmark-based localization.--> |
− | * SLAM: simulataneous localization and mapping. | + | <!-- * SLAM: simulataneous localization and mapping. --> |
− | * Extended Kalman Filter (EKF) SLAM. | + | <!-- * Extended Kalman Filter (EKF) SLAM. --> |
− | * Visual SLAM. | + | <!-- * Visual SLAM. --> |
− | * Particle filter SLAM. | + | <!-- * Particle filter SLAM.--> |
* Graph search path planning. | * Graph search path planning. | ||
− | * Potential field path planning. | + | <!-- * Potential field path planning. --> |
− | * Navigation. Obstacle avoidance. | + | <!-- * Navigation. Obstacle avoidance.--> |
− | * Object search. | + | <!-- * Object search.--> |
− | <span style="color:#AB0000">Robot | + | <span style="color:#AB0000">Robot manipulators</span><BR> |
* Homogeneous transformations. | * Homogeneous transformations. | ||
* Frame-based pose specification. | * Frame-based pose specification. | ||
Line 123: | Line 120: | ||
* Robot kinematics. | * Robot kinematics. | ||
* Analytic inverse kinematics. | * Analytic inverse kinematics. | ||
− | * Iterative approaches. | + | <!-- * Iterative approaches. --> |
− | * Kinematic structure learning. | + | <!-- * Kinematic structure learning. --> |
− | * Kinematics structure correspondences. | + | <!-- * Kinematics structure correspondences.--> |
* Robot manipulation. | * Robot manipulation. | ||
* Frame-based task specification. | * Frame-based task specification. | ||
* Vision-based pose estimation. | * Vision-based pose estimation. | ||
− | * Language-based programming. | + | <!-- * Programming by demonstration.--> |
+ | <!-- * Language-based programming.--> | ||
<span style="color:#AB0000">Cognitive architectures</span><BR> | <span style="color:#AB0000">Cognitive architectures</span><BR> | ||
Line 138: | Line 136: | ||
* CRAM: Cognitive Robot Abstract Machine. | * CRAM: Cognitive Robot Abstract Machine. | ||
* CRAM Plan Language (CPL). | * CRAM Plan Language (CPL). | ||
− | * Knowledge representation, processing, and reasoning. | + | * Knowledge representation, processing, and reasoning: KnowRob and OpenEASE. |
<span style="color:#AB0000">Learning and development </span><BR> | <span style="color:#AB0000">Learning and development </span><BR> | ||
Line 167: | Line 165: | ||
* Perspective taking & Theory of Mind. | * Perspective taking & Theory of Mind. | ||
* Action and intention recognition. | * Action and intention recognition. | ||
+ | * Embodied cognition. | ||
* Humanoid robotics. | * Humanoid robotics. | ||
− | ==<span style="color:#AB0000"> | + | ==<span style="color:#AB0000">Lecture Schedule</span> == |
+ | Refer to the [[Cognitive Robotics Lecture Schedule|Lecture Schedule]] for information on course delivery, including lectures, assignments, and exercises. | ||
+ | |||
+ | ==<span style="color:#AB0000">Faculty</span> == | ||
[http://www.vernon.eu David Vernon] | [http://www.vernon.eu David Vernon] | ||
− | ==<span style="color:#AB0000">Delivery | + | ==<span style="color:#AB0000">Delivery</span> == |
Face-to-face. | Face-to-face. | ||
− | ==<span style="color:#AB0000"> | + | ==<span style="color:#AB0000">Student assessment</span> == |
Lab assignments 50% | Lab assignments 50% | ||
− | Mid-term exam | + | Mid-term exam 10% |
− | Final exam | + | Final exam 40% |
+ | |||
+ | ==<span style="color:#AB0000">Robots and sensors</span> == | ||
+ | [https://www.desertcart.ae/products/1973810-kinect-for-windows Kinect for Windows] | ||
− | |||
[https://www.roscomponents.com/en/cameras/76-orbbec.html Orabec Astra RGBD sensor] | [https://www.roscomponents.com/en/cameras/76-orbbec.html Orabec Astra RGBD sensor] | ||
[https://anki.com/en-us/cozmo Anki Cozmo mobile robot ] | [https://anki.com/en-us/cozmo Anki Cozmo mobile robot ] | ||
− | [http://www.lynxmotion.com/p-868-al5d-robotic-arm-combo-kit-botboarduino.aspx | + | [http://www.lynxmotion.com/p-868-al5d-robotic-arm-combo-kit-botboarduino.aspx Lynxmotion AL5D Robotic Arm with BotBoarduino interface] |
− | ==<span style="color:#AB0000">Software | + | ==<span style="color:#AB0000">Software tools</span> == |
− | + | Please follow the instructions provided in the [[04-801 Software Development Environment|Software Development Environment]] installation guide. | |
− | + | ||
− | + | ||
− | |||
− | + | For reference only, the following is a list of some of the software that may be used in the course. | |
− | [http:// | + | [http://opencv.org/ OpenCV.] |
− | [http:// | + | [http://www.acin.tuwien.ac.at/forschung/v4r/software-tools/ Technische Universität Wien Software Tools.] |
− | [http://www.acin.tuwien.ac.at/forschung/v4r/software-tools/ | + | [http://www.acin.tuwien.ac.at/forschung/v4r/software-tools/blort/ BLORT - The Blocks World Robotic Toolbox from Technische Universität Wien.] |
+ | <!-- [http://www.acin.tuwien.ac.at/forschung/v4r/software-tools/rgb-d-segmentation/ RGB-D segmentation library from Technische Universität Wien. ] --> | ||
− | [http://www.acin.tuwien.ac.at/forschung/v4r/software-tools/ | + | [http://www.acin.tuwien.ac.at/forschung/v4r/software-tools/v4r-library/ V4R Library -The Vision4Robotics library (RGB-D point cloud) from Technische Universität Wien.] |
+ | |||
+ | <!-- Homebrew: /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" --> | ||
+ | <!-- Python: brew install python3 --> | ||
+ | Python 3.5.2 for [https://www.python.org/downloads/ Mac OS X] or [https://www.python.org/downloads/windows/ Windows]. You may need to update Tcl/TK to [http://www.activestate.com/activetcl/downloads version 8.5.18.0] (see [https://www.python.org/download/mac/tcltk/#activetcl-8-5-18-0 Python documentation]). | ||
− | [http:// | + | [http://cozmosdk.anki.com/docs/index.html Anki Cozmo SDK] |
− | + | ||
− | + | ||
[https://github.com/Lynxmotion/Arms/tree/Botboarduino Arduino sketch programs for Lynxmotion ] | [https://github.com/Lynxmotion/Arms/tree/Botboarduino Arduino sketch programs for Lynxmotion ] | ||
+ | <!-- [http://www.simplehelp.net/2015/06/09/how-to-install-ubuntu-on-your-mac/ Ubuntu] --> | ||
+ | <!-- [http://wiki.ros.org/indigo/Installation ROS Indigo] --> | ||
+ | <!-- [http://wiki.ros.org/turtlebot_simulator TurtleBot simulator] --> | ||
+ | <!-- [http://wiki.ros.org/Robots/TurtleBot Turtlebot] --> | ||
+ | <!-- GIT: sudo apt-get install git --> | ||
− | [http://www. | + | [http://www.imperial.ac.uk/personal-robotics/software/ Imperial College London (ICL) Personal Robotics Lab Software Tools] |
− | [ | + | [https://bitbucket.org/miguelsdc/hammer/ HAMMER cognitive architecture based on the simulation theory of mind from ICL] |
− | [http:// | + | [http://dream2020.eu/downloads/Kinect/KinectSDK-v1.8-Setup.exe Kinect SDK v1.8] |
− | [ | + | [https://github.com/robotology/wysiwyd/tree/master/main/src/modules/perspectiveTaking/src Markerless Perspective Taking from ICL] |
− | [http:// | + | [http://www.open-ease.org openEASE web-based knowledge service ] |
− | [http:// | + | [http://www.vernon.eu/downloads/arduino-1.8.1-windows.exe Arduino 1.8.1 for Windows] |
− | + | [http://www.vernon.eu/downloads/inverse_kinematics_demo.ino Arduino inverse kinematics demo sketch] | |
− | [http://www. | + | [http://www.vernon.eu/downloads/CDM21224_Setup.exe CDM21224 COM port on USB.exe] |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | ==<span style="color:#AB0000">Recommended reading | + | ==<span style="color:#AB0000">Recommended reading</span> == |
[https://www.researchgate.net/publication/222219268_A_survey_of_robot_learning_from_demonstration Argall, B. D., Chernova, S., Veloso,M., and Browning, B. (2009). "A survey of robot learning from demonstration". Robotics and Autonomous Systems, 57:469–483, 2009. ] | [https://www.researchgate.net/publication/222219268_A_survey_of_robot_learning_from_demonstration Argall, B. D., Chernova, S., Veloso,M., and Browning, B. (2009). "A survey of robot learning from demonstration". Robotics and Autonomous Systems, 57:469–483, 2009. ] | ||
Line 248: | Line 251: | ||
[http://iccm-conference.org/2013-proceedings/papers/0048/paper0048.pdf Chella, A., Kurup, U., Laird, J., Trafton, G., Vinokurov, J., Chandrasekaran, B. (2013). "The Challenge of Robotics for Cognitive Architectures", Proc. 12th International Conference on Cognitive Modelling.] | [http://iccm-conference.org/2013-proceedings/papers/0048/paper0048.pdf Chella, A., Kurup, U., Laird, J., Trafton, G., Vinokurov, J., Chandrasekaran, B. (2013). "The Challenge of Robotics for Cognitive Architectures", Proc. 12th International Conference on Cognitive Modelling.] | ||
− | |||
− | |||
Demiris, Y. and Khadhouri, B. (2006). "Hierarchical attentive multiple models for execution and recognition (HAMMER). Robotics and Autonomous Systems, 54:361–369. | Demiris, Y. and Khadhouri, B. (2006). "Hierarchical attentive multiple models for execution and recognition (HAMMER). Robotics and Autonomous Systems, 54:361–369. | ||
+ | |||
+ | Fischer, T. and Demiris, Y. (2016), "Markerless Perspective Taking for Humanoid Robots in Unconstrained Environments", 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3309-3316. | ||
[http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA323194 Harmon, M. and Harmon, S. (1997). Reinforcement Learning: A Tutorial] | [http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA323194 Harmon, M. and Harmon, S. (1997). Reinforcement Learning: A Tutorial] | ||
Line 258: | Line 261: | ||
[https://www.researchgate.net/publication/220233671_Developmental_robotics_A_survey Lungarella, M., Metta, G., Pfeifer, R. and Sandini, G. (2003). "Developmental Robotics: A Survey", Connection Science, 17, pp. 151-190. ] | [https://www.researchgate.net/publication/220233671_Developmental_robotics_A_survey Lungarella, M., Metta, G., Pfeifer, R. and Sandini, G. (2003). "Developmental Robotics: A Survey", Connection Science, 17, pp. 151-190. ] | ||
− | |||
− | |||
Merrick, K (2016). "Value systems for developmental cognitive robotics: a survey", Cognitive Systems Research, in press.] | Merrick, K (2016). "Value systems for developmental cognitive robotics: a survey", Cognitive Systems Research, in press.] | ||
Line 281: | Line 282: | ||
[http://vernon.eu/publications/16_Vernon_vonHofsten_Fadiga_BICA.pdf Vernon, D., von Hofsten, C. and Fadiga, L. (2016). "Desiderata for Developmental Cognitive Architectures", Biologically Inspired Cognitive Architectures, Vol. 18, pp. 116-127, 2016. ] | [http://vernon.eu/publications/16_Vernon_vonHofsten_Fadiga_BICA.pdf Vernon, D., von Hofsten, C. and Fadiga, L. (2016). "Desiderata for Developmental Cognitive Architectures", Biologically Inspired Cognitive Architectures, Vol. 18, pp. 116-127, 2016. ] | ||
− | ==<span style="color:#AB0000">Documentation | + | ==<span style="color:#AB0000">Documentation</span> == |
[http://www.diveintopython3.net/ Dive Into Python 3] ([http://www.diveintopython3.net/examples/humansize.py humansize.py] example) | [http://www.diveintopython3.net/ Dive Into Python 3] ([http://www.diveintopython3.net/examples/humansize.py humansize.py] example) | ||
[http://cozmosdk.anki.com/docs/index.html Cozmo SDK API ] | [http://cozmosdk.anki.com/docs/index.html Cozmo SDK API ] | ||
− | |||
− | |||
[http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_tutorials.html OpenCV Python Tutorial ] | [http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_tutorials.html OpenCV Python Tutorial ] | ||
[http://pointclouds.org/documentation/ Point Cloud Tutorial ] | [http://pointclouds.org/documentation/ Point Cloud Tutorial ] | ||
− | |||
− | |||
[https://docs.python.org/3.5/tutorial/index.html Python Tutorial ] | [https://docs.python.org/3.5/tutorial/index.html Python Tutorial ] | ||
Line 302: | Line 299: | ||
[https://www.youtube.com/watch?v=iu_Y7mxDjgA&feature=youtu.be OpenEase ] | [https://www.youtube.com/watch?v=iu_Y7mxDjgA&feature=youtu.be OpenEase ] | ||
− | ==<span style="color:#AB0000">Acknowledgments | + | ==<span style="color:#AB0000">Acknowledgments</span> == |
The syllabus for this course drew inspiration from several sources. These include the following. | The syllabus for this course drew inspiration from several sources. These include the following. | ||
* Course 15-494/694 Cognitive Robotics given by Dave Touretzky at Carnegie Mellon University. | * Course 15-494/694 Cognitive Robotics given by Dave Touretzky at Carnegie Mellon University. | ||
− | * Course VO 4.0 376.054 Machine Vision and Cognitive Robotics given by Markus Vincze, Michael Zillich, and Daniel Wolf at | + | * Course VO 4.0 376.054 Machine Vision and Cognitive Robotics given by Markus Vincze, Michael Zillich, and Daniel Wolf at Technische Universität Wien. |
* Course IT921F Artificial Cognitive Systems given by David Vernon at the University of Skövde. | * Course IT921F Artificial Cognitive Systems given by David Vernon at the University of Skövde. | ||
− | |||
− |
Revision as of 11:18, 18 April 2017
|CARNEGIE MELLON UNIVERSITY AFRICA|
04-801/F3
Cognitive Robotics
Elective
Units: 6 (per Mini Course)
Lecture/Lab/Rep hours/week: 3 hours lecture/week
Semester: Spring
Pre-requisites: Programming skills
Course description
Cognitive robotics is an emerging discipline that draws on robotics, artificial intelligence, and cognitive science. It often exploits models based on biological cognition.
There are at least two reasons why a cognitive ability is useful in robotics:
- It allows the robot to work autonomously in challenging environments, adapting to changes and unforeseen situations, and anticipating outcomes when selecting the actions it will perform.
- It facilitates interaction with people. Humans have a strong preference for interaction with other cognitive agents so being able to exhibit a capacity for cognition encourages human robot interaction. Conversely, a cognitive ability provides the robot with the ability to infer the goals and intentions of the person it is interacting with and thereby allows it to do so in safe and helpful manner.
Cognitive robots achieve their goals by perceiving their environment, paying attention to the events that matter, planning what to do, anticipating the outcome of their own actions and the actions of other agents (people and other robots), and learning from the resultant interaction. They deal with the inherent uncertainty of natural environments by continually learning, reasoning, and sharing their knowledge.
A key feature of cognitive robotics is its focus on predictive capabilities to augment and compensate for perceptual capacities. Also, by being able to view the world from someone else’s perspective, a cognitive robot can anticipate that person’s intended actions and needs.
In cognitive robotics, the robot body can be more than just an instrument for physical manipulation or locomotion: it can also be a component of the cognitive process. In the particular case of humanoid robotics, the robot’s physical morphology, kinematics, and dynamics, as well as the environment in which it is operating, can help it to achieve its key characteristic of adaptive anticipatory interaction by mirroring the actions of the person with whom it is interacting.
This course introduces the key elements of cognitive robotics, touching on all of these issues. In doing so, it emphasizes both theory and practice and makes extensive use of physical robots, both a mobile robots and a manipulator arm, as well as different sensor technologies.
Learning objectives
The primary goal of this course is provide students with an intensive treatment of a cross-section of the key elements of robotics, robot vision, AI, and cognitive science. Students will learn about the fundamentals of 2D and 3D visual sensing, focussing on the some essential techniques for mobile robots and robot arms. They will then learn about the kinematics and inverse kinematics of mobile robots, addressing locomotion, mapping, and path planning, as well as robot arm kinematics and inverse kinematics, task specification, and object manipulation. Based on these foundations, students will progress quickly to cover the topics that gives cognitive robotics its special focus, including cognitive architectures, learning and development, memory, attention, prospection by internal simulation, learning from demonstration, and social interaction.
Outcomes
After completing this course, students should be able to:
- Apply their knowledge of machine vision and robot kinematics to create computer programs that control mobile robots and robot arms, enabling the robots to recognize and manipulate objects and navigate their environments.
- Explain how a robot can be designed to exhibit cognitive goal-directed behaviour through the integration of computer models of visual attention, reasoning, learning, prospection, and social interaction.
- Create computer programs that realize limited instances of each of these models.
Content details
Refer to the Lecture Schedule for information on course delivery, including lectures, assignments, and exercises.
The course will cover the following topics:
Mini Course 1
- Introduction: artificial cognitive systems and robotics.
- Robot vision.
- Mobile robots.
- Robot manipulators.
Mini Course 2
- Cognitive architectures.
- Learning and development.
- Memory and Prospection.
- Internal simulation.
- Learning from demonstration.
- Social interaction.
The detailed content for each of these topics follows.
Introduction: artificial cognitive systems and robotics
- Motivation.
- Goals of the course.
- Syllabus and lecture schedule.
- Course operation.
- Industrial requirements for cognitive robots.
- Artificial cognitive systems.
- Cognitivist, emergent, and hybrid paradigms in cognitive science.
- Autonomy.
- AI and cognition in robotics.
Robot vision
- Computer vision.
- Optics, sensors, and image formation.
- Image acquisition.
- Image filtering.
- Segmentation and edge detection
- Hough transform: line, circle, and generalized transform; extension to codeword features.
- Colour-based segmentation.
- Homogeneous coordinates and transformations.
- Perspective transformation.
- Camera model and inverse perspective transformation.
- Stereo vision.
- Epipolar geometry.
Mobile robots
- Differential drive locomotion.
- Forward and inverse kinematics.
- Holonomic and non-holonomic constraints.
- Cozmo mobile robot.
- Relative and absolute position estimation.
- Odometry.
- Map representation.
- Graph search path planning.
Robot manipulators
- Homogeneous transformations.
- Frame-based pose specification.
- Denavit-Hartenberg specifications.
- Robot kinematics.
- Analytic inverse kinematics.
- Robot manipulation.
- Frame-based task specification.
- Vision-based pose estimation.
Cognitive architectures
- Role and requirements.
- Cognitive architecture schemas.
- Example cognitive architectures including Soar, ACT-R, Clarion, LIDA, and ISAC.
- The standard model.
- CRAM: Cognitive Robot Abstract Machine.
- CRAM Plan Language (CPL).
- Knowledge representation, processing, and reasoning: KnowRob and OpenEASE.
Learning and development
- Supervised, unsupervised, and reinforcement learning.
- Hebbian learning.
- Predictive sequence learning (PSL).
- Learning from demonstration.
- Cognitive development in humans and robots.
- Value systems for developmental and cognitive robots.
Memory and Prospection
- Declarative vs. procedural memory.
- Semantic memory.
- Episodic memory.
Internal simulation
- Episodic future thinking.
- Forward and inverse models.
- Internal simulation hypothesis.
- Internal simulation with PSL.
- HAMMER cognitive architecture.
Social interaction
- Joint action.
- Joint attention.
- Shared intention.
- Shared goals.
- Perspective taking & Theory of Mind.
- Action and intention recognition.
- Embodied cognition.
- Humanoid robotics.
Lecture Schedule
Refer to the Lecture Schedule for information on course delivery, including lectures, assignments, and exercises.
Faculty
Delivery
Face-to-face.
Student assessment
Lab assignments 50% Mid-term exam 10% Final exam 40%
Robots and sensors
Lynxmotion AL5D Robotic Arm with BotBoarduino interface
Software tools
Please follow the instructions provided in the Software Development Environment installation guide.
For reference only, the following is a list of some of the software that may be used in the course.
Technische Universität Wien Software Tools.
BLORT - The Blocks World Robotic Toolbox from Technische Universität Wien.
V4R Library -The Vision4Robotics library (RGB-D point cloud) from Technische Universität Wien.
Python 3.5.2 for Mac OS X or Windows. You may need to update Tcl/TK to version 8.5.18.0 (see Python documentation).
Arduino sketch programs for Lynxmotion
Imperial College London (ICL) Personal Robotics Lab Software Tools
HAMMER cognitive architecture based on the simulation theory of mind from ICL
Markerless Perspective Taking from ICL
openEASE web-based knowledge service
Arduino inverse kinematics demo sketch
Recommended reading
Beetz, M., Mösenlechner, L., and Tenorth, M. (2010). "CRAM - A Cognitive Robot Abstract Machine for Everyday Manipulation in Human Environments", IEEE/RSJ International Conference on Intelligent Robots and Systems, 1012-1017.
Cangelosi, A. and Schlesinger, M. (2015). Developmental Robotics: From Babies to Robots. Cambridge, MA: MIT Press.
Demiris, Y. and Khadhouri, B. (2006). "Hierarchical attentive multiple models for execution and recognition (HAMMER). Robotics and Autonomous Systems, 54:361–369.
Fischer, T. and Demiris, Y. (2016), "Markerless Perspective Taking for Humanoid Robots in Unconstrained Environments", 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3309-3316.
Harmon, M. and Harmon, S. (1997). Reinforcement Learning: A Tutorial
Kragic, D. and Vincze, M. (2010). "Vision for Robotics", Foundation and Trends in Robotics, Vol 1, No 1, pp 1–78.
Merrick, K (2016). "Value systems for developmental cognitive robotics: a survey", Cognitive Systems Research, in press.]
Paul, R. (1981). Robot Manipulators: Mathematics, Programming, and Control. MIT Press.
Russell, S. and Norvig. P. (2014). Artificial Intelligence: A Modern Approach, Pearson Education.
Szeliski, R. (2010). Computer Vision: Algorithms and Applications, Springer.
Vernon, D. (2014). Artificial Cognitive Systems - A Primer, MIT Press, 2014.
Documentation
Dive Into Python 3 (humansize.py example)
Acknowledgments
The syllabus for this course drew inspiration from several sources. These include the following.
- Course 15-494/694 Cognitive Robotics given by Dave Touretzky at Carnegie Mellon University.
- Course VO 4.0 376.054 Machine Vision and Cognitive Robotics given by Markus Vincze, Michael Zillich, and Daniel Wolf at Technische Universität Wien.
- Course IT921F Artificial Cognitive Systems given by David Vernon at the University of Skövde.