Fachbereich Informatik

Doctoral Thesis

"Crossmodal Learning and Prediction of Autobiographical Episodic Experiences using a Sparse Distributed Memory"

by Sascha Jockel


This work develops a connectionist memory model for a service robot that satisfies a number of desiderata: associativity, vagueness, approximation, robustness, distribution and parallelism. A biologically inspired and mathematically sound theory of a highly distributed and sparse memory serves as the basis for this work. The so-called sparse distributed memory (SDM), developed by P. Kanerva, corresponds roughly to a random-access memory (RAM) of a conventional computer but permits the processing of considerably larger address spaces. Complex structures are represented as binary feature vectors. The model is able to produce expectations of world states and complement partial sensory patterns of an environment based on memorised experience. Caused by objects of the world, previously learnt experiences will activate pattern sequences in the memory and claim the system's attention. In this work, the sparse distributed memory concept is mainly considered a biologically inspired and content-addressable memory structure. It is used to implement an autobiographical long-term memory for a mobile service-robot to store and retrieve episodic sensor and actuator patterns.

Within the scope of this work the sparse distributed memory concept is applied to several domains of mobile service robotics, and its feasibility for the respective areas of robotics is analysed. The studied areas range from pattern matching, mobile manipulation, navigation, telemanipulation to crossmodal integration. The robot utilises properties of sparse distributed memory to detect intended actions of human teleoperators and to predict the residual motion trajectory of initiated arm or robot motions. Several examples show the model's fast and online learning capability for precoded and interactively provided motion sequences of a 6 DoF robot arm. An appropriate encoding of sensor-based information into a binary feature space is discussed and alternative coding schemes are elucidated.

A transfer of the developed system to robotic subfields such as vision-based navigation is discussed. The model's performance is compared across both of these domains, manipulation and navigation. A hierarchical extension enables the memory model to link low-level sensory percepts to higher-level semantic task descriptions. This link is used to perform a classification of demonstrated telemanipulation tasks based on the robot's experience in the past. Tests are presented where different sensory patterns are combined into an integrated percept of the world. Those crossmodal percepts are used to dissolve ambiguities that may arise from unimodal perception.

Further Information