cognition.reversed

Research by Erik Billing and colleagues

MAN — MACHINE

— It's all about the relationships we create with technology, how we view the artefacts we design, and what we learn about ourselves in the process.

PSL

— We understand the world and plan our day not as a process from perception to action, but the reversed, as anticipations of sensorconsequences of actions.

Predictive Sequence Learning is a method for robot learning from demonstration, able to anticipate future sensory-motor events in response to the recent interaction history.

A reference implementation in JavaScript, intended for educational purposes, is available at github.com/billingo/psl.js. This implementation is underlying the online demo available here at cognitionreversed.com.

A full-scale implementation in Java, underlying most scientific results referenced here, is available at bitbucket.org/interactionlab/psl.


Norway

Below is a summary of Erik Billing's PhD Thesis, titled Cognition Rehearsed - Recognition and Reproduction of Demonstrated Behavior, defended January 26, 2012.

Thesis Abstract

The work presented in this dissertation investigates techniques for Learning from Demonstration (LFD). LFD is a well established approach to robot learning, where a teacher demonstrates a behavior to a robot pupil. The dissertation focuses on LFD where a human teacher demonstrates a behavior by controlling the robot via teleoperation. After demonstration, the robot should be able to reproduce the demonstrated behavior under varying conditions. In particular, the dissertation investigates techniques where previous behavioral knowledge is used as bias for generalization of demonstrations.

The primary contribution of this work is the development and evaluation of a semi-reactive approach to LFD called Predictive Sequence Learning (PSL). PSL has many interesting properties applied as a learning algorithm for robots. Few assumptions are introduced and little task-specific configuration is needed. PSL can be seen as a variable-order Markov model that progressively builds up the ability to predict or simulate future sensory-motor events, given a history of past events. The knowledge base generated during learning can be used to control the robot, such that the demonstrated behavior is reproduced. The same knowledge base can also be used to recognize an on-going behavior by comparing predicted sensor states with actual observations. Behavior recognition is an important part of LFD, both as a way to communicate with the human user and as a technique that allows the robot to use previous knowledge as parts of new, more complex, controllers.

In addition to the work on PSL, this dissertation provides a broad discussion on representation, recognition, and learning of robot behavior. LFD-related concepts such as demonstration, repetition, goal, and behavior are defined and analyzed, with focus on how bias is introduced by the use of behavior primitives. This analysis results in a formalism where LFD is described as transitions between information spaces. Assuming that the behavior recognition problem is partly solved, ways to deal with remaining ambiguities in the interpretation of a demonstration are proposed.

The evaluation of PSL shows that the algorithm can efficiently learn and reproduce simple behaviors. The algorithm is able to generalize to previously unseen situations while maintaining the reactive properties of the system. As the complexity of the demonstrated behavior increases, knowledge of one part of the behavior sometimes interferes with knowledge of another parts. As a result, different situations with similar sensory-motor interactions are sometimes confused and the robot fails to reproduce the behavior.

One way to handle these issues is to introduce a context layer that can support PSL by providing bias for predictions. Parts of the knowledge base that appear to fit the present context are highlighted, while other parts are inhibited. Which context should be active is continually re-evaluated using behavior recognition. This technique takes inspiration from several neurocomputational models that describe parts of the human brain as an hierarchical prediction system. With behavior recognition active, continually selecting the most suitable context for the present situation, the problem of knowledge interference is significantly reduced and the robot can successfully reproduce also more complex behaviors.

Erik A. Billing

Erik Billing is an Associate Senior Lecturer in Informatics at the Interaction Lab, University of Skövde, Sweden. He holds a master in cognitive science and a PhD in computing science, both from Umeå University.

Erik devotes most of his time to research in the borderlands between artificial intelligence, robotics, and cognitive science. Among other things, Erik has explored how robots can learn from human demonstrations through anticipation of action and developed an algorithm for robot learning, called Predictive Sequence Learning. Examples and reference implementations is found here on cognitionreversed.com.


Publications

  • B. Zhou, C. Velez Altamirano, H. Cruz Zurian, S. Atefi, E. Billing, F. Seoane Martinez, P. Lukowicz (2017). Textile Pressure Mapping Sensor for Emotional Touch Detection in Human-Robot Interaction. Sensors.
  • E. Billing (2017). A New Look at Habits using Simulation Theory. Proceedings of the Digitalisation for a Sustainable Society : Embodied, Embedded, Networked, Empowered through Information, Computation & Cognition.
  • S. Jiong, E. Billing, F. Seoane, B. Zhou, D. Högberg, P. Hemeren (2017). Categories of touch : Classifying human touch using a soft tactile sensor. The robotic sense of touch: From sensing to understanding, workshop at the IEEE International Conference on Robotics and Automation (ICRA), Singapore, May 29, 2017.
  • P. Esteban, P. Baxter, T. Belpaeme, E. Billing, H. Cai, H. Cao, M. Coeckelbergh, C. Costescu, D. David, A. De Beir, Y. Fang, Z. Ju, J. Kennedy, H. Liu, A. Mazel, A. Pandey, K. Richardson, E. Senft, S. Thill, G. Van de Perre, B. Vanderborght, D. Vernon, H. Yu, T. Ziemke (2017). How to Build a Supervised Autonomous System for Robot-Enhanced Therapy for Children with Autism Spectrum Disorder. Paladyn - Journal of Behavioral Robotics, p. 18-38.
  • A. Arweström Jansson, A. Axelsson, R. Andreasson, E. Billing [Eds.] (2017). Proceedings of the 13th Swecog conference. Swecog 2017, Uppsala, October 26-27.
  • A. Montebelli, E. Billing, J. Lindblom, G. Messina Dahlberg (2017). Reframing HRI Education : A Dialogic Reformulation of HRI Education to Promote Diverse Thinking and Scientific Progress. Journal of Human-Robot Interaction, p. 3-26.
  • S. Jiong, S. Redyuk, E. Billing, D. Högberg, P. Hemeren (2017). Tactile Interaction and Social Touch : Classifying Human Touch using a Soft Tactile Sensor. HAI '17 : Proceedings of the 5th International Conference on Human Agent Interaction, p. 523-526.
  • R. Lowe, A. Almér, E. Billing, Y. Sandamirskaya, C. Balkenius (2017). Affective–associative two-process theory : a neurocomputational account of partial reinforcement extinction effects. Biological Cybernetics.
  • B. Zhou, H. Cruz, S. Atefi, E. Billing, F. Seoane, P. Lukowicz (2017). TouchMe: Full-textile Touch Sensitive Skin for Encouraging Human-Robot Interaction. The robotic sense of touch: from sensing to understanding, workshop at the IEEE International Conference on Robotics and Automation (ICRA), May 29 - June 3, Singapore.
  • B. Alenljung, R. Andreasson, E. Billing, J. Lindblom, R. Lowe (2017). User Experience of Conveying Emotions by Touch. Proceedings of the 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), p. 1240-1247.
  • R. Lowe, E. Billing (2017). Affective-Associative Two-Process theory : A neural network investigation of adaptive behaviour in differential outcomes training. Adaptive Behavior, p. 5-23.
  • F. Syrén, C. Li, E. Billing, A. Lund, V. Nierstrasz (2016). Characterization of textile resistive strain sensors. 16th World Textile Conference AUTEX 2016, Ljubljana, Slovenia, 8-10 June 2016.
  • E. Billing, H. Svensson, R. Lowe, T. Ziemke (2016). Finding Your Way from the Bed to the Kitchen : Re-enacting and Re-combining Sensorimotor Episodes Learned from Human Demonstration. Frontiers in Robotics and AI.
  • R. Lowe, E. Barakova, E. Billing, J. Broekens (2016). Grounding emotions in robots : An introduction to the special issue. Adaptive Behavior, p. 263-266.
  • A. Almér, R. Lowe, E. Billing [Eds.] (2016). Proceedings of the 2016 Swecog conference. Swecog 2016, Göteborg, October 6-7, 2016.
  • D. Vernon, E. Billing, P. Hemeren, S. Thill, T. Ziemke (2015). An Architecture-oriented Approach to System Integration in Collaborative Robotics Research Projects : An Experience Report. Journal of Software Engineering for Robotics, p. 15-32.
  • E. Billing, J. Lindblom, T. Ziemke [Eds.] (2015). Proceedings of the 2015 SWECOG conference. The 2015 SweCog Conference, June 15-16, Skövde, 2015.
  • E. Billing, R. Lowe, Y. Sandamirskaya (2015). Simultaneous Planning and Action : Neural-dynamic Sequencing of Elementary Behaviors in Robot Navigation. Adaptive Behavior, p. 243-264.
  • E. Billing, T. Hellström, L. Janlert (2015). Simultaneous recognition and reproduction of demonstrated behavior. Biologically Inspired Cognitive Architectures, p. 43-53.
  • R. Lowe, Y. Sandarmirskaya, E. Billing (2014). A Neural Dynamic Model of Associative Two-Process Theory : The Differential Outcomes Effect and Infant Development. IEEE ICDL-EPIROB 2014 : The Fourth Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, p. 440-447.
  • E. Billing, C. Balkenius (2014). Modeling the Interplay between Conditioning and Attention in a Humanoid Robot : Habituation and Attentional Blocking. Proceeding of The 4th International Conference on Development and Learning and on Epigenetic Robotics (IEEE ICDL-EPIROB 2014), p. 41-47.
  • E. Billing, M. Servin (2013). Composer : A prototype multilingual model composition tool. MODPROD2013 : 7th MODPROD Workshop on Model-Based Product Development.
  • E. Billing (2012). Cognition Rehearsed : Recognition and Reproduction of Demonstrated Behavior. PhD dissertation, Department of Computing Science, Umeå University Report / UMINF ; 11.16.
  • E. Billing, T. Hellström, L. Janlert (2012). Robot learning from demonstration using predictive sequence learning. Robotic systems : applications, control and programming, p. 235-250.
  • E. Billing, T. Hellström, L. Janlert (2011). Predictive learning from demonstration. Agents and Artificial Intelligence : Second International Conference, ICAART 2010, Valencia, Spain, January 22-24, 2010. Revised Selected Papers, p. 186-200.
  • E. Billing, T. Hellström, L. Janlert (2011). Simultaneous control and recognition of demonstrated behavior. Umeå University, Department of Computing Science, Report / UMINF ; 15.
  • E. Billing, T. Hellström (2010). A formalism for learning from demonstration. Paladyn - Journal of Behavioral Robotics, p. 1-13.
  • E. Billing, T. Hellström, L. Janlert (2010). Behavior recognition for learning from demonstration. 2010 IEEE International Conference on Robotics and Automation, p. 866-872.
  • E. Billing (2010). Cognitive Perspectives on Robot Behavior. Proceedings of the 2nd International Conference on Agents and Artificial Intelligence : Volume 2, p. 373-382.
  • E. Billing, T. Hellström, L. Janlert (2010). Model-free learning from demonstration. Proceedings of the 2nd International Conference on Agents and Artificial Intelligence : Volume 2, p. 62-71.
  • E. Billing, T. Hellström (2008). Behavior recognition for segmentation of demonstrated tasks. IEEE SMC International Conference on Distributed Human-Machine Systems (DHMS), p. 228-234.
  • E. Billing, T. Hellström (2008). Formalising learning from demonstration. Department of Computing Science, Umeå University, Report / UMINF ; 08:10.
  • E. Billing (2007). Representing behavior : Distributed theories in a context of robotics. Umeå University, Department of Computing Science, Report / UMINF ; 07:25.