ToM-Dyna-Q: On the integration of Reinforcement Learning and Machine Theory of Mind

DAN EZEQUIEL KRÖHLING; ERNESTO CARLOS MARTÍNEZ

Tandil

Congreso; 24 CACIC - Congreso Argentino de Ciencias de la Computación; 2018

Red de Universidades Nacionales con carreras en Informática (RedUNCI)

The capacity to understand others, or to reason about others´ ways of reasoning about others (including us), is fundamental for an agent to survive in a multi-agent uncertain environment. This reasoning ability, commonly known as Theory of Mind, is instrumental for making effective predictions over others´ future actions and learning from both real and simulated experience. In this work, a novel architecture for model-based reinforcement learning in a multi-agent setting is proposed. The proposed architecture, called ToM-Dyna-Q, integrates ToM simulation alongside with the well-known Dyna-Q architecture to account for articial cognition in a shared environment inhabited by multiple agents interacting with each other. Results obtained for the two-player competitive game of Tic-Tac-Toe demonstrate the importance for a given agent of learning, reasoning and planning based on mental simulation modeling of other agents´ goals, beliefs and intentions.