Decoding sensory-motor neural signals for vocal communication

EZEQUIEL ARNEODO; VIKASH GILJA; GERT CAUWENBERGHS; TIMOTHY Q GENTNER

San Diego

Simposio; 2017 KIBM Symposium on Innovative Research; 2017

Kavli Institute for the Brain and Mind

How does the brain transform intention into action? Brain Machine Interfaces (BMIs), which couple neural responses to external artificial effectors, seek to answer this question by directly inferring intention from neural activity. Even for simple actions, this decoding problem is computationally intensive; as the complexity of the intended behavior increases computations become intractable.  As a result, effective neural prostheses for speech and other natural behaviors remain science fiction. Here we present an ambitious plan with goal of turning this fiction into reality combining recent advances in machine learning, biophysical modelling of vocal production mechanisms, neuromorphic engineering and systems neuroscience. Using birdsong, the preeminent neurobiological model for natural (i.e. complex), learned, vocal motor control, we outline a tractable plan to synthesize song directly from activity in a bird?s brain. The plan develops two solutions.  First, we reduce the computational complexity of the motor-mapping problem. Instead of trying to solve the very difficult task of mapping neural activity to complex muscle movements, we will replace the peripheral output organ with a biophysical model whose low-dimensional dynamics captures the whole complexity of the motor output. Second, we implement powerful machine learning algorithms on a low-power, neuromorphic hardware platform for real-time computation. This novel marriage of biophysics, neuromorphic engineering, and systems neuroscience will produce a powerful tool for testing hypotheses of how complex behaviors (such as speech) are represented and transformed in premotor brain regions. Insights gained through these efforts could enable a new generation of biomimetic systems.