Why do I talk with my hands?

The limitations of current brain imaging technologies make it difficult to unravel the connection between hand gestures and speech.

This is a very interesting question sent to us by Gabriel Castellanos (thank you!) Many of us gesture with our hands when we speak (and even when we do not) as a form of non-verbal communication. How hand gestures are coordinated with speech remains unclear. In part, it is difficult to monitor natural hand gestures in fMRI-based brain imaging studies as you have to stay still.

“Performing hand movements when stuck in the bore of a scanner is really tough beyond simple signing and keypresses,” explains McGovern Principal Research Scientist Satrajit Ghosh. “Thus ecological experiments of co-speech with motor gestures have not been carried out in the context of a magnetic resonance scanner, and therefore little is known about language and motor integration within this context.”

There have been studies that use proxies such as co-verbal pushing of buttons, and also studies using other imaging techniques, such as electroencephalography (EEG) and magnetoencephalography (MEG), to monitor brain activity during gesturing, but it would be difficult to precisely spatially localize the regions involved in natural co-speech hand gesticulation using such approaches. Another possible avenue for addressing this question would be to look at patients with conditions that might implicate particular brain regions in coordinating hand gestures, but such approaches have not really pinpointed a pathway for coordinating speech and hand movements.

That said, co-speech hand gesturing plays an important role in communication. “More generally co-speech hand gestures are seen as a mechanism for emphasis and disambiguation of the semantics of a sentence, in addition to prosody and facial queues,” says Ghosh. “In fact, one may consider the act of speaking as one large orchestral score involving vocal tract movement, respiration, voicing, facial expression, hand gestures, and even whole body postures acting as different instruments coordinated dynamically by the brain. Based on our current understanding of language production, co-speech or gestural events would likely be planned at a higher level than articulation and therefore would likely activate inferior frontal gyrus, SMA, and others.”

How this orchestra is coordinated and conducted thus remains to be unraveled, but certainly the question is one that gets to the heart of human social interactions.

Do you have a question for The Brain? Ask it here.