In face-to-face conversations, people often use their hands to express meaning. But do listeners actually rely on these gestures to predict what someone will say next? Researchers from the Max Planck Institute for Psycholinguistics and Radboud University in Nijmegen explored this question using virtual avatars and found that listeners do use gestures to anticipate upcoming speech.
The team combined behavioral results with brain activity measurements and concluded that hand movements can make language easier to process, highlighting how strongly human communication depends on both speech and visual signals.
For example, when people talk about typing, they may wiggle their fingers as if pressing keys. These meaningful movements, known as iconic gestures, can support comprehension. Previous research has also shown that questions accompanied by such gestures tend to receive faster responses in conversation.
One reason gestures may help is timing: they often begin before the relevant word is spoken. This means they can provide an early clue about what is coming, allowing listeners to form expectations in advance.
To test whether gestures truly help predict upcoming words, the researchers conducted two experiments using virtual avatars. These avatars allowed precise control over both speech and movement while maintaining a natural visual interaction.
Testing whether people can guess the missing word
In the first experiment, participants watched and listened to an avatar asking questions such as, “How old were you when you learned to … type?”, with a pause before the final word. During the question, the avatar either performed a matching gesture (typing motion), an unrelated movement (such as scratching an arm), or no movement at all.
Participants heard the question only up to the pause and were asked to guess the final word. They predicted it more accurately when they had seen the corresponding gesture.
Measuring anticipation in brain activity
In the second experiment, another group of participants listened to full questions while their brain activity was recorded using electroencephalography (EEG). During the pause before the final word, gestures influenced brain signals associated with anticipation.
After the word was spoken, gestures also affected brain responses related to how easily meaning is processed. Specifically, researchers observed a reduced N400 effect, which is typically linked to easier semantic processing. In other words, when participants saw gestures, understanding required less effort.
The findings suggest that even during passive listening, people use hand gestures as predictive cues about upcoming speech.
What this could mean for technology
The researchers also highlighted a practical implication: gestures produced by virtual avatars can improve understanding. If robots or digital assistants are to communicate more naturally and effectively, they may need not only speech but also meaningful gestures.
