People often say one thing while feeling another. But suppressing emotions can take a serious toll, contributing to problems such as anxiety and panic attacks. To help clinicians better understand what patients are really experiencing, researchers led by Penn State scientists have developed a stretchable, rechargeable, sticker-like patch designed to detect genuine emotions by tracking physical signals such as skin temperature and heart rate—even when someone is trying to appear fine.
The team introduced the wearable patch in a study published in Nano Letters. According to the researchers, the device offers a more reliable way to assess emotional states by combining several bodily measurements with facial expression analysis.
“This is a new and improved way to understand our emotions by looking at multiple body signals at once,” said Huanyu “Larry” Cheng, the James L. Henderson, Jr. Memorial Associate Professor of Engineering Science and Mechanics at Penn State and lead author of the study. He noted that relying on facial expressions alone can be misleading because people do not always show how they truly feel.
The Band-Aid-sized patch measures multiple physiological responses associated with emotion, including skin temperature, humidity, heart rate, and blood oxygen levels. Cheng said a key feature is that the sensors are engineered to function independently, reducing interference between different measurements.
The system combines these physiological signals with facial expression data to help distinguish between authentic and performed emotions. It can wirelessly transmit real-time readings to mobile devices and cloud-based systems, which could allow clinicians to evaluate patients remotely. The researchers emphasized that the patch is designed to protect privacy by collecting signal data rather than personal information.
“This technology has the potential to help people who are struggling with their mental health but maybe aren’t being fully honest with others or even themselves about how much they are struggling,” said co-author Yangbo Yuan, a Penn State doctoral student in engineering science and mechanics.
Cheng added that the data could also help reduce cultural or social misunderstandings in clinical settings, where some people may appear more stoic or more expressive than expected. By tracking objective signals, he said, it may be possible to detect conditions such as anxiety or depression earlier.
To build the flexible device, the team layered thin films of metals such as platinum and gold and cut them into wave-like patterns that maintain sensitivity even when stretched or twisted. They also incorporated materials whose electrical behavior changes with temperature, as well as hollow carbon-based structures that absorb water to measure humidity.
The device layout was designed to keep sensors from affecting one another. For instance, the researchers placed a rigid layer beneath the temperature and humidity sensors to shield them from stretching caused by facial movement sensing. A waterproof layer was also used to prevent humidity from distorting temperature and strain readings.
“We’ve engineered this device to measure these different signals independently, without them interfering with each other, providing a much clearer and more accurate picture of what’s happening beneath the surface,” said Libo Gao, co-corresponding author and an associate professor at Xiamen University.
The team then trained an artificial intelligence model to interpret performed versus real emotions. In a pilot study, eight participants were asked to repeatedly display six common facial expressions: happiness, surprise, fear, sadness, anger, and disgust. Each expression was repeated 100 times while the patch tracked facial movement signals, and the data were used to train the model. Three additional participants were later recruited to further evaluate performance. The system classified performed facial expressions with 96.28% accuracy.
To test recognition of real emotions, the researchers monitored the same participants while they watched video clips intended to evoke emotional responses. The device identified emotions with 88.83% accuracy. The sensor data also aligned with known physiological patterns linked to emotion, including increases in skin temperature and heart rate during surprise and anger.
Because the patch can transmit data wirelessly, the researchers said it could support telemedicine by enabling healthcare professionals to monitor patients remotely and provide timely emotional support.
“This sensor can serve a vital function in bridging gaps in access to care,” Cheng said, pointing to increasing stress levels and the value of early warning signs that could enable proactive help.
Beyond emotion monitoring, the researchers believe the platform could contribute to other AI-assisted medical applications, including assessing the mental and emotional state of non-verbal patients, identifying behavioral and psychological symptoms of dementia, and recognizing opioid overdose. They also noted potential future uses in chronic wound monitoring, disease management, tracking neurodegenerative disease progression, and even athletic performance monitoring.
“While still in the research and development phase, this device is a significant step forward in our ability to monitor and understand human emotions, potentially paving the way for more proactive and personalized approaches to mental health care,” Cheng said.
Additional contributors included Hongcheng Xu of Xi’an Jiaotong University. Funding for the Penn State team’s work came from the U.S. National Institutes of Health and the U.S. National Science Foundation.
