Introducing Nikola, the emotional android boy

Rosie the robot.

Rosie the robot maid was considered science fiction when she debuted on the Jetson’s cartoon over 50 years ago. Although the reality of the helpful robot is currently more science and less fiction, there are still many challenges that need to be met, including being able to detect and express emotions. The recent study led by Wataru Sato from the RIKEN Guardian Robot Project focused on building a humanoid robot, or android, that can use its face to express a variety of emotions. The result is Nikola, an android head that looks like a hairless boy.

Inside Nikola’s face are 29 pneumatic actuators that control the movements of artificial muscles. Another 6 actuators control head and eyeball movements. Pneumatic actuators are controlled by air pressure, which makes the movements silent and smooth. The team placed the actuators based on the Facial Action Coding System (FACS), which has been used extensively to study facial expressions. Past research has identified numerous facial action units—such as ‘cheek raiser’ and ‘lip pucker’—that comprise typical emotions such as happiness or disgust, and the researchers incorporated these action units in Nikola’s design.

Some of Nikola’s facial action units (AUs).  Row 1: (1) Inner brow raiser, (2) outer brow raiser, (4) brow lowerer, (5) upper lid raiser, (6) cheek raiser, (7) lid tightener. Row 2: (10) Upper lip raiser, (12) lip corner puller, (14) dimpler, (15) lip corner depressor, (16) lower lip depressor, (18) lip pucker. Row 3: (20) Lip stretcher, (22) lip funneler, (25) lips part, (26) jaw drop, (43) eyes closed. For AU 25, AU 25 + 26 is shown.

Typically, studies of emotions, particularly how people react to emotions, have a problem. It is difficult to do a properly controlled experiment with live people interacting, but at the same time, looking at photos or videos of people is less natural, and reactions aren’t the same. “The hope is that with androids like Nikola, we can have our cake and eat it too,” says Sato. “We can control every aspect of Nikola’s behavior, and at the same time study live interactions.” The first step was to see if Nikola’s facial expressions were understandable.

A person certified in FACS scoring was able to identify each facial action unit, indicating that Nikola’s facial movements accurately resemble those of a real human. A second test showed that everyday people could recognize the six prototypical emotions—happiness, sadness, fear, anger, surprise, and disgust—in Nikola’s face, albeit to varying accuracies. This is because Nikola’s silicone skin is less elastic than real human skin and cannot form wrinkles very well. Thus, emotions like disgust were harder to identify because the action unit for nose wrinkling could not be included.

Test yourself! Can you identify the six emotions that Nikola is expressing? (And remember, this is not computer animation. The video was made by filming the android’s head)

“In the short term, androids like Nikola can be important research tools for social psychology or even social neuroscience,” says Sato. “Compared with human confederates,  androids are good at controlling behaviors and can facilitate rigorous empirical investigation of human social interactions.” As an example, the researchers asked people to rate the naturalness of Nikola’s emotions as the speed of his facial movements was systematically controlled. They researchers found that the most natural speed was slower for some emotions like sadness than it was for others like surprise.

While Nikola still lacks a body, the ultimate goal of the Guardian Robot Project is to build an android that can assist people, particularly those whth physical needs who might live alone. “Androids that can emotionally communicate with us will be useful in a wide range of real-life situations, such as caring for older people, and can promote human wellbeing,” says Sato. ?

Rosie the robot.

Rosie the robot maid was considered science fiction when she debuted on the Jetson’s cartoon over 50 years ago. Although the reality of the helpful robot is currently more science and less fiction, there are still many challenges that need to be met, including being able to detect and express emotions. The recent study led by Wataru Sato from the RIKEN Guardian Robot Project focused on building a humanoid robot, or android, that can use its face to express a variety of emotions. The result is Nikola, an android head that looks like a hairless boy.

Inside Nikola’s face are 29 pneumatic actuators that control the movements of artificial muscles. Another 6 actuators control head and eyeball movements. Pneumatic actuators are controlled by air pressure, which makes the movements silent and smooth. The team placed the actuators based on the Facial Action Coding System (FACS), which has been used extensively to study facial expressions. Past research has identified numerous facial action units—such as ‘cheek raiser’ and ‘lip pucker’—that comprise typical emotions such as happiness or disgust, and the researchers incorporated these action units in Nikola’s design.

Some of Nikola’s facial action units (AUs).  Row 1: (1) Inner brow raiser, (2) outer brow raiser, (4) brow lowerer, (5) upper lid raiser, (6) cheek raiser, (7) lid tightener. Row 2: (10) Upper lip raiser, (12) lip corner puller, (14) dimpler, (15) lip corner depressor, (16) lower lip depressor, (18) lip pucker. Row 3: (20) Lip stretcher, (22) lip funneler, (25) lips part, (26) jaw drop, (43) eyes closed. For AU 25, AU 25 + 26 is shown.

Typically, studies of emotions, particularly how people react to emotions, have a problem. It is difficult to do a properly controlled experiment with live people interacting, but at the same time, looking at photos or videos of people is less natural, and reactions aren’t the same. “The hope is that with androids like Nikola, we can have our cake and eat it too,” says Sato. “We can control every aspect of Nikola’s behavior, and at the same time study live interactions.” The first step was to see if Nikola’s facial expressions were understandable.

A person certified in FACS scoring was able to identify each facial action unit, indicating that Nikola’s facial movements accurately resemble those of a real human. A second test showed that everyday people could recognize the six prototypical emotions—happiness, sadness, fear, anger, surprise, and disgust—in Nikola’s face, albeit to varying accuracies. This is because Nikola’s silicone skin is less elastic than real human skin and cannot form wrinkles very well. Thus, emotions like disgust were harder to identify because the action unit for nose wrinkling could not be included.

Test yourself! Can you identify the six emotions that Nikola is expressing? (And remember, this is not computer animation. The video was made by filming the android’s head)

“In the short term, androids like Nikola can be important research tools for social psychology or even social neuroscience,” says Sato. “Compared with human confederates,  androids are good at controlling behaviors and can facilitate rigorous empirical investigation of human social interactions.” As an example, the researchers asked people to rate the naturalness of Nikola’s emotions as the speed of his facial movements was systematically controlled. They researchers found that the most natural speed was slower for some emotions like sadness than it was for others like surprise.

While Nikola still lacks a body, the ultimate goal of the Guardian Robot Project is to build an android that can assist people, particularly those with physical needs who might live alone. “Androids that can emotionally communicate with us will be useful in a wide range of real-life situations, such as caring for older people, and can promote human wellbeing,” says Sato. ?