Whole Body Emotion Expression
Bipedal Humanoid Robot
KOBIAN-RIV

The head is equipped with 27-DoFs (8-DoFs for eyebrows, 8-DoFs for eye and eyelids, 7-DoFs for lip, and 1-DoF for jaw) to perform facial expressions. The head size is as large as an adult Japanese female, and all mechanisms necessary to move the head parts (eyebrows, eyelids, etc.) are placed inside the head. To express emotions using cartoon marks, KOBIAN-RII uses five Full color LED displays. Thanks to these, KOBIAN-RII is able to display cartoon marks similar to those used in Japanese comics. Those marks were shown to be useful to increase the recognition of the robot emotions by Japanese users.

Related papers
[1] T. Kishi, et al., “Development of Expressive Robotic Head for Bipedal Humanoid Robot," Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4584-4589, Vilamoura, Algarve, Portugal, October, 2012.
[2] T. Kishi et al., "Development of a comic mark based expressive robotic head adapted to Japanese cultural background", Proceedings of the 2014 IEEE Intelligent Robots and Systems(IROS 2014), pp. 2608-2613, Chicago, USA, September 2014.
[3] 岸他， “顔面各部の広い可動域および顔色により豊かな表情表現が可能な 2 足ヒューマノイドロボット頭部の開発，” 日本ロボット学会誌，Vol. 31，No. 4，pp. 106-116，2013年5月．

The eyebrow is molded with Septon^® produced by KURARAY CO.LTD There are four equally spaced points used to control each eyebrow. The control points are driven by magnets through the cover, placing these on the inner part of the cover.

The facial color is controlled using a blue EL (Electro Luminescence) sheet placed on the forehead under the cover.

The eyelid consists of the upper eyelid and the lower eyelid, which are independent. The upper eyelid has 1-DoF for opening and closing on each side and 1-DoF for rotating, while the lower eyelid has also 1-DoF for opening and closing on each side.

The rotating motion of the upper eyelid is done by a wire driven mechanism with an ultrasonic motor.

The eyes of KOBIAN-RII have 3-DoFs (joined eyes pitch and individual eye yaw) driven by a gimbal structure.

The lip is also molded with Septon^® produced by KURARAY CO.LTD. The lip changes its shape pulled from 5 directions. There is one fixed point on the cover of cheek, two active control points on the upper lip and two active control points on the lower lip. Active control points can move in Y and Z directions.

Black lines and black wrinkles are exposed by moving flexible sheets behind the white cover (Fig.2.14). The forehead is a 7-layer structure, in which are included a LED display, a light diffusion sheet, a LED display for cartoon marks and an EL sheet for complexion expression. These layers are sandwiched between the outer and inner layers of the cover (Fig.2.15).

Five LED displays are implemented (Fig.2.16). Thanks to the use of a flexible display board, it can be deformed along the outer cover of the face. In addition, thanks to the world's smallest class of full-color LED, cartoon marks can be displayed in high resolution (Fig.2.17).

The joint between the neck and the head has pitch and yaw axes, while the joint between the torso and the neck has pitch and roll axes. Similarly as the WE-4RII, the upper and lower extremes are the ones equipped with the pitch axes to enable the motion of sticking out and pulling in the neck when fixing the face orientation.

KOBIAN-RIV has a "Shoulder Base Roll" joint that moves shoulder part up and down and a "Shoulder Base Yaw" joint that moves shoulder part back and forth. Totally, 9-DoFs arm with fast and wide motion capability was developed. (Fig.2.19)．

A wrist joint driven by parallel link mechanism with rotary joints was developed. Parallel link mechanisms have higher output than serial link mechanisms. Additionally, parallel link mechanism with rotary joints has wider movable range than the parallel link mechanism with other types of joints. With this kind of mechanism, the CoG (Center of Gravity) of the forearm can be placed near elbow and it reduces the required power of the upper arm.

The system configuration of KOBIAN-RIV is presented on Fig. 2.22. KOBIAN-RIV's control system is a hybrid of a centralized control system that drives the leg and most of body and arm joints and a distributed control system that drives the head, wrist and several arm joints. A main PC (CPU: Pentium M 1.6GHz, RAM: 2GB, OS: QNX) is placed on its back. The distributed control system is configured with motor controller units developed in our lab. 8 units are equipped to drive the head part, and 2 units are equipped to drive the wrist joints.

We developed a motor controller unit composed of the controller module and the motor driver modules. Changing the motor driver modules allows control of various components and decreases debugging time of the communication and control sub-systems. The motor controller can control 4 brushed DC or ultrasonic motors and read analog sensors from 8 extra channels. Its dimensions are 30 x 46 x 18 mm. It is very small considering all its functions. Furthermore, KOBIAN-RIII's head is equal to adult female's head size because the motor controller units are small.

We are focusing on laugher as a good example of human-robot interaction,because laughter is an involuntary behavior that is possible to detect physiologically, and is said to have health benefits. So far, we have been working on a study to induce laughter in humans by whole body motions by KOBIAN.

For human-robot interaction, it is required to interact with each other with an idea about our partner's psychological state (theory of mind). This research objective is to influence the robot's human partner by making him or her laugh with KOBIAN. Experimental results show the realization of human laughter by KOBIAN. In addition, the decrease of "Depression Dejection" and "Anger Hostility" (POMS parameters) in the subject's state of mind has been confirmed.

We examined the impact of the speed and range of movement of the arm of the robot on the funniness of motions to induce human-laughter. The results of these evaluation experiments showed that faster and wider motion of the arm makes increases the funniness.

We examined the relationship between speed of the robot’s motion in skits and subjective funniness. The experiment was conducted to compare the subjective funniness of comedy skits performed by robot (originally done by human comedians) in different stream speed. The stream speeds were ×1/3, ×1/2, ×1 (normal speed), ×2 and ×3. The result shows that the skits were the most funny when the skits were streamed in the speed which was twice as fast as original skit done by human comedians. Actually, KOBIAN-RIV can move in this speed.

Four characters were defined based on temperamental traits of human. They are “Phlegmatic”, “Sanguine”, “Choleric” and “Melancholic.” “What’s in the box” was selected to perform them. The behavior features of these four characters were extracted from performances by actors. As a result, posture, speed and range of motion and facial expressions are especially essential for performing these characters. Robot’s performance based on the temperamental traits were made considering these features. The examination result about the funniness shows that contradictory performances with positive reactions to the negative situations by “Phlegmatic” and ”Sanguine” give funny impression to the subjects.

Refer to the method proposed in the relevant research in CG animation field, novel algorithm is developed that can automatically generate exaggerated trajectory from reference trajectory. Through this algorithm, "take back" and "follow through" is added to reference trajectory. The result of the experimental evaluation shows that the trajectory generated using this algorithm looks faster, wider and significantly funnier than reference trajectory.

The real feeling that a human being has doesn't always agree with the facial expression. We expressed the robot's real feeling with the display in its chest. The way of expression was inpired by the shapes of the dialogue ballons and the backgrounds used to express emotions and feelings in Japanese comics and cartoons. Based on Russell's emotional model, the real feeling was defined by "activaton" and "pleasantness". Activation was expressed by the speed of the objects moving around in the display, and pleasantness was expressed by their shapes and colors.

There are not only visual ways to make people laugh, but also tactile ways such as tickling. Mechanism for making people laugh by tickling have many unexplained parts. So we developed a robot that rub the side of the torso to tickle, which is a reproducible way of tickling. From the evaluation experiments, we we able to make humans laugh using this tickling robot,.

In order to make humans laugh more efficiently by tickling of robots, diversification of type of tickling stimuli is important. Therefore, the robot that tickle human through stroking on the rib surface was developed. The output power from fingertip can be adjusted by changing the initial length of the spring installed in the finger. In addition, magnets are installed in the finger. When there is unexpectedly large force input at fingertip, these magnets are detached and the force input will not be transmitted. It works as a safety mechanism.

In order to make human laugh more efficiently by tickling by robots, diversification of tickling position on the body is important. Therefore, the robot that tickle humans through stroking on the sole surface was developed. The robot can tickle whole sole surface by moving its roller forward and backward. The robot is able to tickle sole surface with constant force even if the direction of sole changes. This is because the roller is pushed against sole surface by constant force springs.

We use the Six Basic Facial Expressions of Ekman in the robot's facial control, and have defined the seven facial patterns of "Happiness", "Anger", "Disgust", "Fear", "Sadness", "Surprise", and "Neutral" emotional expressions. KOBIAN-RII can express these with its whole body.

There is a need of extending the rigid concept of patterns based on only 6 emotions. A Facial expression generation system applied to humanoid robot KOBIAN-R, based on an extension of Plutchik’s model of emotions and on polynomial classifiers, was developed. It can produce thousands of combinations of facial cues to represent expressions of composite emotions and communication acts, including asymmetrical expressions.

A cultural gap in recognition of facial expression was found. A further study on culture-dependent different generation of facial expression was done and the innovation of the use of Japanese comic symbols to display on the face was introduced. Symbols displayed on face are an additional channel of communication. They can enhance recognition rate of Japanese but they may not always work for Westerners.

Emotions are not only expressed through facial expressions or voice (tone and speech). Nonverbal behavior such as gait influence greatly the perception of emotions. For our research we captured motions with different emotions and emotional intensities of professional actors and non-actors subjects. We extracted emotional gait parameters from the captured data and we modeled those parameters to work our Emotion Mental Model. We assessed both in simulation and with the real robot. Subject recognized the expressed emotions quite well and appreciated the emotionalism of the robot.

How are robots perceived when speaking / using gestures that belong to a certain national culture? A greeting interaction between Egyptians and Japanese with two versions of KOBIAN (one Japanese, one Arabic speaking). Culture-dependent acceptance and discomfort were found: Egyptians prefer the “Arabic robot”, and report discomfort when interacting with the “Japanese robot”; the other way round for Japanese subjects.

This research is the implementation in a walking humanoid robot of a mental model, allowing the dynamical change of the emotional state of the robot based on external stimuli; the emotional state affects the robot decisions and behavior, and it is expressed with both facial and whole-body patterns. To evaluate the importance of the proposed system in the framework of human-robot interaction and communication, we conducted a survey by showing videos of the robot behaviors to subjects. The results show that the integration of dynamical emotion expression and locomotion makes the humanoid robot more appealing to humans, as it is perceived as more "favorable" and "useful", and less "robot-like."

Personal robots will become more and more popular in the future and will be required to actively collaborate and live with their human partners. These personal robots must recognize changing environments and must conduct adequate actions like humans. Object tracking can be said to be a fundamental function from the view point of environmental sensing and reflex reaction against it. We developed an object tracking motion algorithm by using the upper body. Then, we integrated it with an online walking pattern generator and developed an object tracking biped locomotion algorithm. Finally, we conducted an experimental evaluation and confirmed its effectiveness.

In order for a robot to be able to navigate unknown environments, it is necessary for it to actively scan the environment with its sensors. This is important both for minimizing perception uncertainty and the probability of collisions. We proposed a gazing strategy whose objective is to minimize the uncertainty of the 3D reconstruction of the environment along the planned trajectory of the robot. We also proposed a new 3D reconstruction approach for stereo camera systems which better integrates measurements through time. That is done by a novel formulation of occupancy grids which includes all information retrieved from stereo.