Motivation and Research Objective

Up to now, the Waseda Flutist Robot has acted mainly as a passive player. While it was able to play a static score together with a human player, it could not adjust its play to the performance of its parter musician in real-time. Recently we have been working on developing an interaction system that allows the robot to detect visual queues from the other members of an ensemble. As far as the research has progressed until now, in this context, we mainly concentrated on detecting visual queues. We examined the case of the robot being part of a Jazz band. In that situation the band`s performance of a song relies on large parts on the concept of improvisation.

Why Research on Musical Robots?

Music communicates human emotion like no other medium
Easier understanding of perception and processing of human emotion
Robots with a emotional understanding integrate well with a human environment
The human shape of the robot allows us to `emulate` the human emotional perception: same physiology / same constraints of a humanoid robot

Waseda Flutist Robot History


WF-1 (1990)	WF-2 (1992)	WF-3RIX (2002)	WF-4 (2003)

WF-4RIII (2006)	WF-4RIV (2007)	WF-4RIV - Visual Interaction (2008)

Music-based Interaction System (MbIS)

We proposed a Music-based Interaction System (MbIS) to allow for two levels of interaction. The purpose of the two level design is to make the system usable for people with different skill levels. Considering a situation of two human musicians intending to play together, the more advanced person would always have to adjust his way of interaction to the less advanced person. Even in case of having the same skill level two players need to get used to the way they interact and musically communicate with each other. We want to introduce the same kind of behavior for our humanoid robot. A person who has no experience in playing together with the robot will need more time to adjust to the particularities of this type of human-machine interaction. For that reason we designed the beginner level interaction system that provides easy-to-learn controllers having a strong resemblance to established studio equipment. Considering an advanced level player we want the robot to offer a way of interaction that satisfies more refined ways of creative expression. The advanced level interaction system thus allows for free control of the performance parameters.

Acoustic Interaction

In the beginner level acoustic interaction module we reduce the level of complexity by only using rhythmic ways of communication. The robot analyzes the timing of a tone sequence and reproduces this timing in its own performance. As the human player in this mode of interaction can concentrate on the rhythmic part of his play, without caring too much about the harmonic content, only the skill level of a beginner is required.

The advanced level of interaction requires more experience in working interactively with the robot, but also allows for more subtle control of the musical performance. In this mode we allow the user to additionally select a harmony for playing with the flutist robot. In other words, besides analyzing the musical content for rhythmic information, we also examine the tonal part of the sequence played by the human musician. The pattern being played by the robot as an answer contains the adapted rhythm and melody.

Performance Movies

	"Passive" Performance Duo with the WF4-RIV (right-click to download movie)
	Acoustic interaction with the WF4-RIV (right-click to download movie)

Special Thanks

This research was conducted at the Humanoid Robotics Institute (HRI), Waseda University. It was and is supported by a Grant-in-Aid for the WABOT-HOUSE Project by Gifu Prefecture and the RT-GCOE Global Robot Academia of Waseda University.


Humanoid Robotics Institute, Waseda University	Wabot-House Laboratory, Waseda University	Global Robot Academia RT-GCOE of Waseda University