GENERAL TRANSFER SKILL SYSTEM
(GTSS)



At Waseda University, the research in humanoid robots has been focused in reproducing human-like activities from an engineering point of view. Such robots are equipped with different sensory systems (visual, auditive, tactile, etc.) in order to interact with humans in different situations. The way of teaching robots to reproduce such kind of skills have been based on different methods (Neural Networks, Motion Capture, etc.). These methods have enable robots to walk, dance, play the flute, show emotions, etc. The utility of such kind of robots may be extended if they are used for transferring skills to unskilled people that not only reproduce the skill learnt previously from an expert, but also evaluate and feedback students in order to enhance their performance. Therefore, the robot must be able of recognizing undergoing actions (Neuronal Networks, Hidden Markov Models, etc.), analyzing the human performance trough the design automated procedures for evaluating the user’s improvements based on quantitative measurement indexes (task quality, task efficiency, etc.), and finally to feedback the student trough different perceptual channels (auditive, graphical, tactile, etc.) that may enhance the student’s performance.

As a first approach, we have been developing a musical teaching system using the Waseda Flutist Robot. The research of the Waseda Flutist Robot has been carried out for more that a decade as a mean for understanding the human flute playing mechanism and as an approach for finding useful applications for humanoid robots to interact with humans. Recently, the introduction of the flutist robot as a novel tool for transferring skills from robot to unskilled flutists has been explored. In the 2004, the idea of using the anthropomorphic flutist robot, as a novel teaching tool for improving the performance of beginner flutist player, has been presented. In that time, a set of preliminary experiments were carried out with Japanese students from high schools. In that experiment, two groups were divided: the first one was taught only by a human professor, while in the second one, the flutist robot was added to the learning process. As a result, the students enhanced their performances quicker in the second case.

Inspired from those preliminary results, a general transfer skill system (GTSS) was introduced in this year.
This proposed system was implemented using the Waseda Flutist Robot No.4 Refined II (WF-4RII). Up to now, this robot has performed musical scores similar as human does. But, by including the GTSS system (figure below), the interaction between humanoid robots and humans can be extended. Basically, the main contributions of the GTSS are the addition of perceptual capabilities to the robot (vision system and recognition system). Furthermore, the idea of implementing a general architecture opens the possibility of using some of the modules to teach different skills and perhaps, to be used by different robots with few modifications. The GTSS is composed by two main modules: internal and external. The internal model is composed by four sub-systems which are used independently of the task to be taught: sensory, recognition, evaluation, and interaction systems. The external module is composed by two subsystems which highly depend on the task to be taught: human skill model and skill evaluation. Inside both subsystems, the experience of a human professor is in someway abstracted. Finally, the connection between the GTSS and the Robotic Control System is done through the TCP/IP. The Robot System communicates the result from the GTSS to the student using an Interaction Interface in order to display the verbal and graphic feedback.





Fig. General Transfer Skill System (from robot to human)




GTSS - System Description

Human Skill Model
Recognition System
The idea of modeling the flute playing skill on the GTSS is useful for two reasons: the robot will be able of autonomously detecting which melody the student is performing and the robot can also use the HMM for evaluating the learner’s performance To achieve such a goals, we have used discrete version of the Hidden Markov Model (HMM) as it doesn’t require a large amount of training data. An ergodic HMM model with eight states have been used.



Model Representation asdsadd
Output Probability dsadasdsasd
The recognition system was implemented in order to identify which melody the student is performing to enable the robot to automatically which melody to play after listening the performance of a flutist. Therefore, we have created a database of five HMM melodies (figure below) by using a total 105 training samples (14 for each melody) of performances of flutist beginners, a professional flutist player and the flutist robot.
The identification process was implemented based on the classical backward and forward algorithms to efficiently evaluate the output probability.







Skill Evaluation

Interaction System

The evaluation of the skill was based on the idea of improving the sound quality by analyzing their performances and then by providing verbal and graphical advices based on the results of such analyses. Due to the complexity of the skill evaluation task, different algorithms were proposed:

1. Evaluation function– to measure the sound quality and it is based on the analysis of the volume level and the harmonic overtone structure. The evaluation function is shown in (1).
2. Power Spectrum analysis– is based on the computation of the short fast Fourier transform to compute the energy of the frequency components of waveform produced by the flute sound.
3. Spectral Entropy analysis– to measure how concentrated or widespread the frequency harmonic distribution of a signal is (2).
4. Symmetric Dot-Pattern1– to graphically display the internal (hidden) structure of the musical performance.

aasdsdasd
add (1)

addsa asd(2)


V:  Volume level.
M-H
: Harmonic – Semi Harmonic level average.
Le-Lo:  Even – Odd harmonic level average.
s: signal to analyze

si: discrete value of the signal s.



The main task of the interaction system is to determine when the robot should start recording the student performance, demonstrating the performance and performing the score together with the beginner player. The robot, to understand such conditions, will mainly used the vision system to identify the presence of a human and to communicate the time the robot is ready to start the interaction. Therefore, as soon as the robot finds a human face, it prompts the student to start the performance and it provides the metronome signal to let know the student when to start the performance (also should send the MIDI start command to the PC sequencer). After that, the robot maintains eye-contact during the performance with the student to increase the interactivity of the learning process.









Evaluation System

The evaluation system is one of the most important modules of the GTSS to implement the proposed learning process; as it enables the robot to analyze and evaluate the student’s performance. The results of this analysis and evaluations will help the student to understand what is going wrong on his/her performance and in some way, to determine what should be improved (by means of the provided verbal and visual feedback from the robot).

In this case, we are proposing to evaluate the following features of the learner's performance:

- Task efficiency (i.e. time to perform the task)
- Task quality (i.e. by comparing with a professor performance).




ACKNOWLEDGMENT

A part of this research was done at the Humanoid Robotics Institute (HRI), Waseda University. This research was supported (in part) by a Gifu-in-Aid for the WABOT-HOUSE Project by Gifu Prefecture. A part of this research has been supported by the Japan Society for the Promotion of Science (JSPS). The authors would like to express thanks to Okino Industries LTD, OSADA ELECTRIC CO. LTD, SHARP CORPORATION, Sony Corporation, Tomy Company, LTD and ZMP Inc. for their financial support for HRI. Finally, we would like to express thanks to SolidWorks Corp., MURAMATSU Inc., CHUKOH CHEMICAL INDUSTRIES LTD. and Mr. Akiko Sato for her valuable help and advice in preparing the experimental setup for testing the music recognition system..

Humanoid Robotics Institute, Waseda University
WASEDA UNIVERSITY WABOT-HOUSE LABORATORY
Japanese Society for the Promotion of Science (JSPS)
Back
Copyright (c) 2005-2006 Team FLUTE / Takanishi Laboratory
All rights reserved.