Development of surgical robot hand system

In recent years, laparoscopic surgery has been spreading rapidly. Laparoscopic surgery is minimally invasive and has the advantage of less burden on patients. However, since surgeons don't insert their hands into the body, the sense of touch is lacking. In addition, it can be said that it is a technique with a high learning difficulty because the difficulty of eye-hand coordination is very high. Therefore, HALS (Hand Assisted Laoarosopic Surgery) was developed to reduce this difficulty level.

HALS is a procedure performed by inserting one hand of the surgeon's into the patient's body, which enables protection operations such as organ grasping, tracking, and excluding. However, HALS results in significant fatigue in the arms and shoulders on the side where the hand is inserted. In order to solve this problem, our laboratory will replace the hand inserted by HALS with a robot hand and develop a system that the operator use the hand intuitively.

2. System Overview

In this system, the robot hand plays the role of the surgeon's hand that is inserted into the patient's body. In many cases of HALS, the non-dominant hand of the surgeon is inserted. So our robot hand will also function as so, making it easier to perform excision by securing the surgical field by grasping and pulling the organ and applying tension to the organ. Since these operations cannot be performed with a robot hand alone, a manipulator with a robot hand as an end effector is also prepared and operated at the same time.

This robot hand and the manipulator is operated by the surgeon via the input interface. In order for the surgeon to operate these intuitively ,we adopted a master-slave system. Also, surgeon can not percieve the surface sense of organ by replacing the surgeon's hand with the robotic hand. So we compensate it by attaching a small tactile feedback device to the operating finger. In addition, we aim to enable safer surgery by presenting the state of the tissue on a monitor that displays the images obtained by the endoscope.

3. Configuration

3.1 System diagram

Fig. 2 System diagram

This research system uses ROS(Robot Operating System). The joint interface of the surgeon's hand is detected by the input interface called Leap Motion, and the robot hand is moved. Then, the tactile device is driven based on the force received and the amount of finger movement. This makes it possible to perceive the touch of an organ with a robot hand inserted into the patient's body.

3.2 Robot hand

Fig. 3 Robot hand DOF

Fig. 4 Robot hand picture

Fig. 5 Robot hand CAD

We designed a robot hand based on human hands. The robot hand has a total of 14 degrees of freedom. In order to minimize mechanical power loss and improve joint motion accuracy, an actuator was placed inside each finger closer to the joint. Actuators drive joints by converting linear motion by screws into rotational motion using a curved guide. At present, this robot hand has been successfully moved on the simulation.

Related paper
・ A.Omer, et al., "Development of a Remotely Controlled Robotic Hand System for Surgical Application". Proceeding of the 15th IEEE International Conference on Advanced Robotics and its Social Impacts (ARSO 2019), Beijing, China, November 2019.

3.3 Input Interface

Fig. 6 Leap Motion

Hand tracking technology has already been realized by various methods. In this study, we decided to use a non-contact type sensor in consideration of the effort of attaching and detaching during surgery and attaching a tactile feedback device to the fingertip. The sensor used is Leap-Motion, a small motion sensor for hand position detection, sold by Leap Motion. An infrared stereo camera is built in, and it is possible to acquire the position information of 3D hands and joints including depth.

In this study, we succeeded in moving the robot hand in the simulation from the hand posture data obtained from Leap Motion ROS is used as a system, and Gazebo, which has a high affinity with ROS, was used as the simulation software.

3.4 Haptic device

Fig. 7 Haptic device

We have developed a tactile device that can present normal force(Y axis) and shear force in two directions(X and Z axis) so that three-dimensional contact with an object can be perceived. Tactile presentation is realized by actively moving a plastic plate and deforming the human finger pad. Aimed at a lighter and smaller design by driving by wire transmission instead of attaching a motor directly to the device. This reduces the burden on the wearer's finger.

4. Acknowledgment

This research conducted with the collaboration of Prof. Jamal Hamdi from the department of Surgery at Umm Al-Qura University, Saudi Arabia. This research project is commissioned by Research Institue for next generation robots . This research is fund by JSPS through the research grant number 15K21436 and 18K18406. Moreover, we want to acknowledge Tmsuk Co., Ltd . For its support and assistance in the design and development of the robotic hand mechanism. Finally, we thank to SolidWorks Japan K.K for the 3D-CAD contribution and local governments and public agencies.