Department Seminar of Alon Mizrahe - TeleFMG: A Wearable Force-Myography Device for Natural Teleoperation of Multi-finger Robotic Hands

School of Mechanical Engineering Seminar
Monday 13.11.2023 at 14:00

ZOOM SEMINAR

TeleFMG: A Wearable Force-Myography Device for Natural
Teleoperation of Multi-finger Robotic Hands

Alon Mizrahi

M.Sc. student of Avishai Sintov

School of Mechanical Engineering, Tel Aviv University, Tel Aviv, Israel

Teleoperation enables a user to perform dangerous tasks (e.g., work in disaster zones or in chemical plants) from a remote location. Nevertheless, common approaches often provide cumbersome and unnatural usage. In this thesis, we propose TeleFMG, an approach for teleoperation of a multi-finger robotic hand through natural motions of the user’s hand. By using a low-cost wearable Force-Myography (FMG) device, musculoskeletal activities on the user’s forearm are mapped to hand poses which, in turn, are mimicked by a robotic hand. The mapping is performed by a data-based model based on the Temporal Convolutional Network. The model considers spatial positions of the sensors on the forearm along with temporal dependencies of the FMG signals. A set of experiments show the ability of a teleoperator to control a multi-finger hand through intuitive and natural finger motion. A robot is shown to successfully mimic the user’s hand in object grasping and gestures. Furthermore, transfer to a new user is evaluated while showing that fine-tuning with a limited amount of new data significantly improves accuracy.       
This research has been submitted to IEEE Robotics and Automation Letters and is currently under review after a round of comments and corrections.

https://tau-ac-il.zoom.us/j/86497933118