Evaluation of the Biosleeve Gesture Control Interface for Telerobotics in Microgravity

PI: Christopher Assad, Jaakko Karras (Co-I), NASA/Jet Propulsion Laboratory

For future missions with human-robot teams, the NASA Space Technology Roadmap highlights challenges in human-robot interfaces for dexterous telerobotic control, gesture control of robotic assistants, and telesupervision. To help address these challenges, JPL is developing a sleeve-based gesture recognition interface that could be embedded and worn in IVA and EVA suits. The JPL BioSleeve incorporates electromyography sensors (to measure muscle activity), inertial sensors (IMUs), and magnetic field sensors, to provide intuitive force and position control signals from natural arm, hand, and finger movements. In the near-term, we plan to propose BioSleeve technology demonstrations aboard the ISS for gestural control of the Canadarm2 and SPHERES robot platforms.

Technology Areas (?)
  • TA04 Robotics, Tele-Robotics and Autonomous Systems
  • TA07 Human Exploration Destination Systems
Problem Statement

The current prototype has successfully achieved >95% accuracy in recognition of 20 static hand gestures and 10 dynamic arm gestures in ground testing. However, both capabilities are affected by the presence of gravity: the patterns of muscle activity change with respect to pose-dependent loading of the arm, and the gravity vector has been used with the magnetic field vector to predict arm pose. Because the near-term planned operational environment is on the International Space Station (ISS), we need to test the gesture recognition performance in the absence of gravity. The parabolic flight aircraft will allow testing in a microgravity environment, which will provide critical data on the performance of the interface, as well as give the team invaluable operational experience that may influence the design, development, and test program for the interface as it advances toward multiple flight opportunities.

Technology Maturation

Key objectives for the reduced gravity flight tests are:
1. Evaluate effects of the microgravity environment on gesture recognition performance, compared to performance in ground tests, after training both on the ground and during flight.
2. Demonstrate BioSleeve interface utility in the microgravity environment by performing a gesture control/manipulation task.
These microgravity tests should mature the BioSleeve technology from TRL 4 to TRL 5 or 6.

Future Customers

This work is significant because NASA and commercial space companies could benefit from BioSleeve gesture control technology for human-robot interactions in multiple applications, such as: natural hand gestures to direct surface rovers by an astronaut on EVA; intuitive telerobotic control for remote manipulation and for high degree-of-freedom robots such as Robonaut and Canadarm2 on ISS; and force control for power-assisted EVA gloves or exoskeletons.

Technology Details

  • Selection Date
    REDDI-F1-15 (Nov 2015)
  • Program Status
  • Current TRL (?)
    Successful FOP Flights
  • 1 Parabolic

Development Team

Web Accessibility and Privacy Notices Curator: Alexander van Dijk Responsible NASA Official: Stephan Ord Last Update: November 16, 2018