National Aeronautics and Space Administration

Glenn Research Center

Natural User Interfaces

Natural User Interfaces (NUIs) are software and hardware systems that allow a user to operate computing devices through intuitive actions related to natural everyday human behavior in the real world. NUIs free the user’s brain for cognition, creativity, and exploration. Over time the user interface becomes invisible to the user. NUIs typically use touch, gestures, force feedback (haptics), and voice as input mechanisms. NUIs allow for more accurate and optimized interaction between physical and digital objects.

A NUI relies on users being able to carry out relatively natural motions, movements, or gestures that they quickly discover control the computer application or manipulate the digital content. When used properly they are more easily understood by novice users.

Recognizing the usefulness of NUIs, the NASA GVIS Team has begun developing systems with NUIs. The GVIS Team is part of a large project, managed by NASA’s Kennedy Space Center, to develop a state-of-the-art launch command and control system. The GVIS Team is working on the Interactive Collaborative Environment sub project. This involves developing:

  • the 3D environment
  • user interactions with the 3D environment
  • communication with the server containing sensor and video data
  • code to interface with a variety of user input devices

Input devices being researched for use on this project include the PixelSense multi-touch screen, Microsoft’s Kinect v1 and v2, the Leap Motion 3D Controller, and the Oculus Rift. The key advantages of using several of these devices are their low cost and the availability of a large quantity of software and libraries, many of which are free and open source.

The GVIS Team is working with researchers who are interested in using some of these devices as data acquisition devices. For example, the Team is currently working with researchers to use the Kinect to measure the height of water in a water table used to study flows around a variety of devices.