Facebook has taken a look at its plans for a new interface for augmented reality, based on technology from CTRL-Labs, the startup it acquired in 2019. In a video, it shows wristbands that use electromyography (EMG) to translate subtle neural signals into actions – such as typing, fighting or playing games like an archery simulator. The bands also provide haptic feedback, creating a system that responds more than basic hand detection options.
Facebook Reality Labs has published a blog post outlining its work on a prototype of wristbands. In the simplest way, the groups can follow basic gestures that Facebook calls ‘clicks’, which are supposed to perform reliably and easily. They are a bit like the everyday Microsoft HoloLens “air tap” gesture, but detected with the nerve signals running along your arms, rather than visual sensors mounted on a headset.
However, the bands can theoretically do much more. For example, they can track the nerve signals your brain sends to your fingers as you type, allowing you to type on a virtual keyboard without a physical button. And unlike a regular keyboard, the straps can slowly adapt to the way you type – so they can “learn” the way your fingers move when you make regular typing mistakes and then automatically correct them and instead determine what you probably meant. .
This would be a big change in the way most people deal with computers, but conceptually it is not a major update on how CTRL-Labs described its work years ago. In fact, the ultimate possibilities for EMG wristbands are much more thought-provoking: finally, you can perform the same typing style gesture by: think to move your hands instead of really moving it. Facebook wants to further streamline user interactions by relying on artificial intelligence and augmented reality glasses, with which it announced last year that it was working.
Even in their simpler versions, these controllers offer an interface that you can carry all the time instead of picking up and holding, like the current Oculus Touch VR controllers. The effect may be similar to smaller boot offerings such as the Mudra Band, which observes gestures via an Apple Watch band.
One important new addition is haptics. Facebook says it is implementing different prototypes that can give you subtle feedback using different methods. One, the ‘Bellowband’, has eight pneumatic bellows around each wrist. It can be inflated or deflated in patterns that produce clear sensations. Another is ‘Tasbi’, which uses vibrating actuators and a ‘new wrist blade mechanism’. Combined with visual feedback from an AR headset, they can provide a wealth of information through a simple and intuitive interface.
Facebook insists that although the group reads neural signals, “it is not similar to mind reading.” This is how it explains the concept:
You have many thoughts and you choose to act on only some of them. When this happens, your brain sends signals to your hands and fingers to tell them to move in specific ways to perform actions such as typing and swiping. It’s about decoding the signals on your wrist – the actions you’ve already decided to perform – and translating them into digital commands for your device.
CTRL-Labs still characterized this technology as a brain-computer interface, but it is in stark contrast to technologies such as Elon Musk’s Neuralink – which reads neural activity directly from the brain through an implant. Implants have unique uses, especially for people with paralysis or amputated limbs, whose bodies simply cannot send nervous signals to a wristband. But Facebook CEO Mark Zuckerberg recently criticized implants as a short-term consumer technology, saying that “we do not think people want to have their heads drilled to use virtual or augmented reality.” Wristbands also do not have quite the same scare factor for privacy as something that reads your mind at the source.
That said, the bands will almost certainly collect a lot of data. This can include incredibly fine variations in typing patterns; overall levels of physical stress; and any biometric information captured by fitness sensors, augmented reality glasses and other technologies that can be integrated with the tires. (Facebook Reality Labs notes that it has a “neuroethical program” that explores the privacy, security, and safety implications of AR and neural interface technology.)
Like most wearable technology, EMG tires offer an intimate look at how our bodies move – and while it may not sound quite as creepy as a tire that reads your mind, it still requires a lot of confidence.