Stanford researchers advance the performance of thought-controlled computer cursors

Citation

Servick, K. (2013), "Stanford researchers advance the performance of thought-controlled computer cursors", Industrial Robot, Vol. 40 No. 2. https://doi.org/10.1108/ir.2013.04940baa.002

Stanford researchers advance the performance of thought-controlled computer cursors

Article Type: Mini features From: Industrial Robot: An International Journal, Volume 40, Issue 2

“Stanford researchers have designed the fastest, most accurate mathematical algorithm yet for brain-implantable prosthetic systems that can help disabled people manoeuvre computer cursors with their thoughts. The algorithm’s speed, accuracy and natural movement approach those of a real arm” (Kelly Servick).

When a paralyzed person imagines moving a limb, cells in the part of the brain that controls movement activate, as if trying to make the immobile limb work again.

Despite a neurological injury or disease that has severed the pathway between brain and muscle, the region where the signals originate remains intact and functional.

In recent years, neuroscientists and neuroengineers working in prosthetics have begun to develop brain-implantable sensors that can measure signals from individual neurons.

After those signals have been decoded through a mathematical algorithm, they can be used to control the movement of a cursor on a computer screen – in essence, the cursor is controlled by thoughts.

The work is part of a field known as neural prosthetics.

A team of Stanford researchers have now developed a new algorithm, known as ReFIT, that vastly improves the speed and accuracy of neural prosthetics that control computer cursors. The results were published November 18 in the journal Nature Neuroscience in a paper by Krishna Shenoy, a Professor of Electrical Engineering, Bioengineering and Neurobiology at Stanford, and a team led by Research Associate Dr Vikash Gilja and bioengineering doctoral candidate Paul Nuyujukian.

In side-by-side demonstrations with rhesus monkeys, cursors controlled by the new algorithm doubled the performance of existing systems and approached performance of the monkey’s actual arm in controlling the cursor. Better yet, more than four years after implantation, the new system is still going strong, while previous systems have seen a steady decline in performance over time.

“These findings could lead to greatly improved prosthetic system performance and robustness in paralyzed people, which we are actively pursuing as part of the FDA Phase-I BrainGate2 clinical trial here at Stanford,” said Shenoy.

Sensing mental movement in real time

The system relies on a sensor implanted into the brain, which records “action potentials” in neural activity from an array of electrode sensors and sends data to a computer. The frequency with which action potentials are generated provides the computer important information about the direction and speed of the user’s intended movement.

The ReFIT algorithm that decodes these signals represents a departure from earlier models. In most neural prosthetics research, scientists have recorded brain activity while the subject moves or imagines moving an arm, analyzing the data after the fact. “Quite a bit of the work in neural prosthetics has focused on this sort of offline reconstruction,” said Gilja, the first author of the paper.

The Stanford team wanted to understand how the system worked “online,” under closed-loop control conditions in which the computer analyzes and implements visual feedback gathered in real time as the monkey neurally controls the cursor toward an onscreen target.

The system is able to make adjustments on the fly when guiding the cursor to a target, just as a hand and eye would work in tandem to move a mouse-cursor onto an icon on a computer desktop.

If the cursor were straying too far to the left, for instance, the user likely adjusts the imagined movements to redirect the cursor to the right. The team designed the system to learn from the user’s corrective movements, allowing the cursor to move more precisely than it could in earlier prosthetics.

To test the new system, the team gave monkeys the task of mentally directing a cursor to a target – an onscreen dot – and holding the cursor there for half a second. ReFIT performed vastly better than previous technology in terms of both speed and accuracy.

The path of the cursor from the starting point to the target was straighter and it reached the target twice as quickly as earlier systems, achieving 75-85 percent of the speed of the monkey’s arm.

“This paper reports very exciting innovations in closed-loop decoding for brain-machine interfaces. These innovations should lead to a significant boost in the control of neuroprosthetic devices and increase the clinical viability of this technology,” said Jose Carmena, an Associate Professor of Electrical Engineering and Neuroscience at the University of California-Berkeley.

A smarter algorithm

Critical to ReFIT’s time-to-target improvement was its superior ability to stop the cursor. While the old model’s cursor reached the target almost as fast as ReFIT, it often overshot the destination, requiring additional time and multiple passes to hold the target.

The key to this efficiency was in the step-by-step calculation that transforms electrical signals from the brain into movements of the cursor onscreen. The team had a unique way of “training” the algorithm about movement. When the monkey used his arm to move the cursor, the computer used signals from the implant to match the arm movements with neural activity.

Next, the monkey simply thought about moving the cursor, and the computer translated that neural activity into onscreen movement of the cursor. The team then used the monkey’s brain activity to refine their algorithm, increasing its accuracy.

The team introduced a second innovation in the way ReFIT encodes information about the position and velocity of the cursor. Gilja said that previous algorithms could interpret neural signals about either the cursor’s position or its velocity, but not both at once. ReFIT can do both, resulting in faster, cleaner movements of the cursor.

This research was funded by the Christopher and Dana Reeve Paralysis Foundation, the National Science Foundation, National Defense Science and Engineering Graduate Fellowships, Stanford Graduate Fellowships, Defense Advanced Research Projects Agency (“Revolutionizing Prosthetics” and “REPAIR”) and the National Institutes of Health (NINDS-CRCNS and Director’s Pioneer Award).

Other contributing researchers include Cynthia Chestek, John Cunningham, Byron Yu, Joline Fan, Mark Churchland, Matthew Kaufman, Jonathan Kao and Stephen Ryu.

Figure 1 From left, Stanford researchers Vikash Gilja, Krishna Shenoy and Paul Nuyujukian

Kelly Servick is a science-writing intern at the Stanford University School of Engineering (Figure 1).

Kelly Servick