One shot, and ‘it was beautiful’
For Dr. Miguel Nicolelis, the day of the 2014 FIFA World Cup opening ceremony in Brazil was one of the most intense days of his life.
That’s because on that day, he and a team of collaborators demonstrated technology they had developed to allow a paraplegic man to walk again.
As part of the multinational collaborative Walk Again Project, they designed and built a robotic suit that could be controlled by activity detected from the brain. Nicolelis’ expertise was the brain-machine interface designed to capture and translate signals from the user’s brain so they could be communicated to the robotic suit.
“We had one shot,” said Nicolelis, the co-director of the Duke University Center for Neuroengineering and the Walk Again Project director. “It was the most intense day I ever (had) in my life.”
Work on the effort launched officially early last year, Nicolelis said, but there were months of planning before they began to build the technology.
The project involved more than 150 people from around the world including technicians, engineers, mathematicians and scientists. Part of the robotic suit – which they call an exoskeleton – was built in Paris, he said.
The project was paid for through a $13.2 million contract with a Brazilian government-owned company under the Ministry of Science, Technology and Innovation, according to a news release.
The work was based out of a laboratory in São Paulo. Nicolelis is originally from Brazil. He got his medical degree and doctorate from the University of São Paulo. He later established a nonprofit neuroscience institute in Brazil that served as a foundation for the project. Part of his desire to launch the project, he said, was to showcase Brazil to the world in a different way.
“As a Brazilian, soccer is a big deal,” he said. “You grow up thinking about soccer all the time. But I wanted to showcase in the World Cup in the country that is known as the land of soccer that Brazil is also about science.”
Nicolelis said he first realized the technology was possible after completing an experiment in 2007 in collaboration with Gordon Cheng, who is now a professor in Munich. In the experiment, Nicolelis said they sent brain signals from a monkey walking on a treadmill in Durham to make a robot walk in Japan.
“When I saw that, with Gordon, I realized that we could go for the idea that we ended up doing this summer,” he said.
He said a graduate student at his lab at Duke led the design of a non-invasive interface that involved sensors placed on the scalp. The device is different from the type of interface Nicolelis has focused his research on: implants.
“I spent my entire career as you can see in (my) book developing these inter-cranial implants, but they’re not ready for clinical use, yet,” he said. “So we had to come up with an alternative technique.”
David Schwarz, a doctoral candidate who works in Nicolelis’ lab at Duke, said in an email that he was one of the leads on the control system for the exoskeleton. He said he had a hand in building the initial version, and then the team in Brazil tested and perfected it. In April, he went to Brazil to help the team and stayed until late May.
“It was a very intense last couple of months, with 12-to-16-hour work days, but they were worth it,” Schwarz said.
To transmit the brain signals to the suit in the World Cup prototype, they used cables, Nicolelis said, but they also have developed a wireless solution. The project also involved training eight different patients to use the exoskeletion using visualization techniques in virtual reality environments.
“We used virtual reality to give the feeling that they were in the middle of the soccer field kicking a ball,” he said.
Nicolelis also said they were able to design a way to give the user sensory feedback. They placed sensors on the patient’s legs and feet that sent signals to the patient’s arm where they still had sensation.
“When you have a spinal cord injury for instance, let’s say a complete spinal cord injury, it’s not only that you can’t move your legs, but also you cannot feel your legs, you have no sensation,” he said. “So what we did was to sensitize the (exoskeleton).”
On the day of the kick-off, Nicolelis said he and others arrived at the stadium at 6:30 a.m. They set up laboratory space in a locker room. The system is complex, he said, so they had multiple tests.
“We had 30 seconds to get it done, and if the kick didn’t happen, or something went wrong, too bad, that was the only shot we had,” he said. “That was pretty nerve-wracking as you can imagine. And it worked. It was beautiful.”
When Juliano Pinto, 29, completed the kick, Nicolelis said they celebrated on the sideline as if they had scored a goal. He said what was more moving was talking to Pinto.
“For me, what was more moving was when he kicked I went to talk (Pinto), he (said he) felt the ball,” he said. “And he had been in a wheelchair for six years.”
The demonstration was not what Nicolelis had originally envisioned. He said it had been shortened and was moved to a sideline. But they were “extremely happy.”
Schwarz also said restrictions mostly placed by FIFA “severely downsized” the significance of the event. But he said they also hit their milestones, and it was fulfilling to see the smiles on the patients’ faces.
“With the project continuing, it will be very exciting to see what happens two years down the line,” he said.
The lab in Brazil is funded for two and a half more years, Nicolelis said. They’re working on a second prototype. He said they want to reduce the weight of the suit, to increase its range of motion and to add to the types of behaviors that patients can control. Nicolelis wants to move toward developing a prototype of the type that patients could use.
“It’s like a big cell phone – the first cell phones are this big, and you realize that people will not carry that brick,” he said. “They wanted something smaller and so that’s how technology evolves.”