Mind-controlled video games are also blossoming. Seventeen hundred software developers are working with NeuroSky, many of them on the company’s $129 million Mindwave Mobile headset. These video games use a small, portable EEG sensor wrapped around your forehead that allows you to navigate in virtual reality, where the movements of your avatar are controlled mentally. As you maneuver your avatar on the video screen, you can fire weapons, evade enemies, rise to new levels, score points, etc., as in an ordinary video game, except that everything is hands-free.
“There’s going to be a whole ecosystem of new players, and NeuroSky is very well positioned to be like the Intel of this new industry,” claims Alvaro Fernandez of SharpBrains, a market research firm.
Besides firing virtual weapons, the EEG helmet can also detect when your attention begins to flatten out. NeuroSky has been getting inquiries from companies concerned about injuries to workers who lose concentration while operating a dangerous machine or who fall asleep at the wheel. This technology could be a lifesaver, alerting the worker or driver that he is losing his focus. The EEG helmet would set off an alarm when the wearer dozes off. (In Japan, this headset is already creating a fad among partygoers. The EEG sensors look like cat ears when you put them on your head. The ears suddenly rise when your attention is focused and then flatten out when it fades. At parties, people can express romantic interest just by thinking, so you know if you are impressing someone.)
But perhaps the most novel applications of this technology are being pursued by Dr. Miguel Nicolelis of Duke University. When I interviewed him, he told me that he thinks he can duplicate many of the devices found only in science fiction.
SMART HANDS AND MIND MELDS
Dr. Nicolelis has shown that this brain-machine interface can be done across continents. He places a monkey on a treadmill. A chip is positioned on the monkey’s brain, which is connected to the Internet. On the other side of the planet, in Kyoto, Japan, signals from the monkey are used to control a robot that can walk. By walking on the treadmill in North Carolina, the monkey controls a robot in Japan, which executes the same walking motion. Using only his brain sensors and the reward of a food pellet, Dr. Nicolelis has trained these monkeys to control a humanoid robot called CB-1 halfway around the world.
He is also tackling one of the main problems with brain-machine interface: the lack of feeling. Today’s prosthetic hands don’t have a sense of touch, and hence they feel foreign; because there’s no feedback, they might accidentally crush someone’s fingers while engaging in a handshake. Picking up an eggshell with a mechanical arm would be nearly impossible.
Nicolelis hopes to circumvent this problem by having a direct brain-to-brain interface. Messages would be sent from the brain to a mechanical arm that has sensors, which would then send messages directly back to the brain, thereby bypassing the stem altogether. This brain-machine-brain interface (BMBI) could enable a clean, direct feedback mechanism to allow for the sensation of touch.
Dr. Nicolelis started by connecting the motor cortex of rhesus monkeys to mechanical arms. These mechanical arms have sensors on them, which then send signals back to the brain by electrodes connected to the somatosensory cortex (which registers the sensation of touch). The monkeys were given a reward after every successful trial; they learned how to use this apparatus within four to nine trials.
To do this, Dr. Nicolelis had to invent a new code that would represent different surfaces (which were rough or smooth). “After a month of practice,” he told me, “this part of the brain learns this new code, and starts to associate this new artificial code that we created with different textures. So this is the first demonstration that we can create a sensory channel” that can simulate sensations of the skin.
I mentioned to him that this idea sounds like the “holodeck” of Star Trek, where you wander in a virtual world but feel sensations when you bump into virtual objects, just as if they were real. This is called “haptic technology,” which uses digital technology to simulate the sense of touch. Nicolelis replied, “Yes, I think this is the first demonstration that something like the holodeck will be possible in the near future.”
The holodeck of the future might use a combination of two technologies. First, people in the holodeck would wear Internet contact lenses, so that they would see an entirely new virtual world everywhere they looked. The scenery in your contact lens would change instantly with the push of a button. And if you touched any object in this world, signals sent into the brain would simulate the sensation of touch, using BMBI technology. In this way, objects in the virtual world you see inside your contact lens would feel solid.
Brain-to-brain interface would make possible not only haptic technology, but also an “Internet of the mind,” or brain-net, with direct brain-to-brain contact. In 2013, Dr. Nicolelis was able to accomplish something straight out of Star Trek, a “mind meld” between two brains. He started with two groups of rats, one at Duke University, the other in Natal, Brazil. The first group learned to press a lever when seeing a red light. The second group learned to press a lever when their brains were stimulated by a signal sent via an implant. Their reward for pressing the lever was a sip of water. Then Dr. Nicolelis connected the motor cortices of the brains of both groups via a fine wire through the Internet.
When the first group of rats saw the red light, a signal was sent over the Internet to Brazil to the second group, which then pressed the lever. In seven out of ten trials, the second group of rats correctly responded to the signals sent by the first group. This was the first demonstration that signals could be transferred and also interpreted correctly between two brains. It’s still a far cry from the mind meld of science fiction, where two minds merge into one, because this is still primitive and the sample size is small, but it is a proof of principle that a brain-net might be possible.
In 2013, the next important step was taken when scientists went beyond animal studies and demonstrated the first direct human brain-to-brain communication, with one human brain sending a message to another via the Internet.
This milestone was achieved at the University of Washington, with one scientist sending a brain signal (move your right arm) to another scientist. The first scientist wore an EEG helmet and played a video game. He fired a cannon by imagining moving his right arm, but was careful not to move it physically.
The signal from the EEG helmet was sent over the Internet to another scientist, who was wearing a transcranial magnetic helmet carefully placed over the part of his brain that controlled his right arm. When the signal reached the second scientist, the helmet would send a magnetic pulse into his brain, which made his right arm move involuntarily, all by itself. Thus, by remote control, one human brain could control the movement of another.
This breakthrough opens up a number of possibilities, such as exchanging nonverbal messages via the Internet. You might one day be able to send the experience of dancing the tango, bungee jumping, or skydiving to the people on your e-mail list. Not just physical activity, but emotions and feelings as well might be sent via brain-to-brain communication.
Nicolelis envisions a day when people all over the world could participate in social networks not via keyboards, but directly through their minds. Instead of just sending e-mails, people on the brain-net would be able to telepathically exchange thoughts, emotions, and ideas in real time. Today a phone call conveys only the information of the conversation and the tone of voice, nothing more. Video conferencing is a bit better, since you can read the body language of the person on the other end. But a brain-net would be the ultimate in communications, making it possible to share the totality of mental information in a conversation, including emotions, nuances, and reservations. Minds would be able to share their most intimate thoughts and feelings.
TOTAL IMMERSION ENTERTAINMENT
Developing a brain-net may also have an impact on the multibillion-dollar entertainment industry. Back in the 1920s, the technology of tape-recording sound as well as light was perfected
. This set off a transformation in the entertainment industry as it made the transition from silent movies to the “talkies.” This basic formula of combining sound and sight hasn’t changed much for the past century. But in the future, the entertainment industry may make the next transition, recording all five senses, including smell, taste, and touch, as well as the full range of emotions. Telepathic probes would be able to handle the full range of senses and emotions that circulate in the brain, producing a complete immersion of the audience in the story. Watching a romantic movie or an action thriller, we would be swimming in an ocean of sensations, as if we were really there, experiencing all the rush of feelings and the emotions of the actors. We would smell the perfume of the heroine, feel the terror of the victims in a horror movie, and relish the vanquishing of the bad guys.
This immersion would involve a radical shift in how movies are made. First, actors would have to be trained to act out their roles with EEG/MRI sensors and nanoprobes recording their sensations and emotions. (This would place an added burden on the actors, who would have to act out each scene by simulating all five senses. In the same way that some actors could not make the transition from silent movies to the talkies, perhaps a new generation of actors will emerge who can act out scenes with all five senses.) Editing would require not just cutting and splicing film, but also combining tapes of the various sensations within each scene. And finally the audience, as they sit in their seats, would have all these electrical signals fed into their brains. Instead of 3-D glasses, the audience would wear brain sensors of some sort. Movie theaters would also have to be retrofitted to process this data and then send it to the people in the audience.
CREATING A BRAIN-NET
Creating a brain-net that can transmit such information would have to be done in stages. The first step would be inserting nanoprobes into important parts of the brain, such as the left temporal lobe, which governs speech, and the occipital lobe, which governs vision. Then computers would analyze these signals and decode them. This information in turn could be sent over the Internet by fiber-optic cables.
More difficult would be to insert these signals back into another person’s brain, where they could be processed by the receiver. So far, progress in this area has focused only on the hippocampus, but in the future it should be possible to insert messages directly into other parts of the brain corresponding to our sense of hearing, light, touch, etc. So there is plenty of work to be done as scientists try to map the cortices of the brain involved in these senses. Once these cortices have been mapped—such as the hippocampus, which we’ll discuss in the next chapter—it should be possible to insert words, thoughts, memories, and experiences into another brain.
Dr. Nicolelis writes, “It is not inconceivable that our human progeny may indeed muster the skills, technology, and ethics needed to establish a functional brain-net, a medium through which billions of human beings consensually establish temporary direct contacts with fellow human beings through thought alone. What such a colossus of collective consciousness may look like, feel like, or do, neither I nor anyone in our present time can possibly conceive or utter.”
THE BRAIN-NET AND CIVILIZATION
A brain-net may even change the course of civilization itself. Each time a new communication system has been introduced, it has irrevocably accelerated changes in society, lifting us from one era to the next. In prehistoric times, for thousands of years our ancestors were nomads wandering in small tribes, communicating with one another through body language and grunts. The coming of language allowed us for the first time to communicate symbols and complex ideas, which facilitated the rise of villages and eventually cities. Within the last few thousand years, written language has enabled us to accumulate knowledge and culture across generations, allowing for the rise of science, the arts, architecture, and huge empires. The coming of the telephone, radio, and TV extended the reach of communication across continents. The Internet now makes possible the rise of a planetary civilization that will link all the continents and peoples of the world. The next giant step might be a planetary brain-net, in which the full spectrum of senses, emotions, memories, and thoughts are exchanged on a global scale.
“WE WILL BE PART OF THEIR OPERATING SYSTEM”
When I interviewed Dr. Nicolelis, he told me that he became interested in science at an early age while growing up in his native Brazil. He remembers watching the Apollo moon shot, which captured the world’s attention. To him, it was an amazing feat. And now, he told me, his own “moon shot” is making it possible to move any object with the mind.
He became interested in the brain while still in high school, where he came across a 1964 book by Isaac Asimov titled The Human Brain. But he was disappointed by the end of the book. There was no discussion about how all these structures interacted with one another to create the mind (because no one knew the answer back then). It was a life-changing moment and he realized that his own destiny might lie in trying to understand the secrets of the brain.
About ten years ago, he told me, he began to look seriously into doing research on his childhood dream. He started by taking a mouse and letting it control a mechanical device. “We placed sensors into the mouse which read the electrical signals from the brain. Then we transmitted these signals to a little robotic lever that could bring water from a fountain back to the mouse’s mouth. So the animal had to learn how to mentally move the robotic device to bring the water back. That was the first-ever demonstration that you could connect an animal to a machine so that it could operate a machine without moving its own body,” he explained to me.
Today he can analyze not just fifty but one thousand neurons in the brain of a monkey, which can reproduce various movements in different parts of the monkey’s body. Then the monkey can control various devices, such as mechanical arms, or even virtual images in cyberspace. “We even have a monkey avatar that can be controlled by the monkey’s thoughts without the monkey making any movement,” he told me. This is done by having the monkey watch a video in which he sees an avatar that represents his body. Then, by mentally commanding his body to move, the monkey makes the avatar move in the corresponding way.
Nicolelis envisions a day in the very near future when we will play video games and control computers and appliances with our minds. “We will be part of their operating system. We will be immersed in them with mechanisms that are very similar to the experiments that I am describing.”
EXOSKELETONS
The next undertaking for Dr. Nicolelis is the Walk Again Project. Its goal is nothing less than a complete exoskeleton for the body controlled by the mind. At first, an exoskeleton conjures up an image of something from the Iron Man movies. Actually, it is a special suit that encases the entire body so that the arms and legs can move via motors. He calls it a “wearable robot.” (See Figure 10.)
His goal, he said, is to help the paralyzed “walk by thinking.” He plans to use wireless technology, “so there’s nothing sticking out of the head.… We are going to record twenty to thirty thousand neurons, to command a whole body robotic vest, so he can think and walk again and move and grab objects.”
Nicolelis realizes that a series of hurdles must be overcome before the exoskeleton becomes a reality. First, a new generation of microchips must be created that can be placed in the brain safely and reliably for years at a time. Second, wireless sensors must be created so the exoskeleton can roam freely. The signals from the brain would be received wirelessly by a computer the size of a cell phone that would probably be attached to your belt. Third, new advances must be made in deciphering and interpreting signals from the brain via computers. For the monkeys, a few hundred neurons were necessary to control the mechanical arms. For a human, you need, at minimum, several thousand neurons to control an arm or leg. And fourth, a power supply must be found that is portable and powerful enough to energize the entire exoskeleton.
Figure 10. This is the exoskeleton that Dr. Nicolelis hopes will be controlled by the mind of a tot
ally paralyzed person. (illustration credit 4.1)
Nicolelis’s goal is a lofty one: to have a working exoskeleton suit ready for the 2014 World Cup in Brazil, where a quadriplegic Brazilian will deliver the opening kick. He told me proudly, “This is our Brazilian moon shot.”
AVATARS AND SURROGATES
In the movie Surrogates, Bruce Willis plays an FBI agent who is investigating mysterious murders. Scientists have created exoskeletons so perfect that they exceed human capabilities. These mechanical creatures are super strong, with perfect bodies. In fact, they are so perfect that humanity has become dependent on them. People live their entire life in pods, mentally controlling their handsome, beautiful surrogate with wireless technology. Everywhere you go, you see busy “people” at work, except they are all perfectly shaped surrogates. Their aging masters are conveniently hidden from view. The plot takes a sharp twist, however, when Bruce Willis discovers that the person behind these murders might be linked to the same scientist who invented these surrogates in the first place. That forces him to wonder whether the surrogates are a blessing or a curse.
And in the blockbuster movie Avatar, in the year 2154 Earth has depleted most of its minerals, so a mining company has journeyed to a distant moon called Pandora in the Alpha Centauri star system in search of a rare metal, unobtanium. There are native people who inhabit this distant moon, called the Na’vi, who live in harmony with their lush environment. In order to communicate with the native people, specially trained workers are placed in pods, where they learn to mentally control the body of a genetically engineered native. Although the atmosphere is poisonous and the environment differs radically from Earth’s, avatars have no difficulty living in this alien world. This uneasy relationship, however, soon collapses when the mining company finds a rich deposit of unobtainium underneath the Na’vi’s sacred ceremonial tree. Inevitably a conflict arises between the mining company, which wants to destroy the sacred tree and strip-mine the land for its rare metal, and the natives, who worship it. It looks like a lost cause for the natives until one of the specially trained workers switches sides and leads the Na’vi to victory.