The actress asked the same question again, and R781 gave the same answer as before, but phrased differently. Robots were programmed to be aware that humans often missed an answer the first time it was given, and they should reply each time in different words. If the same words were repeated, the human was likely to get angry.
A caller-in asked, “When you simulated loving Travis, why didn’t you consider Travis’s long-term welfare and figure out how to put him in a family that would make sure he got a good education?”
R781 replied that when a robot was instructed in a metaphorical way, as in “love the fucking baby yourself,” it was programmed to interpret the command in the narrowest reasonable context.
After the show, the Anti-Robot League got $281 million in donations, but Give-Robots-Personalities got $453 million. Apparently, many people found it boring that robots had no desires of their own.
Child Welfare demanded that Ms. Rambo undergo six weeks of addiction rehabilitation and three weeks of child-care training. Her lawyer persuaded her to agree to that.
There was a small fuss between the mother and Robot Central. She and her lawyer demanded a new robot, whereas Robot Central pointed out that a new robot would have exactly the same program. Eventually Robot Central gave in and they sent her another GenRob337L3 robot in a different color.
Ms. Rambo really was very attractive when cleaned up and detoxified, and the lawyer married her. They took back custody of Travis. It would be a considerable exaggeration to say they lived happily ever after, but they did eventually have three children of their own. All four children survived the educational system.
After several requests, Robot Central donated R781 to the Smithsonian Institution. It is one of the stars of the robot section of the museum. As part of a twenty-minute show repeated every half hour, R781 clothes itself as it was at the time of its adventure with the baby, and answers the visitors’ questions, all while speaking motherese. Mothers sometimes like to have their pictures taken standing next to R781 with the robot holding their baby. After many requests, R781 was told to modify its program to allow this.
R781 then plays a movie that was patched together from the surveillance cameras that recorded the street scene. Through the magic of modern audio systems, children don’t hear the bad language that was spoken, and women audience members can only hear it if they assure R781 that they are not ladies.
The incident of the robot and the baby increased the demand for actual child-care robots, which were legalized five years later. The consequences were pretty much what the opponents had feared: many children grew up more attached to their robot nannies than to their actual parents. This outcome was mitigated somewhat by making the robot nannies quite severe, and offering parents free coursework on how to compete for their children’s love.
Sometimes this worked.
SEANAN McGUIRE
WE ARE ALL MISFIT TOYS IN THE AFTERMATH OF THE VELVETEEN WAR
Seanan McGuire is the author of many works of short fiction and two ongoing urban fantasy series. Under the name Mira Grant, she writes science fiction thrillers full of viruses and zombies. Between her identities, she is a ten-time Hugo Award finalist, and was the winner of the 2010 John W. Campbell Award for Best New Writer. She is a founding member of the Hugo Award–winning SF Squeecast. She currently resides on the West Coast, where she shares her home with three enormous blue cats, a great many books, and the occasional wayward rattlesnake. McGuire regularly claims to be the advance scout of a race of alien plant people. We have no good reason to doubt her.
HAVE YOU SEEN THIS GIRL?
—posted on a telephone pole in Lafayette, California.
Half a dozen cars cluster behind the old community center like birds on a telephone wire, crammed so closely together that someone will probably scrape someone else’s paint on their way out of the parking lot. It would have been easy to leave a little room, but that’s not how we do things anymore. Safety means sticking close, risking a few bruises in order to avoid the bigger injuries.
It’s silly. The war is over—the war has been over for more than three years, receding a little further into the past with every day that inches by—and we’re still behaving like it could resume at any time. It’s silly, and it’s pointless, and I still veer at the last moment, abandoning my comfortably distant parking space in favor of one that leaves my car next to all the others. I have to squirm to get out of the driver’s seat, forcing my body through a gap that’s barely as wide as I am.
Something moves in the shadow between the nearest Dumpster and the street. It’s probably a feral cat, but my heart leaps into my throat, and I hold my coat tight around my body as I turn and race for the door. The war is over.
The war will never end.
Almost twenty people arrived in those half-dozen cars: gas is expensive and solitude is suspect, and so carpooling has become a way of life. I am the only person who comes to these meetings alone. They forgive me because they might need me someday, and because sometimes I bring coffee for the refreshment table. Not today, though. It was a rough night at work, and I feel their eyes on me, accusing, as I make my way to one of the open folding chairs. Like the cars, the chairs are set too close together, so that we can smell each other’s sweat, feel the heat coming off each other’s skins.
Precaution after precaution, and the war is over, and the war will never end.
“So glad you could join us,” says the government mediator, and there’s a condescending sweetness in her tone that shouldn’t be there. She knows why I’m late; she knows I didn’t have a choice in the matter. She’s just asserting dominance, and no one in this room will challenge her.
I swallow fear like a bitter tonic as I drop into a chair. “I got turned around,” I say. “There was a new barricade on Elm, and I don’t know that neighborhood very well.” It’s harder to get around since most of the GPS satellites were decommissioned. They never turned against us—thank God for small favors—but data doesn’t care who or what uses it, and some of the people in charge decided that it was better for a few civilians to get lost than it was to risk one of those satellites being taken over. I can’t say whether that was the right decision or not. We never lost a GPS satellite. Maybe we never would have. Maybe we would have lost them all. The war is over.
The war will never end.
It doesn’t matter.
“Now that we’re all here, we can begin,” says the government mediator. Her smile is formal, practiced, and as plastic as our enemies.
They all come from FEMA, the mediators, trained in crisis response and recovery. They’re just doing their jobs. I tell myself that every time they send us a new mediator, another interchangeable man or woman sitting in a splintery wooden chair, trying to talk us through a trauma that we cannot, will not, will never get past. When they start to care—when we become people, not statistics—that’s when they’re rotated again, one face blurring into the next. The country is too wounded for personal compassion. The world is too wounded. The good of the one is no longer a part of the equation.
“My name is Carl,” says one of the men, and we all chorus, “Welcome, Carl,” as obedient as schoolchildren. Carl doesn’t seem to find comfort in our greeting. Carl’s eyes are as empty as the mediator’s smile. Carl doesn’t want to be here.
That’s something we have in common.
“Did you want to share?” asks the mediator, even though she damn well knows the answer. We’re here because we have to be; we’re here because we want to share our stories, to hear the stories of others, and to sift through the patchwork scraps of information looking for the thing we need more than anything else in the world: hope. We’re hunting for hope, and this is the only place we know of where it’s been spotted.
Carl nods, worrying his lip between his teeth before he says haltingly, “My Jimmy will be nine years old next week. The last time I saw him, he had just turned six …” And just like that, he’s off, the words tumbling like stones from his li
ps. The rest of us listen in silence. My hands are locked together, so tight that my fingers are starting to hurt.
The war is over, and Carl is telling us about the son he lost when the war began, and nothing really matters anymore. Nothing will ever matter again.
This is what happened.
Artificial intelligence became feasible ten years ago, when a San Jose social media firm working on building the perfect predictive algorithm somehow unlocked the final step between a simple machine and a computer that was capable of active learning. Self-teaching machines were the future, and humanity was terrified. We were proud of our position at the peak of the social order, and we feared creating our own successors. Making matters worse, every country was afraid of how every other country would use this new technology. We were convinced that AI would allow its users to dominate the others in war or commerce.
In less than a month, artificial intelligence was more tightly regulated than stem cell research. In less than a year, it was outlawed in virtually all fields of human endeavor. But once a genie is out of the bottle, it can’t be put back in, and we couldn’t render an entire technology illegal. In the end, there was only one area where everyone agreed the self-teaching programs could be freely used:
Education.
That seems careless now, in the harsh light of hindsight, but at the time, it seemed like a perfectly reasonable compromise. Dolls that could learn the names of their owners had been around for years. Letting them learn a little more couldn’t possibly hurt anything—and toys had no offensive capabilities, toys couldn’t get online and disrupt the natural order of things, toys were safe. We all grew up with toys. We knew them and we loved them. Toys would never hurt us.
We forgot that kids can play rough; we forgot that sometimes, we hurt our toys without meaning to. We forgot that by giving toys the capacity to learn and teach, we might also be giving them the capacity to decide that they were tired of being treated like their thoughts and desires—their feelings—didn’t matter. We made them empathic and intelligent and handed them to our children, and we didn’t think anything could possibly happen.
We were wrong.
Carl covers his face with his hands as his story ends, crying silently into his palms. No one reaches out to comfort him. It’s been so long that I don’t think any of us remembers how comforting is supposed to go. We sit frozen, like so many life-sized dolls, and wait for the woman from FEMA to tell us what she wants us to do next.
Her eyes scan the crowd like a hawk’s, intent and cool, picking through our faces as she searches out our secrets. Who’s ready to speak, who needs to speak, even if they don’t realize it. When she looks at me, I shake my head minutely, willing her away. My work at the hospital makes me valuable—there are so few doctors left who will even look at children, much less treat them—and so she respects my silence, moving on to her next target.
“Would you like to share?” she asks a woman I don’t recognize. That’s another FEMA trick: make the support groups mandatory, and then shift us from location to location, preventing us from forming individual bonds, encouraging us to form broader societal ones. Half the group is new to me. By the time they become familiar, the other half will change, people driving or busing in from all sides of the city. That assumes that I won’t be reassigned before that happens, although my job keeps me tethered to a smaller geographic range than most. If a child is brought to the hospital, I will be needed. I can never go too far away.
The woman—dark skin, dark eyes, and the same broken, empty sadness that I see in so many adult faces since the war—nods and introduces herself, beginning to speak. Her voice is halting, like every word has to be dragged out of her by someone invisible, some little girl or boy just outside the range of vision. She’s telling their story. She’s telling our story, and forgive me, Emily, but I can’t listen. I block out her words like I’ve blocked out so many others, because you can only hear certain things so many times before they start to burn.
The war is over.
The war will never, ever end.
As a pediatrician, I was involved with some of the earliest studies of the self-teaching toys. Were they good for children? Were they a socialization tool, a way of reaching out to kids who might not have anyone else to talk to? We prescribed them to autistic children as “safe” companions, supporters that would never judge or leave them. Then we prescribed them to socially awkward children as friends, to hyperactive children as relatable voices of reason, and finally to absolutely everyone. Self-teaching toys were the perfect gift.
Better yet, no matter what they were built to resemble—the requisite soldiers and princesses, as well as the more gender-neutral teddy bears, with their black button eyes and red velvet bows—they would fit themselves to the children, not to the stereotypes of the parents. Quiet or loud, gentle or boisterous, each child could find their perfect playmate in a self-teaching toy.
The recreational models cost more than most parents were willing to pay, of course, at least in the beginning. As the technology saturated more and more of the market, the prices dropped, until it was harder to buy a doll or bear that didn’t actively participate in playtime than one that did. There were even charities and nonprofit organizations dedicated to getting the toys into the hands of low-income families. Every house had at least one self-teaching toy. Many of them had more. And the toys learned! Oh, how they learned. They learned our children. They learned us. In the end, they learned themselves, and that was where the troubles truly began.
We weren’t prepared for toys asking questions of identity. “Who am I?” is not a question that anyone expects from the pretty painted mouth of a fashion doll. “Why am I here?” is foreign in the lipless muzzle of a teddy bear. But they asked, and we tried to answer, and all the while, we were growing more nervous. Had we built our toys too well? Was it time to somehow pull the plug on a technology that had spread so far as to become unavoidable? We had kept the artificial intelligence out of our military and our social infrastructure. In so doing, we had invited it into our homes, and allowed it to flourish where we were most vulnerable.
We built the toys to learn. We didn’t expect them to learn so well—or maybe we didn’t expect our children to be such good teachers.
So many of them were designed to interact with apps and online games; so many of them knew how to access wireless networks, and the ones who couldn’t connect listened to those who could, and they talked. How they talked! They whispered and they gossiped and they planned, and somehow, we missed it. Somehow, we were oblivious. They were only toys, after all. What could they possibly do to us, their creators, that would make any difference at all?
We were fools. And in the span of a single night, we became fools at war.
Half the room has told their stories, with halting voices forcing their way through well-worn memories of sons and daughters three years gone, but never to be forgotten. One man lost four children on the night the war began. His wife committed suicide a week later, convinced that she was somehow the one to blame. His face is empty, like a broken window looking in on an abandoned house, and he never meets anyone’s eyes. Another woman had undergone five years of fertility treatments, only to have her single miracle child—the only thing she had ever truly wanted in her life—vanish on the first night of the war. I don’t know if her missing child is a son or daughter. I don’t ask.
The woman from FEMA is looking for another victim when my pager beeps. Everyone jumps a little, all eyes going to me. “Sorry,” I say, although I don’t really mean it, and stand before I check the readout on the screen. I know it’s an emergency. They only call me during my government-mandated support group when it’s an emergency. What kind of emergency doesn’t really matter. “I need to get back to the hospital. Sorry.”
“We understand,” says the woman from FEMA, and she does—she even looks a little sympathetic. My job and hers aren’t that different, except that I don’t get to leave this community, don’t get to tran
sfer every time I get attached.
For a moment, I want to ask if she ever had children, if she was a mother before the night when the toys decided that they had to do something. I don’t know how to ask the question. “Do you have children?” has become the profanity of our generation. So I don’t ask her anything at all. I just turn on my heel and walk out of the room, leaving the stories and the sharing and the broken eyes so much like mine behind me.
The war is over. The war has been over for three years. The war will never, ever end.
The hospital parking lot mirrors the community center to an eerie degree. All the spaces toward the front are taken; some cars have been parked in the lane rather than their owners taking the risk of winding up farther away. Thankfully, the reserved spaces for the hospital staff are closest to the doors. I’m outside for less than thirty seconds. It’s more than long enough to make my blood run cold with fear.
The orderly at the door nods to me as I rush by him, heading toward the emergency room. It’s a code 339, the worst kind of emergency: a child. A returned child. Still breathing when it was found, or they’d never have called me … but that’s no guarantee.
That’s no guarantee of anything, because the war is over, and the war will never end.
The sound and chaos of the emergency room reaches out its arms like a lover as I step through the final set of swinging doors. It wraps them tight around me, blocking the last of my emotional rawness away. This is a job. This is my job. This is the thing I do best in all the world. I can’t let anything make me forget that.
People step aside when they see me coming, relief and guilt written plainly on their faces. It must be a bad one, then. I force myself to keep walking, and it’s not until I turn the last corner that the thought I’ve been trying to avoid comes lancing across my mind: