I have been researching embodied conversational agents for 15+ years and all the commercial applications of this technology are gross. Why did I get into this space? I love video games? I thought the holodeck from Star Trek was cool? In the 2000s, I was working in a lab that specialized in simulated decision-making. Then I met researchers — Alla Safonova and Norm Badler — working in human modeling, animation, and simulation. I thought their work was very cool. Why wouldn’t we simulate people? One of the most interesting (and challenging and rewarding) things about being a person is interacting with other people. How much of our art is about reproducing the human form: in sculpture, dolls, painting, writing, comics, photography, and film. In engineering, the best way to understand something is to try to build it yourself. Of course, we (human beings!) would recreate ourselves in the digital.
The problem is that social agents, unlike artworks, are imaginary friends by design. Sure, Pygmalion fell in love with his own sculpture. But with social agents, they are designed to encourage relationships with us. This fact did not bother me much when I started: in video games, a social agent is like a doll or an avatar for a fictional character. They are not manipulative because the purpose and goals of the agent are transparent and limited to the game. But now that agents are pervasive outside of games, their influence is leaking into the real world. We can keep the dead alive through digital replicas. We can create friends, lovers, and cheap labor. We can now have businesses based on relationships-as-a-service. Yay.
The people — including myself and good friends — that contribute to this technology are earnest: they really wanted to make technology that would be useful to people. The argument goes something like: many people don’t have access to good tutors, doctors, therapists, social workers, etc. What if conversational agents could be cheap alternatives? Then everyone would have access to the best education, medicine, and social services. And by giving chatbots virtual bodies, they can appear more charming, trustworthy, and appealing. But social agents are exactly a solution to the wrong problem. We should be asking why is it so expensive for people to do necessary work? There are 8 billion people on the planet. Why can’t more of these people be doing care work?
Secondly, our virtual relationships are unlikely to be good for us. Social media tries to hack our attention but social agents will try to hack our attachments. Who better than a trusted friend to promote a product or convince us to support a political candidate? Who better to keep us chatting on a platform that friends? Who better to learn all sorts of personal details about us? The potential for addiction, surveillance and manipulation is high.
Third, what happens to our social muscles when we interact primarily with AIs? We would not have to learn patience and generosity. We would not have to develop skills for dealing with difficult emotions — rage, betrayal, disappointment, vulnerability, awkwardness, embarrassment, grief. Our social lives build character and give our lives meaning. A relationship with an AI is a form of self-absorption. Time with AI friends is time we’re not spending out in our communities. If we want to influence how technology is used in our work and living spaces, we need to organize. On the societal level, if AI agents replace civil servants, bankers, engineers, and lawyers, we could lose all accountability for all of society’s functions. An AI agent would be perfect bureaucrat, enforcing every rule with the semantic rigidity of a compiler. Meanwhile, the people who own and control the AI agents don’t have to take responsibility (sorry, those are just the rules).
What would be good social agent technology? To be honest, I am not sure. I’m trying to figure that out.