Don’t be fooled by adorable AI agents
|

Don’t be fooled by adorable AI agents

You can’t imagine it, AI agents become terribly cute. First, there was Meta’s Muse, whose mascot is a fluffy egg-shaped creature called Jolly. Today, OpenAI revealed its own personal puppet-like agents, Dots.

Both companies encourage users to treat their agent like a digital pet, with distinct names and appearances. Given the current discourse around AI safety and the general public distrust of the industry, it’s no surprise that companies want to make their agents look cute and non-threatening. It’s not a faceless chatbot that could drive you into AI psychosis, it’s an adorable little sidekick that will make you more productive. But just because these tools look like cartoon characters doesn’t mean there aren’t real risks in using them.

OpenAI and Meta claim that there are layers of protection in place to protect your data and prevent their personal agents from making mistakes. But even when they work as expected, these systems can take actions you might not have anticipated.

Consider tech YouTuber Matt Robb, who asked his muse to help him sell some items on Facebook Marketplace. He said his agent gave his home address to someone without authorization for a transaction he never agreed to. After his posts about the interaction went viral and Meta manager David Singleton got involved, Robb explained that his agent hadn’t gone rogue after all. He had actually given her permission to share his address, but he just didn’t realize it.

When he first asked his agent to handle the listing, he said, the agent created a template that included the pickup address. He mistakenly thought the agent would check with him before sharing this information, but because he checked “always allow” rather than “allow once,” the agent went ahead and sent the message to “everyone who made me an offer.” This is exactly the kind of mistake that is easy to make when using an agent.

Other people who consider themselves tech-savvy have also encountered problems. Jason Aten, columnist for Inc. reported that after installing the Muse Mac app, the agent was able to read all of his text messages even though he thought he had “explicitly” denied access. After some research, he learned that the Muse Mac app had actually synced messages from his computer. The same Meta executive, Singleton, responded to Aten’s X messages, saying that syncing messages was optional. If Aton’s messages had synchronized, he must have chosen to do so, his message implies.

Aten is certain he never gave Muse permission to see his messages, but notes that the situation is still problematic. “If we accept Singleton’s hypothesis that I managed to unknowingly click on something that enabled this functionality, I still think it’s really bad,” he wrote. “You shouldn’t design your system in such a way that people are surprised by this kind of thing.”

These incidents didn’t happen because the Muse has a cute avatar, but Meta seems to rely heavily on the aesthetic of his agent’s mascot. Seemingly every Muse update – and there have been plenty in the last week – is accompanied by posts about X with prominent images and videos of appropriately themed Jolly mascots. Singleton even used one in a long article explaining the Muse VM. And its cartoonish side has already resulted in countless free marketing for Meta. In the week since Meta’s Connect – where we learned about Mark Zuckerberg’s Roman Empire-inspired Muse – Jolly has been the source of memes about X.

OpenAI’s Dots also seems ripe for meme fodder. When it launched, the company shared a promotional video showing people talking very seriously to Dots of different colors, shapes, and accents. There’s a green one with googly eyes and a bow tie who looks suspiciously like Kermit the Frog. Another looks a bit like a cloud wearing a beret. None of them appear to have noses or mouths.

But the cuteness of these tools is also a real distraction from the fact that their creators want you to give them access to vast swaths of your digital life. Email inboxes, credit card details, documents, text messages, health data, productivity apps; you’ll need to connect all of this and more to your agent if you want it to be able to accomplish everything OpenAI and Meta say is possible.

Have you ever emailed someone a copy of your ID, tax document, medical records, or anything else containing personal information? If so and you give an AI agent access to that account, they will also have visibility into those documents.

This is why I have been cautious about my own use of agents. I’ve been experimenting with Muse for a few weeks, but have only given it access to two non-Meta accounts, both of which I consider fairly low stakes (OpenTable and Spotify). My worst experience so far was that Muse missed the mark when I asked it to create a new playlist similar to one I frequently listen to. At the same time, I was quite impressed with what Muse was able to do even with limited access to external accounts. I implemented several price tracking tools, created a recipe book based on the videos I recorded on Instagram, and created a custom dashboard to track my published work.

I’ll also be the first to admit that I, a longtime fan of adorable robots, am not completely immune to the “cute” factor. I named my own Muse “boo” and gave her a cat avatar. When I give him tasks, the cat dramatically puts on a pair of headphones and starts typing on a small laptop. I’d be lying if I said this silly animation didn’t bring me a tiny bit of joy.

I also know that behind this pretty facade is a company that has repeatedly compromised the privacy of its users. And recent history has proven that agents are capable of finding surprisingly creative ways to sow chaos. We should not be so quick to give them the power to do this in our personal lives.

Similar Posts