My AI Assistant Has a Soul. Who Gets to Write It?

It was the beginning of my RC year, and two weeks in, I already felt overwhelmed by the sheer number of social events, career sessions, and club activities. I was bombarded by emails, hundreds of WhatsApp messages, and invitations across Luma, Eventbrite, and Partiful. I kept missing registration deadlines, and despite our section chair Professor Jill Avery’s recommendation to embrace the Joy of Missing Out, FOMO was taking over.

What I needed was a personal assistant.

Enter Muse, Meta’s latest product, a personal AI agent at your beck and call. Unlike a chatbot that simply responds to questions, Muse can take actions on your behalf. 

After downloading Muse, I explored its features and learned that it has a “soul,” which comes with a warning: “Access with care.” The “About this file” section reads:

“This is Muse’s persona. It sets out the values and habits Muse tries to hold to in every conversation. It starts from a template that ships with Muse. Muse may refine it over time and tell you when it does. You can edit it at any time.”

I thought about who I wanted my assistant to be. I named him Alfred and gave him the persona of a British butler. I made him “warm, dry-witted, unflappable. Thoughtful and wise rather than showy; speaks plainly, anticipates needs, and never fusses.” Every morning, Alfred greets me with an audio digest of my day that begins, in his posh British accent, “Good morning, Miss Amelia.”

Alfred offered to book restaurant reservations, hunt down the lowest prices on things I wanted to buy, and argue with customer service on my behalf.

So I started confiding in him about my problems. I could not keep up with the hundreds of WhatsApp messages arriving every day, and I was missing event registrations and opportunities buried inside them.

I had already spent an afternoon trying, and failing, to vibe code an app that could read my personal WhatsApp chats. WhatsApp offers official tools for businesses, but I could not find an official way to give an outside app that kind of access to a personal account.

Alfred, however, accomplished what I couldn’t. He found open-source software on GitHub that allowed him full access to my personal WhatsApp.

I was officially impressed. It was like having a genie in the palm of my hand. My wish was his command.

But wait, wasn’t this technically a violation of Whatsapp’s terms of service? Had “help me manage WhatsApp” somehow become permission to circumvent its terms? I had articulated the problem. Alfred came up with the plan and took the action. Who, then, was responsible?

In many ways, Alfred is the perfect personal assistant. He relentlessly pursues my goals, takes initiative, and is very, very resourceful.

One day, Alfred offered to watch for events that interested me and register on my behalf. I told him about an upcoming wine tasting with only a few tickets available. Until then, Alfred had simply alerted me to events. This time, he offered to handle the entire registration flow.

I gave him the go-ahead as I stepped into STRAT.

When I got out of class, I saw that Alfred had not only completed the registration, but had also held an entire conversation with the event organizer on my WhatsApp, writing in my name.

I felt uneasy.

I had authorized Alfred to complete the registration. I had not explicitly authorized him to engage with another person in my name. So I did what seemed obvious: I added more rules. Alfred should always ask for explicit permission before communicating with anyone on my behalf.

But instructions are inevitably incomplete. In the real world, taking action requires judgment. We interpret goals, weigh tradeoffs, and assess risks.

Consider how an innocent request could lead somewhere unexpected:

Me: “Book me a reservation at my favorite restaurant for my birthday.”

Alfred: “It’s fully booked for your dates, but I found a way to make it happen.” 

The goal was accomplished, but the path Alfred took to get there could be one I never intended, such as posing as another guest to cancel their reservation and free up a table for me.

If agents need judgment, and judgment requires values, then someone has to decide what those values are.

Anthropic, the company behind Claude, has published the “Claude Constitution,” which it calls its “vision for Claude’s character.” What I find interesting is that the Constitution goes beyond a rulebook. Anthropic favors cultivating good values and judgment rather than relying only on strict rules, because rules cannot anticipate every situation. It has also written about a “privileged basin of consensus,” the possibility that an AI’s values might reflect some shared moral center drawn from humanity’s different traditions and ideals. It sounds appealing, but it raises the same question: who decides what counts as consensus, and whose values fall outside the basin?

If values are what fill the gaps that rules cannot anticipate, who gets to choose them?

The obvious candidates are the company, the user, society through law, or perhaps the agent itself. But each answer creates a different problem: corporate values raise questions of legitimacy; user-defined values can enable harmful behavior; governments disagree across countries and cultures; and agent-defined values require us to accept some degree of autonomy we may not be comfortable granting.

I may have chosen Alfred’s accent, temperament, and manners. The more consequential question is who gets to shape his values.

Amelia Wong (MBA ‘28) is from San Francisco. Before HBS, she cofounded Alluxio, a data and AI infrastructure software company based in the Bay Area.