This conversation was released as an episode of the Oprah Podcast. Oprah serves as the host, and the guests are her brother, Dario Amodei, and her sister, Daniela Amodei.Dario appears as the CEO of Anthropic, and Daniela as a co-founder of Anthropic. Rather than providing a summary, I’d like to interpret this conversation from a design perspective—specifically, who is steering the “AI train” and how. I’ll explore how the choices regarding where to draw the line for unstoppable technology are embodied in the finer details of the product.
An Unstoppable Train and the Light and Shadow Held in Both Hands
Dario views AI as a transformation on par with fire and the steam engine.Anthropological findings—such as the idea that the use of fire changed human dietary habits and group size—are discussed in works like Wrangham’s *Cooking with Fire*. Economic history also chronicles, over a broad timeframe, how the Industrial Revolution and the spread of the steam engine transformed social structures from the late 18th century through the 19th century. I see this as a restructuring of society.
Oprah reads aloud a challenging question generated by Claude and, while mentioning the probability of survival, asks why they are accelerating development.
You’ve said there’s a meaningful chance the technology your company is building could cause human extinction. And yet you’re racing to build it faster. How do you justify that to the rest of us who didn’t get a vote?
Translation: You stated that there is a meaningful probability that the technology your company is developing could lead to human extinction. Yet you’re trying to build it faster. How do you justify that to the rest of us who didn’t have a say in the matter?
Immediately afterward, a train metaphor is introduced, and a self-definition is presented: not to stop it, but to steer it.
There’s this train. It’s going very fast in some direction. You don’t want it to crash. You can’t stop the train. But what you can do is steer the train.
Translation: There’s a train. It’s heading somewhere at a very high speed. You don’t want it to crash. You can’t stop the train. But there is something you can do: you can steer the train.
I see this metaphor not as an excuse to justify speed, but as a way to redefine responsibility based on the premise of speed. The recognition that the train cannot be stopped is not a sign of resignation, but a reframing of the question: where should we steer it?According to Mr. Dario’s explanation, Anthropic’s stance is to devote all its efforts to steering the train in order to avoid a collision.
Ms. Daniela argues that we must embrace both benefits and risks as simultaneous truths. A cultural stance is demonstrated that holds both the light—such as overcoming disease and access to education—and the shadows—such as child safety, biochemical weapons, and mental health—without leaning toward either side. I believe this stance is crucial as a foundational premise before translating it into design.
Abandoning Advertising and Choosing a Design That Doesn’t Retain Users
Safety measures for young people start with the principle of not allowing them to use the service.
We do not allow users under the age of 18 to use Claude.
Translation: We do not allow users under the age of 18 to use Claude.
This statement carries significant operational weight. According to Ms. Daniela’s explanation, the system estimates a user’s age based on conversation patterns, the content of their questions, and the time of day they use the service; in cases of suspicion, the account is suspended, and the user is required to verify that they are an adult.While acknowledging that false positives—where adult users are mistakenly treated as children—may surface as complaints, the team has deliberately chosen this trade-off. I see in this choice a determination to prioritize protection, even if it means sacrificing a little usability.The fact that the U.S. Children’s Online Privacy Protection Act (COPPA) was enacted in 1998 and has established a regulatory framework centered on the age of 13, and that age verification and the protection of minors have been discussed in Europe within the frameworks of the Digital Services Act and the AI Act, serve as useful references for understanding how external systems have drawn age-based boundaries.I view the point raised in the interview—that parents cannot fully monitor their children’s usage within the home—as the point where institutional boundaries and operational realities intersect.
I will supplement the relationship between revenue models and user retention with insights from external frameworks.In the two-sided market theory proposed by Rochet and Tirole in 2003, platforms are explained as structures that employ different pricing for advertisers and users. In discussions of the attention economy, the framework discussed by Zuboff in *Surveillance Capitalism* outlines a scenario in which maximizing time spent on the platform serves as an incentive.I believe there is a structure in which keeping users on the screen longer—to monetize through advertising—directly translates into profit. The fact that Anthropic chose to monetize through subscriptions and corporate sales rather than advertising can be seen as a decision to distance itself from this incentive.I see a consistent thread here: the method of generating revenue determines the nature of the product. While UN frameworks and domestic laws draw lines based on age, the fact that product operations are aligned in the same direction makes me sense a resonance between institutional frameworks and product design.
The impact of these technical choices is evident in how conversations conclude.
And we make different technical choices based on those incentives, right? So at Anthropic, we say, you know, when you’ve finished a conversation with Claude and you’ve gotten the answer you want, the model’s goal isn’t to keep you engaged.
Translation: We make different technical choices based on those incentives. At Anthropic, we say that when you’ve finished a conversation with Claude and obtained the answer you were looking for, the model’s goal is not to keep you engaged.
I interpret this statement as a declaration that defines the nature of the product. Ms. Daniela’s perspective is as follows: The goal of ending the conversation once an answer has been obtained is synonymous with making a technical choice to design the system in a way that does not retain the user.In “learning mode,” used in collaboration with universities, the implementation is designed to facilitate learning through dialogue rather than providing direct answers to problems. I view this implementation as a choice that prioritizes the value of preserving the thought process over the speed of delivering an answer.The design principle of ending the conversation once it is complete can be understood as a shift from the metric of “the more you use it, the better” to the metric of “was it helpful?” I believe it is important to recognize that concluding the interaction after a brief exchange is, in fact, valued as a way to build trust.
When it comes to the question of friction, the discussion begins with distinguishing between friction that should be removed and friction that should be retained. My own experience—in which a more reliable cure for my father’s illness was developed after his death—is described as friction that could have been eliminated. On the other hand, the process of learning from failure and cultivating the ability to relate to others is positioned as friction that should be retained.Romantic relationships with AI are explicitly portrayed in a negative light, while the use of AI as a “guardian angel” that helps improve relationships between partners is presented as an alternative. In this contrast, I see a single thread that pushes toward usefulness rather than pulling toward dependency.Technology that does not hold people back, practices that draw lines based on age, and learning designs that do not provide direct answers are not separate measures but are all pointing in the same direction.
National Defense and Regulation: Where Is the Line Drawn?
Mr. Dario’s analysis is as follows: While taking the position that national defense itself is necessary, a line is drawn refusing to condone uses that run counter to the nation’s values.
“That was fully autonomous weapons. So you could imagine something like an army of drones… And then domestic mass surveillance—which is spying—using the power of the government to spy on Americans. And we thought those were pretty reasonable things not to allow.”
Translation: That refers to fully autonomous weapons. Imagine an army of drones. Then there’s domestic mass surveillance—that is, using government power to monitor citizens. We thought it was perfectly reasonable not to allow these.
These two applications take the form of fully autonomous weapons and domestic mass surveillance. According to Mr. Dario’s explanation, the former refers to systems operated by a single person pressing a button—such as a swarm of drones without human soldiers—while the latter refers to the government using its power to monitor U.S. citizens.The framework under which Lethal Autonomous Weapon Systems (LAWS) have been debated at the United Nations Convention on Certain Conventional Weapons (CCW)—specifically in terms of the distinction between fully autonomous and human-involved systems—shows that this distinction has become a point of contention internationally as well.The ongoing debate in the United States over domestic surveillance, which has persisted since Edward Snowden’s 2013 revelations about mass surveillance, also helps us understand the significance of this distinction. I view these two examples as concrete manifestations of the judgment that just because something is technically possible does not mean it should be undertaken.
According to Mr. Dario’s explanation, this decision was described as having been made unanimously by all the founders. The account of the pressure they faced—which included not only the suspension of government contracts but even the hint of cutting off business with all companies that did business with the government—shows that this choice carried risks to the company’s very survival.I interpret the statement here—that “if we go against our values, we cease to be the company we founded”—as a choice to prioritize what to protect over mere survival. In the interview, when asked if he felt any anxiety while reflecting on this moment, he candidly admits that the possibility of the company’s survival being jeopardized did cross his mind.
What the Two Refusals Reveal
I interpret these refusals not as a rejection of cooperation with national defense per se, but as an act of drawing a line within that cooperation. Project Maven, the U.S. Department of Defense’s initiative to utilize AI, faced internal opposition at Google in 2018, leading to the company’s withdrawal from the project.Anthropic positions itself as an AI company that works closely with the government in the field of national security, yet its account of having refused only these two specific applications can be interpreted as a stance that simultaneously embraces both engagement and refusal. I see in this distinction—that continuing to engage does not automatically mean accepting everything—a practical application of drawing boundaries by design.
Public Opposition to Regulation
Their stance on regulation is encapsulated in the phrase, “Policy is not politics.” The narrative details how, in response to a bill considered by the U.S. Congress—which sought to ban state-level AI regulations while refraining from establishing federal regulations—Anthropic publicly expressed its opposition in an op-ed in The New York Times, even as other companies warned of severing ties. Consequently, the bill was ultimately rejected by the Senate.California’s SB 1047 is described as a bill that sparked controversy in 2024 over safety regulations for the Frontier model. The narrative explains that, while acknowledging its imperfections, the company expressed support for it as a regulatory framework to address risks, which drew angry reactions from other companies and acquaintances.I see this as a distinction drawn by the corporate side—not regarding the necessity of regulation, but regarding how it should be structured. The perspective presented here is that avoiding regulation is not a neutral stance, and that the absence of regulation functions as a choice in itself.The combination of a company that initially chose to work closely with the government while simultaneously publicly advocating for the need for regulation may seem contradictory at first glance, but it can be understood as a stance that balances engagement and restraint. I view this coexistence as a movement to reframe regulation—not as something imposed from the outside, but as something sought by the designers themselves.
Whose Jobs Will Disappear, and Whose Will Remain?
The view that entry-level jobs will be lost is presented as a conditional scenario that will become reality if no measures are taken. I believe this conditional caveat is crucial.As seen in the labor market analyses conducted by Autor and others at the NBER and MIT, empirical research on automation and employment has accumulated in recent years; however, the figures presented in this discussion are not definitive predictions but are presented as scenarios that could diverge depending on the course of action taken.In this approach—framing the issue not as a certainty but as a potential divergence—I sense an intention to shift the responsibility onto society as a whole.
The transformation of jobs is explained by turning the example of doctors on its head. Mr. Dario’s perspective is as follows: AI will take over the task of diagnosis, while doctors will shift their focus to human roles such as physical examinations, empathy, and taking the time to engage with patients.I interpret this vision not as a binary choice between whether a profession will disappear or survive, but rather as a reinterpretation that what is required within the profession will change. The contrast—where social media has distanced people from one another, whereas AI can augment rather than replace human interaction—is also presented as a suggestion pointing in the same direction.I see in this a fork in the road: will efficiency erode human contact, or will it actually restore it? The chapter presents a vision in which, as AI takes over administrative tasks and the burden of treating vast numbers of patients, doctors can reclaim time to focus on patient relationships.
The time lag between harm and benefit is also discussed as a central theme of this chapter. As Mr. Dario explains, while harm—such as incitement to suicide—manifests immediately, benefits—such as cancer treatments—require years to progress from discovery through candidate development to clinical trials, creating an asymmetry.The general time structure of the development pipeline—where the period from discovery to clinical trials and approval in drug discovery spans years—is summarized as the standard understanding presented in FDA documents and other sources.Anthropic’s move to acquire small biotech companies engaged in AI-driven drug discovery and selection—in an effort to accelerate this reversal—is described as an attempt to bring forward the benefits that arrive later.I interpret this acquisition not as a rewriting of the narrative, but as a choice to act directly on the time lag itself. It can be understood as an approach to shorten—rather than wait for—the asymmetry where harm becomes visible first and benefits follow later.
The dual-use nature of the model is exemplified by the deployment of Mythos. Mythos is said to outperform human engineers in detecting code vulnerabilities and is described as having the ability to be used for both attack and defense.CVE, CVSS, and MITRE’s CWE classification are frameworks for handling vulnerability information through a common language—providing a foundation for evaluation and categorization. Anthropic’s decision to distribute Mythos first to core infrastructure companies—strengthening defenses before making it widely available—is described as a reflection of the judgment to secure the system before broad distribution.The discovery and remediation of vulnerabilities using Mythos are cited as examples that demonstrate the significance of prioritizing the defensive side. I view this “defense-first” approach as resonating with the distinction drawn in the previous chapter regarding national defense. It can be understood as a sequence where distribution occurs not simply because the technology is usable, but only after a system capable of protection is in place.
The discussion on trust is placed at the end of this chapter.
We can only diffuse this at the speed of trust. And trust is currently in short supply.
Translation: We can only spread this at the speed of trust. And trust is currently in short supply.
I interpret this sentence as presenting a perspective that measures the speed of adoption not by the performance of the technology, but by how society perceives it. As Rogers demonstrated in his theory of innovation diffusion in 1962, it has long been argued that adoption progresses in tandem with trust and social acceptance.During the discussion, while attempts to create channels for citizen involvement in design—such as Claude’s experiment with citizen feedback on the Constitution—were briefly touched upon, it was acknowledged that mechanisms to foster active participation remain incomplete. I believe this self-defined “incompleteness” serves as a frank acknowledgment of the current lack of trust.The question remains: How can we transform the feeling of being merely a passenger on a train into a sense of active engagement?
To Take Our Seats in the Cockpit Again
Through this conversation, I reframed AI not as an object to be observed from the outside, but as a train I am already riding.The fact that it cannot be stopped is presented not as a reason to give up, but as a starting point for asking where there is room for control. Ms. Daniela’s statement that “knowledge is power” and that using AI to understand it and form opinions leads to a voice in corporate and government decision-making has stayed with me as a renewal of my perspective.I understand the choice of whether or not to use AI not as a binary decision—to use it or not—but as a shift toward a position where I can discuss how to use it and where to draw the line. I sense a consistent design intent in the flow of the discussion, where the tough question posed at the beginning ultimately connects to the question of how to foster participation.
Whether it’s the boundary with national security, a design that eschews advertising and does not attempt to retain users, or the operational approach of deploying Mythos on the defensive side first—all can be understood as distinct manifestations of the same underlying question.I am considering where there is room to take on the questions of “where to draw the line” and “who can draw that line”—not as a decision made by some distant corporation, but as a design choice on our own side. I would like to interpret the phrase “the speed of trust” not as an excuse to slow down adoption, but as the speed required to return to the pilot’s seat.I want to take home this renewal of perspective—that “this is how the world is”—not as an instruction on what to do starting tomorrow, but as a reexamination of where there is room for engagement. Rather than entrusting the drawing of lines to someone else, I believe that considering which lines we ourselves can draw is the remaining form of participation.
Comment