Pope Leo called AI an 'instrument of domination, exclusion and death.' Anthropic was in the room

Pope Leo XIV has called for comprehensive regulation of artificial intelligence and urged developers to prioritize the common good over profit, as he issued a profound manifesto aimed at protecting humanity from the extensive implications of AI technology, which reaches into various domains from employment to military engagements.

Titled "Magnifica Humanitas" (Magnificent Humanity), this encyclical marks Leo's initial foray into addressing the moral complexities posed by AI, following his declaration shortly after his election that he regards AI as the preeminent challenge confronting humanity today.

In the encyclical, Leo critiques the prevailing "culture of power" that fuels the competitive race in AI, particularly the advancement of sophisticated remote warfare technologies. He stated that it is "not permissible" to allow AI systems to make irreversible, lethal decisions, thereby highlighting a significant point of contention between the papacy and the Trump administration, which has been actively seeking to reduce regulatory barriers surrounding AI development.

"Artificial Intelligence now demands to be disarmed, freed from logics that turn it into an instrument of domination, exclusion, and death," the pope asserted during a special Vatican presentation of the encyclical, a significant type of authoritative teaching document issued by the pope.

Industry experts, academics, and advocates of Catholic ethics concur that this document is poised to become a key reference point in the ongoing discourse surrounding AI, serving as a guiding framework for lawmakers, researchers, and the general public alike. This comes in response to the rising frequency of advancements in AI that have sparked fears of job displacement and a potential decline in human cognitive capabilities.

Taylor Black, an executive at Microsoft leading their AI initiatives and director at the Catholic University of America’s AI institute, noted that the encyclical will compel those "at the forefront of these tools" to contemplate crucial questions, including "What does it mean to be human?"

Pope calls out AI companies even as he hosts Anthropic

The Vatican event prominently featured comments from Christopher Olah, co-founder of Anthropic, which is currently engaged in a legal dispute with the Trump administration concerning access to its AI technology. The decision to include Anthropic in this dialogue stems from the Vatican's long-standing initiative to engage with Silicon Valley regarding the human implications of AI development.

However, within the text of the encyclical, Leo voiced strong disapproval of the consolidation of power and data among a limited number of private sector entities, citing it as a significant threat, particularly to children and vulnerable populations. He called for rigorous regulation of their operations.

"It is not enough to invoke ethics in the abstract; robust legal frameworks, independent oversight, informed users, and a political system that does not abdicate its responsibility are required," he wrote. "A more moral AI is not enough if that morality is determined by a few."

Leo urged AI developers and regulatory leaders to pause and introspect on their actions. He called for adherence to ethical and spiritual principles that encourage a commitment to the improvement of humanity rather than individual profit or power.

AI industry leaders OpenAI and Anthropic rank as the second- and third-most valuable privately held companies in the U.S., each valued at hundreds of billions of dollars, surpassing the GDP of many nations. Both firms are poised for impending IPOs that may approach the trillion-dollar mark.

Christopher Olah expressed appreciation for the pope's critique and concerns, describing the necessity of establishing external checks as vital to ensuring the technology benefits humanity, particularly given the significant risks involved, including potential widespread displacement of human labor.

"We need more of the world — religious communities, civil society, scholars, and governments — to do what His Holiness has done here: to take this seriously, to look closely, and to push events in a better direction," Olah asserted. "We need moral voices that the incentives cannot bend."

Experts say the text will become a benchmark

In his meticulously crafted encyclical, the math-major pope drew upon the historical precedent of the Catholic Church's social teachings, applying foundational concepts such as justice, solidarity, the dignity of work, and the universal distribution of resources to the complexities of the digital age.

"I am convinced that this will prove to be a defining document for our era, a profound and prophetic document," stated Paolo Carozza, a law professor at Notre Dame and chair of the Meta Oversight Board.

"Pope Leo is offering a clear, comprehensive, and coherent voice urging us to take responsibility for constructing a world in which technology will serve humans rather than degrade them," he emphasized.

In its most compelling passages, the encyclical rebuked the role AI has played in accelerating the "normalization of war," leading to a desensitization to its impacts. While not specifying particular conflicts, Leo referenced "opposing imperialisms, between powers that wish to preserve their supremacy, and those that aspire to seize that supremacy."

He called for transparency and accountability in AI technology development, insisting that the chain of command involved in deploying AI weaponry remains transparent. Leo asserted that the Catholic Church's "just war" theory, historically providing guidelines for justifiable use of force, is now "outdated" in light of contemporary advancements in warfare.

A text in the church's social justice tradition

Leo signed the encyclical on May 15, coinciding with the 135th anniversary of "Rerum Novarum" (Of New Things), the seminal document authored by his predecessor and namesake, Pope Leo XIII, which addressed workers' rights amid the rise of industrialization.

The teachings of "Rerum Novarum" laid the groundwork for modern Catholic social thought, and the current pope referenced this document at the inception of his papacy concerning the AI revolution, perceiving it as raising comparable existential concerns to those posed by the Industrial Revolution a century prior. "Magnifica Humanitas" thus emerges as a continuation of a century-long tradition wherein popes update "Rerum Novarum" to address contemporary social dilemmas, focusing significantly on the dignity of work vital for human flourishing.

AI evokes both existential anxieties and visionary possibilities amid heated discussions on whether it will act as a force enriching humanity or as a detrimental technology that undermines human intellect while eradicating millions of high-wage jobs.

"The pursuit of greater profits cannot justify choices that systematically sacrifice jobs, because the human person is an end, not a means, and the economic order must remain subordinate to human dignity and the common good," emphasized Leo.

In a groundbreaking move, Leo extended his commitment to upholding human dignity in labor by issuing the first-ever papal apology, acknowledging the Holy See's historical role in endorsing slavery by granting European monarchs the authority to subjugate and enslave "infidels."

A decade-long dialogue with Silicon Valley

While Vatican officials did not disclose the individuals involved in crafting Leo's encyclical, it is known that church representatives have engaged in discussions with technology firms in Silicon Valley over the past decade.

The inclusion of Anthropic during the Vatican launch faced criticism from some observers who interpreted it as a papal endorsement of the AI company, which is in litigation against the Trump administration due to an order halting the use of its technology by U.S. agencies, stemming from its refusal to permit unregulated military applications.

Brian Boyd, the U.S. faith liaison for the nonprofit Future of Life Institute, viewed the presence of Anthropic's co-founder Olah as a recognition of the firm's influence in the field, akin to a papal audience with a national leader: not an endorsement but an acknowledgment of its significance.

Boyd characterized Anthropic as an "enormous corporation that is taking onto itself an enormous risk and responsibility," noting that the company has "demonstrated genuine goodwill and integrity and interest in dialogue."