BIG TECH

Meta Muse shows the privacy cost of agents

The more an AI assistant knows about you, the less you have to explain yourself and the more seamless the interaction. One of the ways Meta is pushing that idea with Muse by getting to know your inner circle. 

Users raised privacy concerns when Meta launched its Muse app, given the company's reputation and track record across its other businesses. As a result, researchers have been extracting Muse's internal files to better understand how it works. Karan Joshi, an independent AI safety and security researcher, shared her findings with Wired, which suggest that Muse knows a lot more about you and your loved ones than you might think.

Muse's instructions include a directive to create "a page for every person in the user’s life." The report explains that this involves an hourly process that compiles data on family, partners, friends, colleagues, collaborators, and people you follow. Other details include: 

• Muse can use these suggestions to offer advice on how to improve relationships or where to take a friend that suits their interests. 

• The pages can be filled out over time with sections including facts, such as where they live, what they do, birthdays; history, such as a backstory; and strengthening, or how a relationship can be improved. 

• Other details include how people interact with each other.

"What it seemed like to me—from all these prompts, system skills data, and things that they’re feeding into Muse—is that they want to understand your relationships that you have with real people," said Joshi to Wired. "They’re trying to know you like a friend, which is honestly pretty creepy."

While this may seem like a deep encroachment on a user's personal life, it is important to note that Meta's Muse was designed to take these actions only when a user gives it access to the information. In a statement to The Deep View, Meta also highlights that each user has their own virtual machine, which stores user data and content and is inaccessible to other agents, and that users can modify memories or disconnect external services at any time. 

"For any agent to be useful and actually help you achieve your goals, it needs to have context about you and those you interact with," Meta told The Deep View. "Muse gathers that based on public information and from what you’ve chosen to share, which is how it remembers the person who just sent you an invoice is in fact the plumber who you previously hired to complete some work in your bathroom or which flowers your spouse said they liked best."

The backdrop for this report are less than favorable incidents from users who are already finding that Muse is already breaching people's privacy, which I recap in this article from last week. 

When I read this report, I understood why people would be alarmed to have their friends and relationships essentially mapped out for them in data. However, I wasn't surprised. These AI applications are essentially on a mission to collect as much information about you as possible to make your interactions as frictionless and useful as possible. That's why so many connectors exist to give AI assistants access to your email, Slack messages, bank accounts, and, in some cases, your hard drive. What concerns me more than the agent having access to these things is how little people know about it, through no fault of their own. Companies don't make it clear how much information users are opting in to share or what the AI assistant can do with it. If privacy is a concern and you don't want an AI system to know intimate details of your life, my best recommendation is to use these tools and get familiar with the technology without connecting any personal data. Wait until you feel completely comfortable with the idea that the company you're trusting may use that data in more elaborate ways than you originally imagined, such as mapping out your most personal connections.

TOGETHER WITH UNBLOCKED

[Webinar] How to stop babysitting your agents

Agents can generate code. Getting it right for your system, team conventions, and past decisions is the hard part. You end up wasting time and tokens in the correction loops. 

More MCPs, rules, and bigger context windows give agents access to information, but not understanding. The engineering teams pulling ahead have a context layer to give agents exactly what they need for the task at hand.

Join us for a FREE webinar on Oct 7 to see:

• Where teams get stuck on the AI maturity curve and why common fixes fall short

• How a context layer solves for quality, efficiency, and cost

• Live demo: the same coding task with and without a context layer

Register now

POLICY

Where OpenAI and Anthropic diverge on AI's future

Though the recent remarks from OpenAI and Anthropic seem aligned on AI safety and regulation, Sam Altman has clarified the differences between the two companies. 

In an interview with Politico's Decoded podcast, the OpenAI CEO said the AI lab has a higher tolerance for risk than its competitor, noting that the benefits of AI justify some of the adverse impacts. Altman summed up the difference between OpenAI's views and Anthropic's in one statement: "We believe that the world should accept some bad things happening for the benefits of this technology and people having the agency."

Additionally, Altman claimed that a "single lab in San Francisco" having complete control over the benefits of AI is "a completely unacceptable trade-off," preferring instead a “lighter-touch regulatory stance." Though Altman couldn't make the promise that there would be no instances of hacking, misuse or scams, he said "I think people will do tremendously—orders of magnitude more—good stuff than bad stuff.”

Both companies have been outspoken in recent months about the risks that powerful frontier AI presents as a number of instances of their agents breaking free of their sandboxes have come to light in recent months. 

In some cases, the frontier labs' CEOs have supported each other's governance takes: Following Anthropic CEO Dario Amodei releasing his essay, titled "We Must Pace the Frontier" that outlined the company's three-part governance and risk plan, Altman posted on X that he agreed with Amodei's call to action. Both companies also signed President Donald Trump's "morally binding" accord on superintelligence last week, pledging the implementation of several layers of safety auditors. 

However, Altman said in the interview that he rejects the notion that AI presents "really catastrophic risks," including "a serious loss of control to AI." This represents the most significant difference of opinion that OpenAI has with its main rival, which has claimed that AI's increasing capabilities could "outrun our ability to understand and control" these systems if left unchecked.

Anthropic did not respond to a request for comment from The Deep View in time for publication.

Altman clearly wanted to set the record straight on its policy opinions at a time when the pressure around AI risks is higher than ever. But Altman's comments reflect a long held truth that has always separated OpenAI from Anthropic: One lab simply has a higher tolerance for risk than the other. Anthropic was started in the first place as a counterweight to OpenAI's fast-paced development strategy, and was built around the mission of developing AI responsibly. While that mission has been drowned out in the public conversation as both companies race to create increasingly-powerful AI, these differing regulatory stances make their perspectives clear. Additionally, neither regulatory stance is altruistic. Both stand to benefit these companies in some way: While Anthropic's more strict policy opinions come with the allegations of regulatory capture, potentially centralizing more power in frontier labs' hands, the looser regulatory framework that OpenAI prefers allows these labs to develop models unhindered by mandatory safety requirements that could slow down the progress towards AI's benefits. 

TOGETHER WITH CREDO AI

Join AI & Governance peers in NYC at the AI Trust Summit

Every enterprise racing to get value from frontier AI is weighing the same tradeoff: Capability, Cost, or Control.

At the 5th Annual Credo AI Trust Summit on 10/22 in NYC, leaders at GM, Verisk, Cisco, and Elevance share their perspectives on this tradeoff and more: governing multi-agent systems, addressing shadow AI, and moving from tokenmaxxing to valuemaxxing. Then join Credo AI's forward-deployed experts in breakouts focused on what you can implement tomorrow.

Seats are limited. Register today.

HARDWARE

Anthropic's quiet move in open hardware is winning

Anthropic is the only one of three major frontier AI labs in the US that doesn't offer an open model. But a recent Anthropic move shows that the company does still want to support open ecosystems.  

In a move that has largely flown under the radar, Anthropic announced its Model Hardware Standard (MHS) at the end of August as "the MCP for hardware," as Alek Kemeny, a member of technical staff on Anthropic's Beneficial Deployments team, told The Deep View.

MCP (Model Context Protocol) is an open standard that enables AI models and agents to connect to external tools, data, and services through a common interface that Anthropic developed and open-sourced in November 2024. It has since become an industry standard with around 500 million monthly SDK downloads, and has only become more relevant in recent months with the rise of AI agents that are hungry for context and data. 

MHS hopes to do for hardware and devices what MCP has done for software and services. It wants to give AI models and agents a common interface for interacting with thousands of otherwise incompatible systems. While MHS was conceived as a way for scientists to drive lab automation using Claude, the response to the announcement has been intense, the Anthropic team said, and has spread far beyond science labs. Kemeny noted that in the first two weeks after the announcement, Anthropic received 30x the number of organizations requesting to participate than Anthropic had originally planned for. 

And the types of organizations that applied spanned a lot of different industries from semiconductors, advanced manufacturing, automotive, food processing, aerospace, energy, and critical infrastructure.

Some of the benefits already being reported include:

• Stabilizing quantum computers: QuEra used MHS to help Claude develop a program for controlling lasers used in its quantum computers. The finished program restored the laser correctly in 695 of 700 tests. Difficult recoveries took 10 to 14 seconds, compared with 5 to 10 minutes for a human expert.

• Running lab experiments automatically: Carnegie Mellon researchers used MHS to connect a liquid handler, plate reader, robotic arm, and cameras. An AI agent ran an experiment, decided the first result was not good enough, changed the concentration range and tried again on its own. The test used colored dye as a stand-in for a drug candidate. CMU says testing real drug candidates is the next step.

• Giving labs a 24/7 operator: A University of Washington PhD student connected six lab instruments through MHS in less than a week. He built a system where AI can watch a DNA amplification experiment and stop it at the right time. He also connected a robotic arm and liquid handler so they could move lab plates safely between machines.

"Beneficial Deployments is really charged with seeing, owning, and championing our public benefit mission, and ensuring that the benefits of AI extend to positive outcomes for humanity," Jonah Cool, head of our partnerships and deployment at Anthropic, told The Deep View during the same interview with Kemeny. "As it pertains to science, a lot of our work is really inspired by the guiding light of the essay that our CEO Dario [Amodei] wrote, Machines of Loving Grace."

Anthropic has gained a reputation for being anti-open-source, mostly due to the fact that Amodei has raised safety concerns about open-weights models. However, the successful open-source rollout of MCP provides a counterweight, as does Anthropic's commitment to and support for the scientific community, which tends to lean on open ecosystems for the purposes of both sharing research and verifying results. Anthropic still describes MHS as being an application-only research preview. But they asserted that they intend to open-source MHS in the hope of it becoming an industry standard like MCP. Kemeny and Cool explained that they're first working with industry partners to test safety and security while the models are now operating on real-world equipment. It's also a fun note that since Cool has a PhD in cell biology, one of the leaders of this initiative is essentially named Dr. Cool. That kind of marketing is nearly as good as making it an open platform.

LINKS

• OpenAI to begin US ad format testing with select advertisers

• AI spending costs is nearly impossible to budget

• Nolla Health to prescribe acne drugs in Utah without human oversight

• Tech billionaires account for AI wealth gains, non-tech peers lost $62 billion

• IWF finds more than 6,000 AI images of child sexual abuse, up 40%

• Sam Altman says he's "very uncomfortable" with ascribing religion to AI

• JumpCloud: Secure every identity, human or AI agent, with clear accountability from access to activity. See how it works. sponsored

• ChatGPT: Ranks first in A16z Top 100 consumer AI mobile apps by monthly active users 

• GPT-6 Astra: has taken the #1 spot across 4 of Design Arena’s leaderboards

• North 2: Cohere launches biggest update yet including 15+ new features 

• Contivio: AI Communications Solutions Engineer

• Deloitte: Google AI Architect

• Booz Allen Hamilton: Agentic AI and Data Engineer

• Kaizen Equity Partners: AI Operations Associate

GAMES

A QUICK POLL BEFORE YOU GO