The Deep View
Posts
Why persistent memory could transform AI agents

Why persistent memory could transform AI agents

The Deep View
October 21, 2026

Welcome back. New research finds that AI's use in scientific contexts may be accelerating discovery. Meanwhile, Amazon researchers created an internal tracing system that improves its AI coding agent, Kiro, from its own runs. And Databricks new memory layer for agents could solve one of the most critical problems agents face: A better understanding of a user's history. —Nat Rubio-Licht

1. Agents need better memory more than better models

2. Amazon tests a safer path to self-improving AI

3. Can AI free up scientists to think bigger?

ENTERPRISE

Why persistent memory could transform AI agents

What if the path to more capable AI in the real world wasn't building more powerful models?

Databricks launched new technology on Tuesday that offers today's existing models more extensive context and more persistent knowledge to improve AI performance. The company unveiled MemCurrent, a new memory layer for agents that can retain knowledge across sessions and create institutional memory for enterprises—and it offers some pretty impressive results.

Databricks reported these findings on the performance of MemCurrent:

Agent success rate in carrying out real-world tasks tripled from 22% to 68%, based on an internal benchmark; and it did things 26% faster and with 31% fewer tokens
Using three public benchmarks, MemCurrent performed as well as giving the agent every past conversation, but did it with 12x fewer tokens
Databricks found that MemCurrent outperformed three open-source memory systems, OpenClaw, Mem0, and PropMem, on accuracy and efficiency

This is one of the hardest problems in the AI industry today. It's why startups like Pathway have built entirely new architectures to fix the endemic memory problems in LLMs. The issue leads to a number of challenges for agents, especially wasting time and money relearning the same behaviors over and over again. That creates a tradeoff to those using agents—you can make them smarter or you can make them faster and cheaper.

Databricks tackles the problem by giving agents two types of memory: episodic (the raw conversation history of what was learned) and derived (what can be discerned from the patterns, preferences, and workflows). Based on Databricks' own internal benchmark of 114 enterprise tasks, the average task with no memory was completed 22.3% of the time. With only episodic memory, it was completed 35.8% of the time. With only derived memory, it was completed 66.1% of the time. With both episodic and derived memory, it was completed 68.4% of the time.

"On your next project, you don’t have to explain the same thing again," Michael Bendersky, research director of AI at Databricks, told The Deep View. "The agent will … store your preferences and understand how you work on projects. We can propagate these preferences … from project to project implicitly without you doing anything."

Databricks customers can start using MemCurrent today, but keep in mind that it's an underlying architecture and not a product. It's now integrated into the company's AI data assistant, Genie, where it can immediately boost agents. Customers can also use the new Agent Bricks Memory, launching in beta, to add the MemCurrent capabilities to any of their own custom AI agents running any model or harness. It all happens within Databricks' Lakebase database.

Beyond the immediate value for Databricks' customers, MemCurrent is another sign that many of the most important breakthroughs in AI are coming from better systems surrounding the models and not just the advancements in the models themselves. Memory continues to emerge as one of the most problematic limitations of LLMs, the predominant architecture of today's AI. The memory problem leads to inconsistencies, slow answers, and above all, inefficiencies that waste a lot of tokens, compute, and money. It's like having an employee use their time to learn how to do something one day and then forget it at the end of the day and have to waste time learning the same thing over again the next day, and the next day—and so you keep hiring more and more employees to overcompensate. In this analogy, more and more employees are equivalent to the additional GPUs, compute, and data centers required to run agents. Right now, the industry's solution is to just build more and more data centers, but it's already very clear that's not a sustainable or desirable path forward. What we need instead is efficiency breakthroughs like this one from Databricks.

TOGETHER WITH HIGHTOUCH

What if AI made us prove we’re creative?

This year AI shook up creative strategy. Most teams are still working like it didn't.

Ari Murray, Chief Digital Officer at Salt and Stone sits down with Bryan Harvey, Head of Brand at Hightouch, for an honest conversation about what's breaking on creative teams right now, how the roles are changing, and what a day looks like when creatives spend it on ideas, concepts, and craft while agents handle the manual execution.

Leave with new insights from 3 experts in the field and 5 prompts to instantly improve your creative output.

Save your seat

RESEARCH

Amazon tests a safer path to self-improving AI

One mistake may have gotten Amazon researchers a step closer to self-improving AI.

On Tuesday, Amazon unveiled Kiro-trace, an internal tracing system that improves Kiro, Amazon's AI coding agent, from its own runs. Put simply, this system sifts through Kiro's previous work to find mistakes or wasted effort and proposes changes, then tests whether those changes actually improved the coding agent.

The idea originated from a mistake in Kiro's file-read and search tool, in which one incorrect description resulted in more than 61,000 errors. Though these errors were inconsequential, as they didn't cause anything to break down, because they didn't cause any larger problems, they likely would have gone undetected. Kiro-trace, however, spotted the error and removed it.

The researchers claim that Kiro needed to solve four problems simultaneously: Find problems that are worth fixing; create a solution that won't cause cascading errors in other places; make the solution general rather than "tuned to a benchmark;" and finally prove that the change actually worked.

Additionally, Amazon's researchers found a solution to a common problem in self-improving AI: Separating the system that's proposing the change from the system that's judging its efficacy.

The tracing system will find problems in Kiro's runs, propose changes one at a time, and only keep changes when a separate "deterministic check" proves it's a good change. After that, the change is sent to a human reviewer who approves it.
Most Kiro-trace change candidates don't actually make it to a human reviewer before the deterministic test rejects it, the research notes. "The ones that do are the changes that can meaningfully improve the developer experience," the company said.
And it's already starting to show results: In the first week of operation, Kiro-trace opened eight changes to the agent, seven of which shipped. One of those changes improved the agent's ability to perform long tasks from 82% to 92%.

"When the same model proposes a change and scores it, it can optimize for its own score system, instead of the real goal, and the system drifts without anyone seeing it," Shweta Garg, applied science lead for Kiro at AWS, told The Deep View. "Kiro-trace is built around keeping those two jobs apart. The part that proposes a change never decides whether it worked. That decision runs in deterministic code the model cannot reach."

The result for the Kiro user is an agent that simply works better, the researchers said. For instance, users can expect fewer wasted calls, more careful task delegation, and better tracking of long context conversations.

The conversation around recursive self-improvement is often paired with a growing fear of a loss of control, begging the question: If AI is capable of endlessly improving itself, will it eventually grow so advanced that humans cannot understand it? But Amazon's research may suggest a way to develop self-improving AI while mitigating that fear. By keeping humans in the loop and separating the checks and balances from a system that may be incentivized to improve in nonoptimal ways, we are automatically putting a harness on the system before we arrive at a point where it's too late to control it.

TOGETHER WITH WIZ

A Practical Framework for AI Threat Readiness

As attackers use AI to identify vulnerabilities and accelerate exploitation, security teams need operational strategies that keep pace.

Wiz’s AI Threat Readiness Playbook provides a four-pillar framework security leaders can use to strengthen programs and prepare for AI-enabled threats.

Learn how to:

Reduce critical attack surface exposure
Improve zero-day response and remediation
Strengthen application security with AI-assisted analysis
Modernize detection and response using AI-driven workflows

Get the Playbook

RESEARCH

Can AI free up scientists to think bigger?

Concerns over AI-generated content polluting scientific journal submissions have led many to question its role in research, but recent studies from MIT point to a more nuanced reality.

The use of AI in science raises questions on several levels. The most visible concern is "AI slop," the worry that AI-generated work is eroding research quality. A less discussed but equally important question is economic: even if AI is useful, does the time it saves scientists justify what it costs? MIT CSAIL’s paper AI Disruption and Falling Costs in Theory Research tackles exactly this question, and its findings are promising.

To determine the true value of these AI models, the researchers compare how models released from December 2024 to April 2026 solved cutting-edge theory questions in cases that have ground-truth human answers that were entirely unavailable to the AI. The answers came from 254 recent arXiv papers from seven scientific domains spanning mathematics, physics, and economics. The researchers reported that in those sixteen months, the changes in the models were "enormous":

The share of problems for which models reached the same conclusion as the human authors (got full marks graded against the original research), rose from 3% (o1) to 35% (GPT-5.5) marking a 22 percentage point increase per year.
These results were achieved at minimal cost, approximately $5 per attempt (or $15 per success) for GPT-5.5.
Compute spending per full-marks answer fell from roughly $170 to $15, including failed attempts but excluding human labor and independent verification.
A "back-of-the-envelope" calculation suggests that AI costs are less than 0.3% of what it would cost to have a human expert solve these problems.

Despite AI becoming this much more helpful, it doesn't mean its role in research is as simple as plug and play. The researchers observe that the value driven from the AI depends on the share of research costs and the savings achieved with them, citing an example of an AI model which in order to perform well would need extensive and costly training.

The researchers also draw attention to the fact that when some tasks become cheaper to do, the structure of science will need to evolve to take advantage of it. For instance, even when scientists save seven hours per week using AI, they have a growing backlog of untested hypotheses and substantial demand for output verification.

"Under these conditions, we would expect the field to evolve so that more scientists are focusing on these other complementary tasks," the researchers stated in the paper. "Based on our results, we would expect similar adjustments within the theory community, as some of their historical tasks become much cheaper to do with AI."

Some of the more exciting, useful, and benign use cases of AI involve breakthroughs in research. Yet there seems to be a lot of hesitancy about using it in this domain, because the ability to rapidly create more content with AI has led to watering down the quality of some research submissions. Arxiv wrote about this issue just this month as it has had to update its rate limit policy due to the influx of submissions, fueled by AI, and also, the increase in "dense, AI-written papers" described as "low-value." This MIT research presents the other side, in which AI could help fuel further breakthroughs by performing tasks that net out to be less expensive than the scientist doing the work themselves. The key here is that it doesn't replace the scientist, as they would still be the person checking the results, picking the right questions, and making the larger-picture connections (which they would have more time to do). It's also important to highlight that these models are extremely capable, as highlighted in the report or everyday breakthroughs. Just today, Anthropic published a report explaining how Claude designed protein binders against scorpion neurotoxins, part of a broader experiment to see how Claude can accelerate drug discovery.