Learning, Remembering, and ActingFoundations of Reliable AI Agents

Research Vision

When humans take action, they observe the environment, draw upon their own existing knowledge, consult external records, or seek information from others. Similarly, AI agents pursue their goals by observing the environment, utilizing tools, and interacting with humans or other agents. I focus on two key components of a single agent: the foundation model, which encodes internal knowledge, interprets information, and proposes actions; and the harness (or controlling system), which connects the model to the external world. The harness assembles observations and retrieved records into model inputs, executes tool calls, and returns their results.

The internal knowledge employed by a model is acquired during training and encoded within its parameters, whereas external information is obtained during task execution—through the retrieval of records, the output of tools, or interactions with others. Both types of knowledge may be incomplete or erroneous. This raises several questions: Can a model assess the likelihood of its own success based on available information, and can it determine when it needs to seek external assistance? Furthermore, when a model communicates such an assessment, can other participants form an accurate expectation of its capabilities based on what it communicates?

These questions link knowledge concerning the modeling of the agent itself, the environment, and other participants. Through experience, an agent should be able to gather evidence about which tasks it can perform, how the environment responds to its actions, and what other participants know and can do. I plan to investigate how such evidence is learned, recalled, and utilized. As agents accumulate experience, they must also be able to revise their existing knowledge. Understanding how agents identify and incorporate these revisions is a critical step toward ensuring the long-term reliability of their behavior.

Learning and Encoding

What should be internalized, and what should remain external?

Research Foundations.

In Rote Learning Considered Useful, we show that a model can generalize over memorized data after learning how to interpret the stored associations ¹. I also collaborate with my colleagues to explore further the relationship between memorization and learning during the training process ² and compare the effectiveness of fine-tuning versus in-context learning within controlled formal language tasks ³. These studies distinguish between the memorization of data samples and the learning of reusable patterns, prompting us to further contrast learning via parameters with learning via context.

Future Directions.

Agents deployed in a dynamic world receive feedback from the environment. My research aims to investigate when experience should be used to update model parameters versus when it should be retained in external storage. One hypothesis is that updating model parameters is more advantageous for stable, recurring patterns, whereas information that is highly variable or context-specific is better retained in explicit records that can be updated. User modeling provides a concrete context for this comparison: a shared model can leverage retrieved personal history, while a personalized model can learn patterns specific to an individual. By conducting a comparative analysis of these approaches, we can uncover their distinct characteristics regarding transfer to new scenarios, adaptation, and correction.

A key point of comparison is the extent to which an agent can improve its performance by adjusting its external harness, its model weights, or both. Memory-based adaptation—which keeps the underlying model static—offers a practical entry point for this inquiry ⁴. By utilizing the same set of experiences across these three approaches, we can evaluate when parameter updates are truly effective and assess the relative ease with which each method allows for corrections. Furthermore, the patterns learned by a model can be translated into verifiable statements, which can then be reviewed by users or external auxiliary mechanisms.

Memory and Retrieval

How do agents acquire and integrate internal and external knowledge?

Research Foundations.

My research explores what information a model retains and when that information can be retrieved. We first try to extract the latent knowledge in parameters that might otherwise be missed using other prompting methods ⁵. Our analysis of memory systems in conversational platforms examines which parts of a user's history are transformed into persistent external memory, and what types of information are saved there ⁶. Our research on temporal context reinstatement examines how models recover event order from information present in long contexts ⁷. These findings reinforce the view that for an agent to operate over the long term, it must be able to retain both its experiences and the contextual environments in which they occurred ⁸.

Future Directions.

My upcoming research focuses on how the harness can select and provide the information a foundation model needs, given its internal knowledge and the changing environment. Various retrieval and memory management methods offer different ways to surface relevant records ^{9, 10, 11, 12}. My starting point is that retrieval should depend not only on the task, but also on what the model already knows. The question is whether the model needs the retrieved information, and how it uses that information alongside its internal knowledge. An external record can be stored persistently, but the model can only use it once the harness makes it available in the current context.

In a changing environment, a further question is: which earlier successes and failures are relevant now? A previous success may have depended on a tool or collaborator that is no longer available. Comparing ordinary retrieval with reference contexts containing the relevant evidence can separate failures of access from failures of interpretation. The same approach can examine how agents revise their expectations of collaborators when their capabilities change.

Behavior & Agentic Action

How should internal and external knowledge guide an agent's decisions?

Research foundations.

In To Call or Not to Call, we evaluate tool decisions through necessity, utility, and affordability, identify mismatches between models' perceived needs and outcome-based reference decisions, and develop hidden-state estimators to improve tool use ¹³. Collaborative work studies latent source preferences ¹⁴ and variation in generative web search ¹⁵. My ongoing work on deference to tool outputs further investigates conflicts between internal knowledge and external evidence. These projects connect knowledge assessment to observable information seeking and use.

From knowledge to action.

Can an agent predict its success well enough to choose between acting, acquiring information, and seeking help? Existing work on answer correctness studies whether models can estimate the probability that their answers are correct ¹⁶. My focus is on action outcomes, where success also depends on tools, retrieved experience, and execution conditions. This makes capability assessment a question about the agent in its current situation, not just the underlying model.

Initial experiments will vary internal knowledge and external evidence independently, including agreement and contradiction. Self-reports, model representations, and execution histories provide candidate signals for predicting success when conditions change. Reliable decisions require accepting well-supported corrections without blindly following external claims. Task completion, information-acquisition costs, and consequential errors will provide joint criteria for assessing the resulting policies.

Capability communication and multi-agent collaboration.

Choosing whom to consult requires judging what another agent can do. Learning-to-defer research provides methods for selecting between a model's prediction and an expert's decision ¹⁷, while human–AI research measures whether people appropriately accept or reject AI advice ¹⁸. These provide methodological foundations for my investigation of collaboration among agents and humans. I distinguish three sources of failure: inaccurate self-assessment, inaccurate capability expression, and a recipient's incorrect interpretation.

A capability claim links internal and external knowledge across agents. For the speaker, it expresses an assessment based on learned knowledge and execution evidence. For the recipient, it is new evidence to weigh against prior expectations and interaction history. The research question is when recipients should accept a claim, request supporting evidence, or give greater weight to observed performance.

To separate competence from its communication, experiments will control tool access and task knowledge while varying capability descriptions and interaction histories. Humans and other agents will predict performance and choose whom to consult, delegate to, or verify. Prediction accuracy, erroneous delegation, verification cost, and joint success capture different consequences of these judgments. Changes in capabilities and access to private or shared information can reveal whether collaborators update their expectations or amplify a common unsupported belief. The objective is reliance that matches capability.

Testbeds and Long-Term Goals

Personalized agents and human behavioral simulation provide testbeds for the foundational agenda. Personalization examines whether knowledge about an individual supports new tasks and adapts to changing preferences. Simulation examines whether agents reproduce behavior under specified information and social conditions. Generative-agent architectures and empirical studies of collective reasoning motivate these tests ^{19, 20}. One target is leakage of the foundation model's general knowledge into a simulated person who should not possess it; effective assistance alone does not establish behavioral fidelity.

Across these settings, matched interventions on parameters, context, and memory will separate failures of learning, access, interpretation, and action. Capability estimates will be checked against actual outcomes, their communication against the estimates, and recipients' expectations against subsequent collaboration. This links individual learning to how agents model and trust one another.

My long-term goal is reliable and controllable AI systems grounded in an understanding of internal–external knowledge interaction: how experience becomes knowledge, how that knowledge becomes accessible, and how it supports individual and collective action. The three themes define a sustained research agenda while leaving room for methods to evolve as the roles of model weights, memory, and interaction become clearer.