Safety and alignment for multi-agent systems

Where to begin?

Technical AI safety, as currently practiced, targets a single model. Evaluations, red-teaming, mechanistic interpretability, AI control techniques are all designed to interrogate and mitigate behaviors of one specific LLM. What about when models interact? Do they stay aligned, individually? Do their collective actions retain that alignment?

Emergent behaviors

Anthropic recently studied whether individually-aligned models translated to aligned groups when working together. The title of the paper says it all: AI Organizations Are More Effective but Less Aligned than Individual Agents (Shen 2026)¹. Essentially, the researchers observed that an AI group coordinating autonomously was collectively more capable than its individual constituents, but it also suffered an alignment penalty.

Anthropic’s experiment captured how systems of interacting agents are complex systems with emergent properties. The group’s behavior is not a sum of per-agent behaviors; see, for example, More Is Different (Anderson 1972)². When two agents are each aligned in isolation, they can jointly discover a misaligned strategy neither would produce alone; an error that would be harmless or undetected per single-agent tests can propagate and amplify through a team; coordination can arise through channels that emerge without being designed; a group can encode its coordination jointly such that any single agent’s record would raise no flags.

The observer’s vantage point

Examining agents individually can’t reveal the products of their interactions, leaving open the question: what is this group doing, as a group? Single-model safety practices are one part of the picture, but there’s more. How do we examine the more?

Traditional technical AI safety techniques can be categorized as either white box, with privileged access (e.g., model internals), or black box, with actionable information derived from external observations. In the multi-agent case, white box access is possible in limited scenarios — say, where a model developer is evaluating a group of its own agents, or where regulations compel participating model developers to provide access.

In contrast, the more general case is multi-principal (a variety of models from different developers) and assumes no privileged access to weights, activations, private reasoning traces, or any other internal computation that produced the observable behaviors. Here, the reliably available information derives from observable behaviors (the actions each agent takes, the messages that pass between them) and the state of the shared environment.

On the record

The behavioral record includes model-specific information in addition to observed interactions and details of the environment. A model’s output transcript, as revealed within the environment, becomes part of the interaction record. Its deployment instructions (the system prompt and task specification), are potentially disclosable information which may or may not be included in the record. The private reasoning traces some models produce are excluded from the record for three reasons: no other agent sees them, so they don’t contribute to interactions; an observer generally cannot obtain them; and a trace is not reliable evidence of the underlying computation (Reasoning Models Don’t Always Say What They Think (Chen 2025)³).

The available behavioral record, then, is data about interactions within the shared environment. What can an observer actually measure about a group, based solely on behavior? And what are the requirements for such a measure to be a construct-valid metric that can legitimately inform decisions about safety and alignment?