Building self-reorganizing software systems

TL;DR

• Software systems can learn to reorganize their own work.
• Keep lessons learned when agents or workflows change.
• Use consistent requirements when comparing old and revised workflows.
• Give systems more freedom as they prove reliable.

Software already performs work in teams

AI agents can investigate a defect, write a repair, and check whether it works. A software maintenance system might divide these activities among several agents, each with its own instructions and tools. A person decides what the system should achieve and which changes can reach users. Becoming a manager of agents describes this shift toward directing and reviewing work. That division of work is a design choice. Someone decides who receives a task, which information they get, and when another participant takes over. The same applies to an automated customer support service or a system that processes invoices. The model is one part of a larger arrangement of people, software, and responsibilities. Research on agent organizations describes who does what, what the agents work toward, and what each agent is allowed to do. [1]

Fixed roles can outlive their usefulness

Imagine three agents maintaining a customer app. One writes a repair, another checks that it works, and a third checks security. The security reviewer discovers that a customer could see another customer’s records. The repair agent was never told which records customers should be allowed to see. Its work has already passed the general review, but now it needs rewriting and checking again. If this keeps happening, the system could change the workflow so the security reviewer provides those rules before the repair is written. Both reviews would still take place. The diagram shows only the repair agent and security reviewer to make the change in their interaction clear.

---
config:
  sequence:
    mirrorActors: false
    actorMargin: 380
    width: 180
    height: 48
    actorFontSize: 14
    messageFontSize: 14
    noteFontSize: 14
    diagramMarginX: 16
    diagramMarginY: 8
    messageMargin: 18
    noteMargin: 6
  themeCSS: |
    svg { background: transparent !important; }
    rect.actor, rect.note { fill: transparent !important; stroke: var(--border) !important; }
    text, tspan { fill: var(--foreground) !important; }
    .actor-line, .messageLine0, .messageLine1 { stroke: var(--foreground) !important; }
    marker path { fill: var(--foreground) !important; stroke: var(--foreground) !important; }
---
sequenceDiagram
    participant A as Repair<br/>agent
    participant S as Security<br/>reviewer
    Note over A,S: Before: late rules
    A->>S: Repair ready
    S-->>A: Missing rule.<br/>Rewrite repair.
    Note over A,S: After: early rules
    A->>S: Access rules?
    S-->>A: Own records only
    A->>S: Repair uses rule
    S-->>A: Rule checked:<br/>approved

Improving AI agents from mistakes covers how failures can improve shared definitions and checks. The next step is to make the division of work itself open to revision. A system could learn that a responsibility belongs elsewhere, that a check should happen earlier, or that a role has become unnecessary.

Deciding which parts of the organization can change

Self-reorganization means that a software system can revise how its components divide and coordinate work. Hübner’s model describes how agents can review the way they work together, suggest changes, choose one, and put it into practice. [2] In the maintenance example, it could propose checking access rights before drafting a repair and involving a security reviewer whenever a change affects them. It might also give the repair agent the relevant security requirements from the start. These changes would alter who participates, what they know, and the order in which they work. AI could support that process throughout. It could examine completed tasks and rejected changes, identify repeated delays, propose a different arrangement, and carry out an approved revision. A person’s observation could start the same process. The system would retain the evidence behind the proposal so others could inspect it.

Adding agents would be one option. Removing a duplicate review or combining two handovers could be another. The power of system design makes the case for questioning and removing unnecessary steps. A system that reorganizes itself should be able to do that too. The permitted changes need to be explicit. Moving a check earlier is different from removing it. A system allowed to revise its working instructions should still have to obey the access restrictions and security requirements set by people. Required checks should remain separate from the instructions the system can rewrite.

Reorganizing through small changes with lasting evidence

Start with a limited trial of the proposed arrangement in a separate copy of the system. Run comparable maintenance tasks through the existing process and the proposed one. Check whether repairs remain correct, whether fewer handovers are needed, and how much time the agents and people spend. Keep a record of the previous arrangement so an unsuccessful change can be reversed. The experience needs to survive the agents that produced it. A replacement agent should receive the reason for the earlier security check, the cases that exposed the problem, and the limits of the resulting rule. Keeping every past conversation would leave the new participant to reconstruct those lessons. Useful shared knowledge needs an explanation of when it applies, with the evidence available behind it. When a tool or task changes, that explanation needs reconsideration. Wang’s 2026 preprint proposes keeping track of lessons learned, changes to the workflow, who may approve them, and the effort they require. [3]

The system could also improve how it evaluates proposed changes. If every trial consumes days of review, it could investigate which checks provide useful evidence and which repeat the same work. A revised review procedure would itself need evaluation and approval under the existing rules. The system should retain the history of those revisions so a later participant can understand how the current procedure came about. Direct human supervision could decrease for activities where the system demonstrates reliable performance. It might eventually rearrange routine work without asking each time, while changes affecting access to customer records still require approval. Scerri and colleagues study when agents should make decisions themselves and when they should ask a person, including the cost of that handover. [4] Here, the practical principle is to give the system more freedom for tasks it handles well, while keeping closer review where a mistake could cause harm. Where more agents are involved, it should be clear who is responsible for each decision. A failure, unfamiliar task, or replacement model may require closer supervision again.

Antifragility means a system benefits when the conditions it faces vary more. Taleb and Douady give this idea a mathematical definition. [5] Learning from unexpected problems could help a software system do that, within limits. Grassi and colleagues describe a software design in which a system learns from change and uses those lessons to revise its plans. [6] The maintenance example follows that idea by using a rejected repair to reconsider how future work is organized. Black Swans and AI covers the broader argument for designing around uncertainty. Reorganization can improve a system without proving antifragility. To show a benefit from variability, compare systems exposed to different amounts of change over a stated period, accounting for differences in information and resources. A newer model or a larger budget could otherwise explain the improvement. The comparison needs to count experiments, failed proposals, and human review alongside successful repairs. It should also include ordinary tasks, where a new security step might add delay.

A faster repair process that exposes customer records would violate a binding requirement, even if its average results looked better. Use the same requirements throughout a proposed change and its evaluation, so the old and revised workflows are judged on the same basis. Changing what counts as a successful repair during that comparison could make the numbers improve without helping users. Keeping metrics tied to what matters explains why the measure must remain tied to the purpose. People should retain authority over that purpose and the requirements that protect it. Those requirements can change separately over time as people’s needs change. The system should record which requirements applied to each evaluation and reassess earlier decisions when those requirements change. The aim is a software system that can improve its own organization through experience while retaining the knowledge needed to judge each revision against the requirements that apply to it.

References

1. J. F. Hübner, J. S. Sichman, and O. Boissier. A model for the structural, functional, and deontic specification of organizations in multiagent systems. In Advances in Artificial Intelligence, SBIA 2002, LNAI 2507, pp. 118–128, 2002.
2. J. F. Hübner. Um modelo de reorganização de sistemas multiagentes. Doctoral thesis, Universidade de São Paulo, 2003.
3. Z. Wang. Recursive organization improvement: A modeling specification for human–agent organizations. arXiv:2609.38643v1, 2026. Preprint.
4. P. Scerri, D. V. Pynadath, and M. Tambe. Towards adjustable autonomy for the real world. Journal of Artificial Intelligence Research, 17:171–228, 2002.
5. N. N. Taleb and R. Douady. Mathematical definition, mapping, and detection of (anti)fragility. arXiv:1208.1189, 2012. Preprint version.
6. V. Grassi, R. Mirandola, and D. Perez-Palacin. A conceptual and architectural characterization of antifragile systems. Journal of Systems and Software, 213:112051, 2024.