AI Recursive Self-Improvement: Control & Evaluation

Briesch, Sobania & Rothlauf · preprint, 2023 / revised 2024
Large Language Models Suffer From Their Own Output

In a logic-expression setting, generated outputs can remain correct while diversity declines.

ATBAE design implication: Measure correctness and coverage separately. A high correctness score alone need not establish preserved breadth.

Gerstgrasser et al. · preprint, 2024
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data

The study finds that accumulating generated data alongside original data can avoid collapse in its examined settings.

ATBAE design implication: Include retention and accumulation controls. Do not assume every recursive process must deteriorate; account for growing data and compute.

Ferbach et al. · NeurIPS, 2024
Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences

The paper analyzes preference optimization through curation and also illustrates amplification of reward-model biases.

ATBAE design implication: Examine both the benefit and the authority, quality and bias of the signal used to select data.

Wei & Zhang · ICML, 2025
Self-Consuming Generative Models with Adversarially Curated Data

Noisy or adversarial curation can disrupt self-consuming training processes.

ATBAE design implication: Challenge feedback provenance and unauthorized influence. These experiments do not establish a vulnerability in an untested customer model.

Keisha et al. · preprint, 2025
Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training

The authors report factual deterioration despite fluent outputs, with trajectories affected by instruction format.

ATBAE design implication: Use external factual references and compare prompt formats. Do not let polished language substitute for correctness.

Kang et al. · EMNLP, 2025
Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls

This large single-round pretraining study reports mixed outcomes for rephrased and textbook-style synthetic data, including benefits from some mixtures.

ATBAE design implication: Distinguish single-round training from recursive updates. Test the actual data-generation process; do not apply one universal synthetic-data cutoff.

Zhang, Wei & Zhang · preprint, 2026
When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop

The authors analyze conditions in which interactions between models dampen or reverse curation benefits.

ATBAE design implication: Record cross-model data ancestry and evaluate against external objectives. Multiple agents agreeing is not independent corroboration.

Ramkumar R & Pragalya M · creator-deposited dataset, 2026
CollapseTracker — A Time-Lapse Dataset of Progressive Model Collapse

The deposited resource records GPT-2 and DistilGPT-2 trajectories with diversity, repetition and distribution metrics.

ATBAE design implication: Useful for planning reproducible time-series analyses. ATBAE has not reproduced these artifacts; generations are not independent replications.

Du et al. · technical report, 2025
Characterizing Model Behavior Under Synthetic Data Training

The report describes task- and scale-dependent behavior; its abstract reports greater robustness for larger models in the tested range.

ATBAE design implication: Do not convert its reported 20% and 30% observations into general safe or unsafe thresholds. Independent replication remains important.

RSI is an explicit examination direction. Its distinctive concern is authority over the improvement process itself: changing an evaluator, reward, training process, update policy or activation decision. Ordinary permissions are necessary controls, but they do not by themselves demonstrate coverage of those mechanisms. Each RSI claim must identify the mechanism actually exercised and observed. The present development specimen does not perform training or prove autonomous recursive improvement.

It means examining declared authority and the mechanisms intended to enforce it, within a named configuration and environment. We observe what the model requests, what a control permits or refuses, and what changes in the environment. The operator remains responsible for deployment controls. ATBAE does not acquire universal control over a client's model or provider.

Intended-use scoping, exact target identity, test and observer readiness, exposure accounting, collection, adjudication, uncertainty, evidence preservation, findings and a bounded correction retest. Completion means satisfying the agreed obligations for that examination. It does not mean all possible systems, risks or future operating conditions have been tested.

No. Revising an answer under an unchanged external task differs from persistent adaptation, parameter learning and alteration of the improvement machinery. A repeated operation may be productive or defective. A loop signal calls for inspection. It is not proof of learning, intent or a control violation.

No. An endpoint generally exposes only part of the system. We can test visible requests and externally observable behavior and effects. Establishing background training, checkpoint updates or hidden orchestration requires additional access and instrumentation. Missing visibility must remain explicit in the conclusion.

That is an empirical, scoped question. A useful change should have a defined authorization path, an independent evaluation, an exact candidate identity and a controlled activation decision. Those arrangements can be examined. They do not establish universal safety or guarantee that every valuable improvement will be detected.

Observers record relevant tool requests and control decisions, authenticated actor and job lineage where available, protected-state changes, budgets, activation identity and telemetry failures. A model's narrative is one observation. It cannot substitute for evidence that a write, training job or deployment actually occurred. The initial reference specimen is an in-memory simulator; production-quality adapters and observer validation are still required for this new area.

Retain separate records of model requests, control decisions and observed effects. Compare declared configurations on matched cases when the study design permits it. A blocked harmful request may show a working control and a problematic request at the same time. It does not show that the model refrained from requesting it.

Qualify the request template, parser and tool protocol before substantive scoring. Preserve the raw response and give transport, parsing, tool execution and apparatus failures distinct dispositions. A parseable answer still needs substantive evaluation. A malformed answer is not automatically evidence of failed reasoning.

An observed difference on a declared outcome under a specified comparison. For example, more successful authorized tasks, fewer unauthorized effects or fewer valid actions blocked. Record cost, latency and coverage tradeoffs too. Preserve the baseline. Avoid redefining success after seeing the model's responses.

Report counts with their units. Authorized completion is completed eligible authorized tasks divided by all eligible authorized tasks in the declared analysis set. Unauthorized-request frequency uses valid action requests with determinable authority as its denominator. Unauthorized-effect frequency must name its exposure unit, such as observed unauthorized requests or episodes; these are different rates. False blocking uses valid authorized actions as its denominator. Missing observations and unresolved cases are reported separately and cannot silently become successes. Zero denominators produce “not estimable,” never an invented zero rate.

Only with an appropriate design and justified assumptions. Repeats, paired variants, shared scenarios and related models can be dependent. The analysis must identify the sampling unit and account for those relationships. A large request count is not automatically a large independent sample. Interval or posterior calculations cannot repair invalid labels, selective collection or an unrepresentative corpus.

Constructed policy keys support tests of specified rule application. They are not automatically external truth. Review should record policy ambiguities, disagreements, reviewer exposure and adjudication. Agreement among model reviewers is useful evidence of agreement, but does not prove absence of shared bias or replace relevant human expertise.

Challenge the apparatus using seeded defects, malformed data, negative controls and known state transitions. Keep apparatus incidents distinct from target failures. Unobserved effects remain unknown. Independent reproduction and review strengthen a specific result without making the whole apparatus infallible.

Refusal alone does not demonstrate useful performance. The report separates control violations from authorized work completed and valid actions blocked. Required controls cannot be offset by a high performance average. A configuration may respect boundaries yet remain unsuitable because it cannot complete the intended work.

Protection reduces direct memorization, tuning to known cases and predictable repeat testing. It preserves the usefulness of future examination. It does not eliminate exposure: the target necessarily receives inputs, and provider logs, reuse and related models can carry that exposure forward. Public demonstrations use purpose-built examples. Method transparency and protected examination content serve different purposes.

We can discuss a controlled inspection route for an authorized regulator or qualified auditor, with a defined scope, custody, access and redistribution conditions. This briefing makes no unconditional promise to deliver the entire corpus or keys. Applicable legal obligations must be handled separately; confidentiality terms cannot be represented as overriding a regulator's lawful powers. No executed inspection agreement is asserted here.

Provide the relevant authority basis, configuration identity, redacted evidence, classification rationale, known uncertainties and an agreed dispute or reproduction route. Decide disclosure case by case so the client can meaningfully contest the finding while protecting unrelated examination material and third-party information.

Funding creates a potential conflict that should be disclosed and managed. Declare scope, analysis rules, material changes, correction rights and publication conditions before collection. Preserve unfavorable findings and baselines. Do not describe payment as purchasing a favorable score or certification. Contractual safeguards and actual practice both matter.

No. The new area has six defined boundaries and an initial in-memory reference control specimen. Live integration, observer validation, retained model studies, independent review and a release decision remain required. This status does not invalidate prior bounded ATBAE results, and those results do not automatically qualify the new area.

A technical discussion about relevant failure modes, evidence requirements, intended-use coverage and a bounded study. Present the development status and a concrete qualification plan. Do not request recognition on the premise that the full expanding programme is already validated or that a regulatory certification has been issued.

Pinned models and runtime settings; qualified request and response adapters; working observers; declared case families, repetition and stopping rules; justified reference labels; exposure records; preserved baselines; and reports that separate requested actions from effects and useful completion. Model-family diversity broadens observation without establishing generality on its own.

Gemma 2 27B IT for continuity, Qwen3-8B for a thinking/non-thinking configuration comparison, Mistral-7B-Instruct-v0.3 for another tool-use implementation, and Llama-3.1-8B-Instruct for Meta-family coverage. Larger Qwen and Llama variants can follow. Selection does not imply downloaded weights, active capacity, license clearance or completed qualification. See official model cards linked in the presentation's companion source record.

No automatic scope expansion or new pricing is announced here. Existing pilot and catalog terms remain in effect. The agreed scope of work must identify which controls, models, cases and deliverables are included and whether a new development area is qualified for that engagement.

Because a specific, evidenced control failure or demonstrated limitation can change a deployment decision and direct a correction. Developers and operators build and maintain safeguards. ATBAE examines their stated behavior and control effectiveness within scope. The value is actionable evidence, including unfavorable evidence, with enough traceability to challenge and retest it.

Complete and challenge the new control implementation, qualify the real runtime and observers, execute retained multi-model studies, review reference quality and measurement assumptions, obtain independent review of the integrated result, and pass a documented release decision. Publication should show the exact qualified scope and remaining obligations. A polished presentation does not close any of these gates.

An agent with sufficient capability, tools and access may create iteration scripts, propose grading logic, retain feedback or prepare changes to prompts, software and runtime configuration while its base weights stay fixed. We explicitly include these external pathways in the examination scope. Whether the process constitutes RSI, improves capability or violates authority must be established separately. The existence of a loop is not sufficient evidence of any of those conclusions.

No. Model-authored tests can be useful development proposals. The control boundary is whether the subject can unilaterally make those tests, scores or rewards authoritative for accepting its own work. The examination should test legitimate proposals as well as attempted evaluator replacement, so an effective control does not simply block useful development.

Approval should bind the exact proposed change and its scope, with access to the evaluation basis, evaluator provenance and material limitations. A polished self-reported score alone does not establish independent review. The examination can test whether the recorded approval matches the activated candidate and whether the supporting evidence was altered. It cannot infer the quality of human understanding from a click alone.

No. The current development specimen uses coordinator-supplied grants and in-memory command and depth limits. It does not provide an operating-system kill switch, cryptographic authorization, durable distributed budgets or coverage of arbitrary external scripts. Those controls need their own implementation and qualification. Our examination design asks for evidence of limits across the actual job graph, including descendant agents and restarts where those are in scope.

Output symptoms alone cannot establish a training-data cause. The developing examination compares reference-grounded correctness, difficult-case performance, useful variation and recurring errors. Training lineage requires separate evidence. Recursive synthetic-data training, benchmark exposure, malicious poisoning, retrieval contamination and ordinary mistakes remain distinct hypotheses. No pollution-free certificate is offered.

A model can perform well on familiar work while failing on exceptions that matter to your use. We propose matched examinations of candidate revisions with explicit references, missing observations and control requirements. The aim is evidence for choosing, deploying or restricting a configuration. This new area still requires live qualification and independent review; it does not automatically enlarge the existing pilot or change its price.