Dream-RSI: Recursive Self-Improvement
through Evolving Worlds

Tong Zheng^1,2, Xidong Wu^1✉, Zheng Zhang^1✉, Zhankui He³, Chaoyi Zhang¹, Benjamin Coleman³, Ruoqiao Wei¹, Di Bai³, Haolin Liu⁴, Rui Liu², Xue Wang¹, Yue Zhuan¹, Wang-Cheng Kang³, Renkai Xiang¹, Heng Huang², Xinwu Cheng¹, Yunsong Guo¹

¹Google  ·  ²University of Maryland, College Park  ·  ³Google DeepMind  ·  ⁴University of Virginia

^✉ Corresponding authors

• Sep 2026 — Preprint and project page released: paper (PDF) · dream-rsi.com

Progress in recursive self-improvement hinges on effective exploration. As self-improvement targets get harder, discovery stretches over thousands of proposal–evaluation cycles, and poor exploration wastes substantial computation on ineffective search directions. Managing and improving the exploration strategy is the bottleneck, and current systems face a dilemma: fixed strategies cannot adapt as search spaces scale, while optimizing the policy online means navigating a vast meta-search space under delayed and expensive feedback — assessing one exploration policy requires observing how it shapes an entire discovery process.

Our key insight is that accumulated discovery history can serve as a replay simulator over the realized search space. A completed discovery process already records a structured tree of past exploration decisions and their realized code-execution outcomes. An alternative policy can traverse that tree differently: different subsets of recorded branches, in different orders, with different parallel groupings and stopping decisions. Because all outcomes are already saved, evaluating it requires only reading past records — no rerunning of the discovery agent or the evaluator. By analogy with model-based RL and world models, the history becomes a world the agent can dream in.

Dream-RSI closes a self-improvement loop at the exploration layer. A lightweight orchestration layer makes exploration explicit and programmable — branching, parallel exploration, stopping — while leaving the underlying coding agent unchanged. ❶ the current policy drives online discovery and logs its traces; ❷ the recorded trees are converted into a reusable simulator pool; ❸ candidate policies are evaluated and refined by dreaming over that pool, which returns immediate, low-cost off-policy feedback. The improved policy is redeployed online, continuously expanding the pool.

Method, results and interactive walkthrough → dream-rsi.com

• History as replay simulator. We conceptualize completed discovery histories as replay simulators, which makes delayed exploration feedback reusable for efficient meta-exploration policy evaluation.
• Meta-layer RSI loop. Dream-RSI continuously collects discovery histories through online exploration, constructs replay simulators from them to refine meta-exploration strategies via dreaming, and redeploys the upgraded policy online.
• Empirical validation. Across algorithm engineering, mathematical optimization and GPU kernel engineering, Dream-RSI improves both discovery effectiveness and efficiency in several settings.

@article{zheng2026dreamrsi,
  title   = {Dream-RSI: Recursive Self-Improvement through Evolving Worlds},
  author  = {Zheng, Tong and Wu, Xidong and Zhang, Zheng and He, Zhankui and
             Zhang, Chaoyi and Coleman, Benjamin and Wei, Ruoqiao and Bai, Di and
             Liu, Haolin and Liu, Rui and Wang, Xue and Zhuan, Yue and
             Kang, Wang-Cheng and Xiang, Renkai and Huang, Heng and
             Cheng, Xinwu and Guo, Yunsong},
  journal = {arXiv preprint arXiv:2609.14858},
  year    = {2026}
}