TencentARC/GameHorizon

Yiran Wang^1,*,†, Xingyilang Yin^1,2,6,*, Junfu Pu^1,*, Guangzhi Wang^1,*, Kaifeng Li², Mingyu Ouyang^1,3, Huiqiang Sun^1,4,
Lingen Li^1,5, Cheng Cheng¹, Wangbo Yu¹, Honghao Chen¹, Xiaodong Cun^1,2,✉, Chi-Man Pun⁶, Zhiguo Cao⁴, Ying Shan¹

_{¹ARC Lab, Tencent   ²GVC Lab, Great Bay University   ³NUS   ⁴HUST   ⁵MMLab, CUHK   ⁶University of Macau
† Project Lead    * Equal Contribution    ✉ Corresponding Author}

• [2026.09.22] 📄 The paper and project page of GameHorizon Suite are released!
• We will progressively release the code, benchmark, and data starting around 2026.10.25. Stay tuned! ⭐

We are actively preparing the release of our code, benchmark, and data. Star the repo to follow our progress.

• 🌐 Project page
• 📄 Technical report / paper
• 🏆 Public leaderboard
• 🏗️ GameHorizon-Annotator — annotation toolkit & pipeline
• 📊 GameHorizon-Bench (offline) — MCQ suite & evaluation code
• 📊 GameHorizon-Bench (online) — online task environments
• 🎯 GameHorizon-Data — 5,000-hour AAA gameplay corpus

We introduce GameHorizon, a large-scale data and evaluation suite spanning multiple horizons and AAA games. It serves as a unified yardstick across a broad range of model types. It consists of three key components. GameHorizon-Annotator automatically produces a three-level pyramid of short-horizon operations, medium-horizon goals, and long-horizon strategies. GameHorizon-Data contains 5,000 hours of gameplay across 21 game titles, with temporally aligned videos, actions, and multi-horizon instructions. GameHorizon-Bench provides reproducible offline and stepwise online testing. The offline track contains thousands of standardized MCQs across three primary tasks and diagnostic variants. The online track tests order-dependent causal and order-flexible thematic tasks via the verifiable subtasks for failure localization.

Accuracies are reported as percentages. Overall is the mean across the three tasks.

Bold = best, underline = second-best. Models are grouped into four tiers by Overall score.

🥇 Tier 1 · Rank 1–11 # Model T1 T2 T3 Overall 🥇 GPT-6-Astra 69.4 79.6 91.5 80.2 🥈 Gemini 3.8 Flash 65.9 81.2 84.8 77.3 🥉 Gemini 3.7 Flash 66.2 80.1 83.9 76.7 4 Gemini 3.6 Flash 62.3 79.1 84.6 75.3 5 GPT-5.6 Sol 65.3 77.5 81.5 74.8 6 Kimi-K3 64.4 79.3 79.7 74.5 7 Gemini 3.5 Flash 64.9 77.4 80.8 74.4 8 GPT-5.5 64.6 78.9 76.2 73.2 9 Gemini 3.1 Pro 64.5 75.8 78.2 72.8 10 Doubao-Seed-2.1-Turbo 63.1 79.4 75.5 72.7 11 Doubao-Seed-2.1-Pro 62.9 76.7 76.6 72.1

Tier 2 · Rank 12–22 # Model T1 T2 T3 Overall 12 Doubao-Seed-2.0-Pro 59.2 74.9 80.0 71.4 13 Claude Fable 5 62.9 72.7 78.1 71.2 14 GPT-5.6 Terra 62.1 75.6 74.4 70.7 15 Qwen3.7-Plus 58.8 75.9 71.5 68.7 16 Qwen3.8-27B 59.9 77.4 68.1 68.5 17 GPT-5.6 Luna 61.6 70.7 71.1 67.8 17 Kimi-K2.6 61.2 73.2 69.0 67.8 19 Doubao-Seed-2.0-Lite 55.2 69.0 76.3 66.8 20 Gemma 4 31B-IT 53.8 68.6 76.6 66.3 21 MiniMax-M3 57.8 68.5 70.6 65.6 22 Qwen3-VL-235B-A22B-Thinking 58.2 71.6 65.9 65.2

Tier 3 · Rank 23–33 # Model T1 T2 T3 Overall 23 Claude Opus 4.8 59.1 62.1 73.1 64.8 24 Claude Sonnet 5 54.4 57.7 82.0 64.7 25 Step-3.7-Flash 57.5 71.3 63.4 64.1 26 Qwen3-VL-235B-A22B-Instruct 55.1 62.1 73.1 63.4 27 GLM-5V-Turbo 53.4 65.4 70.8 63.2 28 GPT-5.2 58.9 58.9 70.6 62.8 29 Qwen3.5-397B-A17B 53.3 60.1 73.5 62.3 30 BAGEL-7B-MoT 51.7 57.3 75.1 61.4 31 Qwen2.5-VL-32B-Instruct 54.8 58.0 69.7 60.8 32 Step3-VL-10B 55.4 64.7 61.7 60.6 33 SenseNova-U1-8B-MoT 48.2 59.2 67.5 58.3

Tier 4 · Rank 34–44 # Model T1 T2 T3 Overall 34 Qwen3.6-35B-A3B 53.3 52.9 66.3 57.5 35 Qwen3-Omni-30B-A3B-Instruct 52.5 55.6 63.7 57.3 36 GELab-Zero-4B-Preview 51.8 46.8 72.7 57.1 37 Qwen2.5-VL-7B-Instruct 52.0 54.3 63.7 56.7 38 InternVL3.5-8B 51.8 50.2 65.5 55.8 39 GLM-4.1V-9B-Thinking 54.9 54.8 55.3 55.0 40 GPT-4o 55.7 46.3 61.3 54.4 41 UI-TARS-1.5-7B 49.1 47.1 62.9 53.0 42 Ovis-U1-3B 44.7 39.6 59.2 47.8 43 InternVL3.5-2B 45.1 40.3 54.7 46.0 44 InternVL-U-4B 46.1 37.9 49.8 44.6

Each entry reports the success rate (%) with the number of passed tasks in parentheses.

The offline and online rankings show a clear positive association.

If you find GameHorizon Suite useful, please consider citing:

@article{GameHorizonSuite2026,
      title={GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay}, 
      author={Yiran Wang and Xingyilang Yin and Junfu Pu and Guangzhi Wang and Kaifeng Li and Mingyu Ouyang and Huiqiang Sun and Lingen Li and Cheng Cheng and Wangbo Yu and Honghao Chen and Xiaodong Cun and Chi-Man Pun and Zhiguo Cao and Ying Shan},
      year={2026},
      journal={arXiv preprint arXiv:2609.25001},
      url={https://arxiv.org/abs/2609.25001}, 
}