A small plugin for Claude Code and Codex to diagnose and optimize end-to-end performance in PyTorch training and inference workloads.

Blog · Installation · Quick Start · Roadmap

🔥 Optimizations developed with the plugin have been merged into FunASR, FastVideo, gsplat, Ultralytics, Unsloth, and SGLang, with measured speedups of up to 3.6× on the tested workloads.

Provides a harness for end-to-end PyTorch job performance optimization, with:

• Workload context and profiling analysis to guide the agent’s reasoning
• Correctness checks to evaluate changes
• Guidance for the agent to ask questions that uncover users’ goals and constraints.

Optimizations can range from configuration changes to Python code and GPU kernels. The plugin works on individual training or inference jobs with a repeatable launch command, e.g. jobs submitted through Slurm.

Tell Claude Code or Codex:

Install the who-ate-my-flops plugin from
https://github.com/OpenPerfAgent/who-ate-my-flops
for this client. Follow the installation instructions in its README.

Run these commands in Claude Code:

/plugin marketplace add OpenPerfAgent/who-ate-my-flops
/plugin install who-ate-my-flops@openperfagent

Run these commands in your terminal:

codex plugin marketplace add OpenPerfAgent/who-ate-my-flops
codex plugin add who-ate-my-flops@openperfagent

Have your code repository, a command that launches the training or inference job, and access to an idle GPU environment ready. Start Claude Code or Codex in the repository you want to optimize.

Before running init, enable Auto mode in Claude Code or Approve for me in Codex so the agent can work with fewer interruptions.

Run init and give the agent your launch command and GPU environment. It will ask about your optimization goal, constraints, and correctness requirements, and record them in contract.md.

Choose diagnose for one investigation and a measured fix, or optimize to let the agent work through multiple improvements.

Wait for the agent to finish, then start with latest-report.md to review the results. Reports and experiment records are in workspace-who-ate-my-flops/.

workspace-who-ate-my-flops/
├── latest-report.md   # Results, correctness checks, and reproduction commands
├── benchmark.csv      # Baseline and optimization measurements
├── contract.md        # Agreed goals and constraints
├── records/           # Process records and planning notes
├── tools/             # Helper scripts
├── runs/              # Run outputs
└── commits/           # Profiling traces and diagnoses by commit

Questions, bugs, or feedback? Open an issue.

• Add skills for NVIDIA Nsight Systems.
• Further improvement of the alignment with user intent.
• Enable recursive delegation to kernel optimization agents when needed.

You can cite our blog post with:

@misc{zhao2026whoatemyflops,
  title        = {Beyond kernels: Letting agents optimize the whole {AI} job},
  author       = {Zhao, Hexu and Xi, Haocheng and Wang, Yichuan and Yin, Shaofeng and Lv, Zhaoyang and Feng, Haiwen and Li, Xiuyu and Panda, Aurojit and Li, Jinyang},
  year         = {2026},
  url          = {https://openperfagent.github.io/who-ate-my-flops/}
}

Copyright © 2026 Impossible, Inc.

Licensed under the Apache License, Version 2.0.