Run the biggest language model you can on the smallest hardware you own.
TV boxes, old phones and tablets, single-board computers and routers all have a CPU, some RAM and flash storage that sit idle most of the day. smolmaxxing finds the biggest model each of them can really run, then pushes it further with custom kernels, careful memory use and weights streamed from flash. Every number in this repo was measured on the device itself.
| 📺 Xiaomi TV Box S (3rd Gen) | 📱 iPad Air (2013) | 📲 OnePlus Nord CE 2 (2022) | 📺 BuzzTV XR4500 | 📖 Kindle Paperwhite 3 (2015) | 📲 Realme 5 Pro (2019) | |
|---|---|---|---|---|---|---|
| Chip | Amlogic S7D, 4× Cortex-A55 | Apple A7, 2 cores | MediaTek Dimensity 900, 2× Cortex-A78 + 6× A55 | Amlogic SM1 (S905X3 class), 4× Cortex-A55, 32-bit | NXP i.MX6 SoloLite, 1× Cortex-A9, 32-bit | Snapdragon 710, 2× Kryo 360 (A76-class) + 6× A55, no dotprod |
| RAM | 1.9 GB (about 1 GB usable) | 1 GB (about 700 MB usable) | 7.4 GB (about 3 GB free) | 2 GB (about 1.3 GB free) | 512 MB (about 440 MB free) | 6 GB (about 3.3 GB free) |
| OS | Android 14, no root | iOS 12.5.7, jailbroken | Android 13, no root | Android 9 (32-bit), no root | Jailbroken, own Alpine Linux userspace | Android 10, no root |
| Everyday model | Qwen3.5-2B at 7.7 tok/s | Bonsai-1.7B (1-bit) at 24-26 tok/s | Qwen3.6-35B-A3B at 4.9-5.7 tok/s | Qwen3-0.6B (Q8_0) at 1.7 tok/s | SmolLM2-135M at 1.7 tok/s | Qwen3-0.6B (Q8_0) at 14.6 tok/s |
| Biggest model | Ling-mini-2.0, 16B, at 5.1 tok/s | Ling-mini-2.0, 16B, at 2.6-4.5 tok/s | Qwen3-Next-80B-A3B at 2.6-3.1 tok/s | Qwen3.5-2B at 0.4 tok/s | Bonsai-4B (1-bit) at 0.03 tok/s, from flash | Qwen3-1.7B (Q8_0) at 5.7 tok/s |
| Full results | docs/tv-box.md | ipad-air/README.md | docs/phone.md | docs/buzztv-xr4500.md | docs/kindle.md | docs/realme-5pro.md |
The TV box and the iPad run 8B and 16B mixture-of-experts models on about 1 GB of RAM, and the phone runs 35B and 80B ones with about 3 GB free. They all keep only part of each model in memory and read the rest from flash as it's needed. The BuzzTV box, measured by a contributor, is the first 32-bit device: the same cores as the Xiaomi box, but a 32-bit Android that rules out the arm64-only kernels. The Kindle runs a chatbot and a coding assistant on one 32-bit Cortex-A9 with 512 MB of RAM. The Realme 5 Pro, also measured by a contributor, is the first device without the dot-product extension: its A76-class cores carry fp16 but not dotprod, so it runs the plain-NEON build and its numbers show what that costs.
The chart below compares the two 1 GB devices size class by size class; the phone plays in a bigger league, and its results are in docs/phone.md.
More devices (single-board computers, routers, more phones) are coming. The techniques aren't tied to one box: the 1-bit kernel needs only basic NEON, and the flash streaming works on any Linux or Android device with more model than RAM.
Got a small device? Run a model on it and submit your numbers: we'll add them to the results.
All of it is a patch on llama.cpp. The main ideas:
- Kernels built for the chip. A 1-bit lookup-table kernel, repacked 2-bit and ternary formats for the Cortex-A55, and versions of them for the A7, which has no dot-product instructions.
- Less RAM per model. Most of a small model's vocabulary is rarely used, so we drop those output rows. On the TV, a "headless" mode also stops the launcher and Google services while a model runs.
- Experts paged from flash. A mixture-of-experts model only uses a few experts per token. We keep the popular ones in RAM, read the others from flash on demand and prefetch the next layer's. This is how 8B and 16B models run on 1 GB, and 35B and 80B models on a phone.
- Streaming for dense models bigger than RAM, at the flash's full speed.
docs/how-it-works.md has every technique with its measured gain.
The quickest start is a ready-made model file from Hugging Face. The TV box and iPad files need the patched llama.cpp (stock llama.cpp can't load their pruned heads); the phone files load anywhere, but their speeds need the patch's expert pager.
- TV box or another ARMv8.2 Android device: docs/tv-box-setup.md
- 32-bit ARM (armv7) Android device: docs/buzztv-xr4500.md
- Jailbroken iPad on iOS 12: ipad-air/README.md
- Android phone with 8 GB of RAM: Qwen3.6-35B-A3B (12.3 GB) and Qwen3-Next-80B-A3B (27.6 GB, two files), set up as in docs/phone.md.
- ARM device without the dot-product extension (
asimddpmissing from/proc/cpuinfo): docs/realme-5pro.md - All model files and how to rebuild them: docs/models.md
| Page | What's in it |
|---|---|
| docs/tv-box.md | device #1: every model run on the TV box, speed and quality |
| ipad-air/README.md | device #2: the iPad Air results, setup and scripts |
| docs/phone.md | device #3: the OnePlus phone, from 16B to an 80B model |
| docs/buzztv-xr4500.md | device #4: a 32-bit Android TV box, submitted by a contributor |
| docs/kindle.md | device #5: a jailbroken Kindle Paperwhite 3 on one Cortex-A9, and an fp16 fix for llama.cpp on older 32-bit Arm |
| docs/realme-5pro.md | device #6: an Android phone whose A76-class cores lack the dot-product extension, and what that costs |
| docs/how-it-works.md | the kernels, memory tricks, expert pager and streaming engine |
| docs/models.md | downloads for all three devices, rebuild targets and checksums |
| docs/lessons.md | what we learned, including what didn't work |
| EXPERIMENT.md | the full lab notebook: 171 experiments, failures included |
| Path | What |
|---|---|
patches/ |
every change to llama.cpp for the TV box and the phone (the iPad's patch is in ipad-air/patches/) |
tools/ |
model rebuilding, GGUF rewriting, benchmarks and kernel checks |
tv-llm-ui/ |
the live token-stream chat page and its launcher |
ipad-air/ |
device #2: iOS 12 build, smolchat, deploy and benchmark scripts |
bench/ |
every chat benchmark (results.jsonl) and the raw logs |
data/ |
vocabulary lists and evaluation texts |
tiny-llm-web/ |
the first experiment: a 260K-parameter story model |
zigcc/ |
zig wrappers for cross-compiling static aarch64 binaries |
zigcc32/ |
the same for 32-bit ARM (armv7) devices |
kindle/ |
device #5: the Kindle build, the fp16 patch and the benchmark scripts |
Other people running language models on hardware that was never meant for them:
| Project | Hardware | What it runs |
|---|---|---|
| On-chip LLM on a KV260 | AMD Kria KV260 FPGA ($250) | a 3.16M-parameter INT4 transformer held entirely in on-chip memory, about 60K tok/s |
| ps2-llm | PlayStation 2 (294 MHz MIPS, 32 MB RAM) | a 10M-parameter instruct model |
| iiama2.c | Nintendo Wii (729 MHz PowerPC, 88 MB RAM) | llama2.c with stories15M, 61.9 tok/s on the GPU |
| esp32-ai | ESP32-S3 microcontroller | a 28.9M-parameter model at about 9.9 tok/s |
| go-pherence | CPUs with AVX2, NEON or RVV, and NVIDIA GPUs | pure-Go inference for language, speech and vision models |
| Ryan Seddon on X | Sega Dreamcast, via Windows CE | a language model running under the Dreamcast's Windows CE |
MIT. The llama.cpp patches are derivatives of llama.cpp (MIT). Model weights keep their own licenses, and the evaluation texts are Wikipedia excerpts (CC BY-SA 4.0).
Built on llama.cpp and llama2.c. Models from Qwen, unsloth (GGUFs and importance matrices), PrismML (Bonsai), inclusionAI (Ling-mini-2.0), tablinum (Moon-Beam), bartowski and mradermacher (GGUFs), Google (Gemma) and stories260K. The iPad is jailbroken with Amethyst. The banner uses Archivo (SIL OFL) and is rendered by tools/banner/render.sh. The 🤏 logo is the pinching-hand emoji from Google's Noto Emoji (Apache 2.0).

