Gesture-controlled 3D models made of glowing light particles, running entirely in your browser.
Snap your fingers in front of your webcam and up to 250,000 GPU particles swirl into existence. Make a fist and they form the Eiffel Tower, a beating heart or a V8 engine. Open your hand and the model morphs into the next one, or explodes into a labelled diagram of every part. No mouse, no controller, no install beyond Node.js.
You need Node.js 18 or newer and a webcam (optional: everything also works with the mouse and keyboard).
git clone https://github.com/AkbarSheikh-debug/wondersnap.git
cd wondersnap
npm install
npm startThen open http://localhost:5173 in Chrome or Edge and click Start with camera, or Continue without camera to drive it with the on-screen buttons and keyboard.
The browser only allows camera access on
localhostorhttps, which is why the app comes with its own tiny local server. To use a different port:npm start -- 8080.
| Layer | What it does |
|---|---|
| Hand tracking | MediaPipe Hand Landmarker tracks up to two hands (21 landmarks each) from the webcam, fully on-device |
| Gesture recognition | Custom classifiers turn landmarks into poses (fist, open, point, pinch, peace), a finger-snap detector, hand twist/tilt and two-hand zoom, fed through a debouncer and a state machine |
| Particle engine | A hand-written WebGL2 renderer. Particle physics runs on the GPU with transform feedback: no Three.js, no game engine, no framework |
| Models | 33 procedural models built from real measurements and sampled into point clouds, with named parts that can explode, glow and be pulled out |
| Server | A zero-dependency Node.js static server (server.mjs) |
Nothing is sent anywhere: the video never leaves your machine.
| Gesture | What it does |
|---|---|
| 🫰 Snap | Summon the particles, or dissolve the current model |
| ✊ Fist | Form the wonder, organ or machine |
| ✋ Open hand | Wonders morph to the next one. Organs, engines and vehicles explode: how far you open your hand sets how far the parts fly apart, and closing it puts them back together |
| 🔄 Twist / raise your hand | Turn and tilt the formed model |
| ☝️ Point | Hold your finger on a part to select it. It glows, and a card explains what it does |
| 🤏 Pinch | Pull the selected part out toward you; pinch again to put it back |
| 🙌 Two hands | Move them apart or together to zoom |
| ✌️ Peace | Jump to the next model |
| Key | Action |
|---|---|
Space |
Snap |
F / O / V |
Fist / open hand / peace sign |
← → |
Previous / next model |
E, ↑ ↓, mouse wheel |
Explode amount |
+ - 0, ctrl + wheel |
Zoom |
C |
Camera on/off |
L |
Part labels |
R |
Auto-rotate |
G |
Hand rotation on/off |
X |
Cut-away cross-section (, and . nudge the plane) |
Q |
Quiz mode |
M |
Voice commands and read-aloud |
K |
Record a video |
D |
Play the demo |
I / Esc |
Describe / deselect the selected part |
H |
Help |
Click a part to select it, drag to rotate.
- Beating heart and breathing lungs. The heart contracts in a lub-dub rhythm at 72 bpm, the lungs inflate every 4.5 s, and pulses of light travel through them like blood or air.
- Exploded views with named parts. Every part has a leader-line label saying what it does.
- Quiz mode. "Find: Hippocampus": point at (or click) the right part. Five questions, with a score.
- Voice control. Say "show me the heart", "open it up", "where is the right atrium", "zoom in", "quiz" and more. Parts are read aloud with speech synthesis (Chrome or Edge).
- Cut-away. A cutting plane follows your hand and reveals a glowing cross-section.
- Recording. Save a WebM video of the scene.
| Category | Models |
|---|---|
| Wonders (11) | Turtle Tower, Eiffel Tower, Statue of Liberty, Burj Khalifa, Great Pyramid, Colosseum, Leaning Tower of Pisa, Taj Mahal, Big Ben, Christ the Redeemer, Sydney Opera House |
| Anatomy (10) | Brain, beating Heart, Kidney, breathing Lungs, Eye, Ear, Tooth, Skull, Skeleton, Human Body (skin, organs, nerves, arteries, veins, skeleton) |
| Biology (2) | DNA double helix that unzips, Animal cell |
| Engines (4) | Inline-4, Supercharged HEMI V8, Turbofan jet, 9-cylinder radial |
| Vehicles (4) | Sports car, Motorcycle, Airliner, Saturn V (with stage separation) |
| Machines (2) | Mechanical wristwatch, EV battery pack (280 cells, busbars, cooling, BMS) |
The Gesture sensitivity control in the status panel defaults to Standard, which preserves the original gesture thresholds. More forgiving slightly relaxes static hand-pose recognition and openness. Snap and pinch detection are unchanged, and the selection lasts only for the current page session.
| Option | Effect |
|---|---|
?n=250000 |
Particle count |
?model=12 |
Start on a given model |
?autostart=camera / ?autostart=nocamera |
Skip the start screen |
?trails=0 |
Turn off particle trails |
?dpr=1 |
Force the device pixel ratio (useful on slower GPUs) |
39 end-to-end and unit tests with Playwright, driving the real app with synthetic hands on a deterministic clock.
npx playwright install chromium # one time
npm test| Spec | Covers |
|---|---|
app.spec.js |
The full gesture story, every model, explode/contract, keyboard, wheel, tabs, demo, phone layout, hand twist and tilt |
features.spec.js |
Heartbeat and breathing, two-hand zoom, point-to-pick, pinch-to-pull, quiz, voice commands, cut-away, video recording |
camera.spec.js |
Real getUserMedia to MediaPipe on Chromium's fake webcam, plus the camera-denied fallback |
gpu.spec.js |
The GPU physics shader matches its CPU twin to ~1e-7 in every mode |
logic.spec.js |
Pose classifiers, snap detector, debouncer, state machine, controller, voice-command parser |
models.spec.js |
Every model is deterministic, finite and fast, with real measurements and correctly exploding parts |
index.html, styles.css page and styles
server.mjs zero-dependency static server
models/ MediaPipe hand landmark model
src/app.js render loop, explode, zoom, picking, quiz, cut-away, HUD, labels, demo
src/hands.js webcam + MediaPipe Hand Landmarker
src/features.js voice commands, read-aloud, video recorder
src/gl/ WebGL2 shaders and renderer (transform-feedback physics)
src/logic/ gestures, state machine, controller, CPU physics twin
src/lib/ vector math, samplers, procedural shapes
src/models/ wonders, anatomy, biology, engines, vehicles, machines
tests/ Playwright specs
- Camera doesn't start: open the app via
http://localhost:5173, not by double-clickingindex.html, and allow camera access when the browser asks. - Hand tracking never loads: run
npm installfirst; the tracking runtime is served fromnode_modules. - Low frame rate: try
http://localhost:5173/?n=100000&dpr=1. - Port already in use:
npm start -- 8080and openhttp://localhost:8080.
MIT © 2026 Akbar Sheikh