See decision-model runtime support for the current Core ML model matrix and handoff notes.

Local computer use on Apple silicon. FluidUse reads a form in a running Mac app or browser through the Accessibility API, asks a small on-device model what belongs in each field, and types the answer into the real app. About 1 ms per decision on the Neural Engine, nothing leaves the machine.

The first model is CUA-S1-FORMS, a 706K-parameter form specialist from Cua, converted to Core ML and served by FluidAudio.

.package(url: "https://github.com/FluidInference/FluidUse.git", from: "0.3.0")

import FluidAudio
import FluidUse

let model = try await CuaS1FormsManager.load()
let driver = AccessibilityFormDriver(application: safari)   // any NSRunningApplication
let page = try await driver.snapshot()                       // fields, labels, values
let options = FormSchema.renderOptions(entities: profile)    // "fill Email: …", check, click, skip

for field in page.elements where field.isActionable {
    let context = FormSchema.renderContext(formTitle: page.title, element: field)
    let decision = try await model.score(context: context, options: options)
    // decode with FormSchema.decode, then driver.type / driver.click
}

WebFormDriver does the same for an embedded WKWebView. DocumentEntities turns a PDF or text file of Label: value lines into the profile; an optional PredeterminedAnswer sheet covers question-style fields the model does not decide.

laya (Convai Innovations, Apache-2.0) is an open Jev-style decision model: a 322M mmBERT encoder plus decision head that answers typed choice / score / noul questions about a text state with calibrated probabilities in one pass, no generated tokens. LayaManager runs the Core ML buckets from FluidInference/laya-coreml (128/256/512/1024 tokens, 32 options) on the Neural Engine: 3.7 ms per short question on an M5 Pro, about 7× faster than the ~27 ms upstream reports on an M1 Max GPU.

let laya = try await LayaManager.load()  // downloads the 128 + 512 buckets and tokenizer.json
let answers = try await laya.answer(
    state: "Customer: I was charged twice for order #4471 and nobody replies. Refund me today.",
    questions: [
        .choice("What does the customer want?", options: ["refund", "order status", "technical help"]),
        .score("How frustrated is the customer?", levels: ["calm", "annoyed", "angry"]),
        .noul("Is the customer likely to churn?"),
    ])
print(answers[0].selectedLabel, answers[1].expectedScore!, answers[2].noul!)

LayaManager.Configuration picks the buckets to load, their compute units (128 → CPU+ANE, longer → all units) and the weight precision (fp16, or e8 with an int8 embedding table at 30% less weight and the same accuracy); a prompt runs on the smallest loaded bucket that fits, and the largest one truncates the state on the right like laya's max_len. The tokenizer is a Swift port of the mmBERT/Gemma byte-fallback BPE and matches HuggingFace tokenizers on the conversion fixtures.

On laya's own published suites (3,899 questions rebuilt from upstream's scripts) the Core ML buckets match the PyTorch reference's accuracy on every suite (AG News 0.935, Emotion 0.537, MASSIVE-20 0.657, spam/phishing 0.993, guardrails 0.808, …) at 5.2 ms median per question versus 61.6 ms for PyTorch on the same Mac's CPU. Full tables in Benchmarks.md; the questions, reference answers, reports and conversion live in mobius models/computer-use/laya/coreml.

swift run -c release FluidUseLaya answer --state "…" --type choice \
    --instructions "What does the customer want?" --options "refund|order status|technical help"
swift run -c release FluidUseLaya tetris --shortlist --describe graded --pieces 200  # headless Tetris, P(clean) per landing
swift run -c release FluidUseLaya 2048 --precision lut8 --games 10
swift run -c release FluidUseLaya benchmark --suites <mobius>/benchmark/suites.jsonl --reference <mobius>/benchmark/reference-rows.jsonl
swift run -c release LayaTetrisDemo                              # SwiftUI: GLiClass/laya play Tetris
swift run -c release GLiClass2048Demo                            # SwiftUI: GLiClass plays 2048
swift run -c release Decision2048BenchDemo                       # SwiftUI: GLiClass vs laya at 2048

LayaTetrisDemo (Sources/LayaTetrisDemo, 25 s clip) defaults to GLiClass comparing the heuristic's two strongest legal landings in one call. Its policy control can instead run laya, which scores each surviving landing independently with "Is this a clean placement?", or the heuristic and random baselines. The window is a single column so it sits next to a terminal; the app prints decisions and model latency to stdout for presentations:

tmux new-session -d -s laya -c . && tmux send-keys -t laya 'sudo asitop' C-m
tmux split-window -v -l 22 -c . && tmux send-keys -t laya:0.1 'LAYA_DEMO_AUTOLOAD=1 swift run -c release LayaTetrisDemo' C-m
tmux attach -t laya

The current ten-seed capped run averages 3,667 pieces for GLiClass LUT8 and 2,874 for the corrected heuristic control. See Benchmarks.md for the exact policy, per-seed results, and the limits of comparison with the older laya measurements.

GLiNER2Manager runs the published base or multilingual classification head on device. Both packages use an L128 bucket, support up to eight labels, and quantize only the embedding table to eight bits. The manager downloads pinned, SHA-256 checked Core ML assets, source config files, and the matching Unigram tokenizer from base or multilingual on first use.

let classifier = try await GLiNER2Manager.load(variant: .base)
let answer = try await classifier.classify(
    text: "The rocket launched successfully.",
    task: "topic",
    labels: ["science", "sports", "politics"])
print(answer.selectedLabel, answer.probabilities)

Use .multilingual for the multilingual checkpoint. The base package is 291 MB and the multilingual package is 385 MB. Inputs that exceed 128 tokens raise an error. These packages contain the classification path; the native entity, relation, and record extraction heads are not exposed by this Swift manager. A ten-seed 2048 comparison with GLiClass is in Benchmarks.md.

KevFastManager runs Kev-0.8B (Qwen3.5 backbone, Apache-2.0) on the GPU through Core ML. One call reads the text and answers all of a request's questions (multiple choice, yes/no, or a score, with calibrated probabilities); questions that do not fit fall back to one call per question. The pinned, checksummed snapshot downloads from FluidInference/kev-0.8b-coreml on first use (~3.5 GB, macOS 15 / iOS 18).

let kev = try await KevFastManager.load(from: try await KevModelStore.ensure())
let answers = try await kev.answer(
    state: "Shoes arrived two weeks late and in the wrong size.",
    questions: [
        .choice("Which team should handle this?", options: [("returns", nil), ("shipping", nil), ("billing", nil)]),
        .noul("Does this need urgent human attention?"),
    ])
print(answers.map(\.best))

On an M5 Pro a short ticket with two questions takes 18 ms and a Wikipedia bio with twelve yes/no questions about 38 ms. swift run -c release KevGuessWhoDemo plays Guess Who over 80 Wikipedia people with it (Sources/KevGuessWhoDemo).

ClefVisionManager runs clef-vision-0.8b, a 0.92B image decision model distilled from Cloudflare's Clef-flash (Qwen3.5-0.8B backbone + Clef's joint schema head, Apache-2.0), on the GPU through Core ML. One call reads a state, any number of images and typed questions (yes/no, choice, score) and returns a probability for every option, like Clef / Jev. The pinned, checksummed bundle downloads from the Hub on first use (vision tower 377 MB, one ~960 MB language-model bucket per sequence length you load, 264 MB head per bucket, 508 MB embeddings; macOS 15 / iOS 18).

let clef = try await ClefVisionManager.load(from: try await ClefVisionModelStore.ensure(buckets: [1024]), buckets: [1024])
let result = try await clef.answer(
    state: ["task": "select every image that contains a cat"],
    images: [tile],  // CGImage
    questions: [("tile_1", .noul(instructions: "Is there a cat in image 1?"))])
print(result.answers[0].noul!)  // probability of "true"

Held-out test accuracy 92.2% (teacher 94.1%); a 3×3 "select all images with cats / dogs" grid scored one tile per record solves 90% of grids. On an M5 Pro one image and ~500 tokens take about 260 ms (vision 50 ms, language model 190 ms, head 14 ms). swift run -c release ClefVisionCheck parity Tests/FluidUseTests/Fixtures/clef-vision checks the Swift host against the Python reference.

InternDecisionManager runs Intern-Decision-0.8B (Shanghai AI Laboratory, Qwen3.5 backbone, Apache-2.0) on the GPU through Core ML. A request is a JSON state and up to 16 named questions (multiple choice, yes/no, or a score); every question is answered in one call from the logits before its <decision> marker, with the checkpoint's calibration temperature. The prompt is byte for byte the checkpoint's own compiler and chat template, so answers match its reference engine (0 of 60 differ on its published suites, max probability delta 0.004). The pinned snapshot downloads from FluidInference/intern-decision-0.8b-coreml on first use (~3.4 GB, macOS 14 / iOS 17). Text only; images are not exported.

let model = try await InternDecisionManager.load(from: try await InternDecisionModelStore.ensure())
let result = try await model.decide(
    state: ["channel": "email", "message": "Charged twice for my annual renewal. Refund the duplicate before Friday."],
    questions: [
        ("team", .choice("Which team should handle this?", options: [("billing", "Refunds."), ("technical", "Bugs.")])),
        ("frustrated", .noul("Is the customer frustrated?")),
        ("urgency", .score("How urgent is this?", levels: ["Low", "Medium", "High"])),
    ])
print(result.answers.map { "\($0.field): \($0.decision) \($0.confidence)" })

On an M5 Pro that request (319 tokens, three fields) takes 61 ms in the 320-token bucket; the checkpoint's own PyTorch path on the same Mac takes 150 ms (bf16). Buckets are 320, 512 and 1,024 tokens and the pass costs the bucket, not the request. swift run -c release InternDecisionCheck bench <model directory> reproduces the number.

A second snapshot, InternDecisionModelStore.ensure(.showdown), is the same model fine-tuned to pick Pokémon Showdown battle actions (FluidInference/intern-decision-0.8b-showdown-coreml), distilled from Intern-Decision-4B on a MacBook. It answers the same typed questions; given a battle state and the legal moves and switches as choice options it matches its 4B teacher (24-6 vs poke-env's max-power player, 9-21 vs its heuristic player over 30 battles). Buckets are 512, 640 and 1,024 tokens with int8 weights: about 0.85 GB in memory with one bucket in use, 1.9 GB to download, the same 90 ms per decision as fp16.

ShortReplyManager drafts one short reply to a social post with a sub-1B model: short-reply-0.6b-coreml, a Qwen3-0.6B fine-tune (Apache-2.0) in one 724 MB package whose prefill runs on the Neural Engine and whose decode runs on the GPU. About 190 ms per reply on an M5 Pro; drafts are for a person to review, nothing is posted automatically.

let replies = try await ShortReplyManager.load(from: modelDirectory)   // config.json, tokenizer.json, .mlpackage
let draft = try await replies.draft(for: "Finally passed my driving test on the third try.")
print(draft.reply, draft.timing.totalSeconds)                           // "Congrats on passing!" 0.19

ShortReplyDemo is a menu-bar app: open a post's reply box in any app (X in Chrome or Safari, Slack, Mail), press 9, and the draft is pasted into the box; 0 regenerates it. Sources/ShortReplyDemo/demo.sh --x launches it with a macmon + log terminal (Sources/ShortReplyDemo).

swift run -c release FluidUseDemo

Pick a running app, load a profile, press Fill form (or 9 from any app). Every model call is logged with its input, ranked options, and time. Requires Accessibility access for the launching terminal.

The on-device models, measured on the same Mac with checked-in reports: Benchmarks.md. CUA-S1-FORMS: 0.9 ms per decision on the Neural Engine, accuracy identical to PyTorch on the 24,370-row synthetic test. laya: 3.6 ms per short question, identical to PyTorch on laya's ten published suites, e8 buckets 30% smaller at the same accuracy. GLiClass Edge Apps v2: 1.61 ms FP16 or 1.81 ms LUT8 for a two-option L128 decision. Its Core ML packages and config are on Hugging Face, with the conversion pipeline in mobius PR #101. GLiClass demos download the selected Core ML bucket and tokenizer on first use. The loader reads the published config.json, checks each file against checksums.json, and caches the assets under ~/Library/Application Support/FluidUse/Models/gliclass-edge-apps-coreml. GLICLASS_MODEL_DIR or CLI --model-dir still selects a local directory. The Hub publishes FP16 at L128/L256/L512 and LUT8 at L128; other local precision variants require an explicit local directory.

The model matches a field to a value it is given. It does not read résumés, reason about dropdown options, or write free text. The harness handles observation, typing, selection, and an answer sheet; uploads and essays are left to the person. Submit is never clicked unless enabled.

Apache 2.0. CUA-S1-FORMS is MIT, from Cua.