Mirai is positioning as a seed horizontal AI infrastructure play, building foundational capabilities around micro-model meshes.
With foundation models commoditizing, Mirai's focus on domain-specific data creates potential for durable competitive advantage. First-mover advantage in data accumulation becomes increasingly valuable as the AI stack matures.
Deploy and run models of any architecture directly on user devices. Fastest inference engine built from scratch for Apple devices.
Combination of a hardware‑aware, proprietary runtime optimized for Apple Silicon (uzu) plus a conversion/optimization pipeline that includes trained draft models for speculative decoding, end-to-end validation, and commercial SDKs/bindings—enabling significantly faster and reliable on‑device inference for iOS/macOS apps.
Mirai implements a multi-model strategy: a lightweight 'draft' model is trained to predict tokens ahead (speculative decoding) and a heavier verifier model validates predictions. They also describe routing between on-device models and larger cloud models for complex requests, indicating orchestration of specialized small models and larger models for performance and cost trade-offs.
Cost-effective AI deployment for mid-market. Creates opportunity for specialized model providers.
Mirai provides explicit output constraints and validation layers: GrammarConfig / schema-driven generation (and Zod integration) ensure structured outputs, while a validation pipeline (layer-by-layer comparisons and correctness checks) enforces fidelity and flags deviations. These act as guardrail layers ensuring safe, well-formed, and correct outputs.
Accelerates AI deployment in compliance-heavy industries. Creates new category of AI safety tooling.
The product provides SDK snippets, schema-driven prompts, and parsing helpers that let developers drive structured outputs (e.g., JSON) from natural-language prompts and then map them back into typed code constructs (Zod, Swift structs). This bridges NL to structured code/data, though the primary focus is structured-output generation rather than full program synthesis.
Emerging pattern with potential to unlock new application categories.
While not directly claiming proprietary domain datasets, Mirai offers bespoke conversion/optimization and custom-pipeline support for clients' unique models and layers. This bespoke engineering and per-customer conversion/validation pipeline can functionally create a competitive moat around customers' domain models and specialized assets.
Unlocks AI applications in regulated industries where generic models fail. Creates acquisition targets for incumbents.
Mirai builds on Qwen/Qwen3-0.6B, MLX, llama.cpp, leveraging OpenAI and Qwen infrastructure with @trymirai/uzu, GrammarConfig in the stack. The technical approach emphasizes hybrid.
They support converting user-provided fine-tunes and adaptations; also train lightweight "draft" models for speculative decoding. Specific tuning methods (LoRA / full fine-tune) are not stated. — Not specified in the content; only the existence of draft model training and support for 'your own fine-tunes and adaptations' are mentioned.
Basic hybrid orchestration: single API surface that can target either on-device or cloud models; no evidence of complex multi-model chaining, agent orchestration, or tool calling beyond API-level cloud fallback.
Developer-controlled hybrid routing: local on-device inference by default with an option to choose a cloud model for complex requests via the same API (client-side selection exposed by SDK).
Proven entrepreneurs who built and scaled consumer AI leaders; involved with Reface and Prisma.
Previously: Reface, Prisma
Founders have hands-on experience building scalable consumer AI products and optimizing models for on-device inference on Apple Silicon, aligning with Mirai's mission. Strong fit, though public founder identities are not disclosed in provided content.
developer first
Target: developer
api pricing
hybrid
On-device AI inference for Apple Silicon devices, enabling local model execution with optimized runtime and a model registry
Training a dedicated draft model per target model and integrating it as a first-class optimization (with presets) is more engineered than ad-hoc speculative decoding—it formalizes the draft/verify split and exposes it in the SDK, enabling measurable speedups (claimed up to 2x).
Per-layer comparison as part of a production conversion pipeline demonstrates strong emphasis on correctness—this goes beyond “black-box” output checks and helps ensure optimization steps (like fusion or quantization) preserve functional behavior.
Mirai operates in a competitive landscape that includes llama.cpp, MLX (Apple/MLX-like runtimes), MLC (mlc.ai / MLC-LLM & related projects).
Differentiation: Mirai claims a proprietary, Rust-based runtime (uzu) tuned specifically for Apple Silicon, an end-to-end conversion/optimization pipeline (lalamo), speculative decoding with a trained draft model, and an integrated SDK/Platform with official language bindings and validation tooling. Mirai advertises better performance on supported models and tighter integration for iOS/macOS apps.
Differentiation: Mirai explicitly claims to outperform MLX on supported models, emphasizes a full-stack solution (conversion + optimization + runtime + SDK + platform), and provides custom converters, layer-level validation and draft-model speculative decoding. Mirai also offers hybrid cloud fallbacks and developer bindings (Swift/TS/etc.).
Differentiation: Mirai presents itself as a tightly engineered, proprietary stack optimized for Apple Silicon with production-ready SDKs, automatic converters for many model families, explicit support and services for custom/non‑standard layers, and commercial platform features (API keys, licensing).
End-to-end, hardware-targeted stack: They built a full pipeline (converter, optimizer, runtime) in under a year with Rust core + language bindings (Swift, TypeScript). The exported model format is modular (unified blocks) and paired with a reference implementation to enable byte-for-byte correctness checks — an unusual, engineering-heavy choice for a small startup.
Trained 'draft' models for speculative decoding integrated into the conversion pipeline: instead of opportunistically using an off-the-shelf small model as a proposal network, Mirai trains a lightweight draft model matched to the target model so it can predict multiple tokens ahead and have the main model verify them in one pass. They advertise up to ~2x generation speedup. Packaging draft-model training as part of optimization/validation is technically novel for a conversion/edge stack.
Per-layer correctness and validation tooling baked into conversion: their pipeline runs layer-by-layer comparisons against a reference output and flags deviations, exposing subtle numerical/quantization bugs early. That level of automated fidelity measurement (not just end-to-end sample checks) is uncommon in product-focused inference toolkits.
Hardware-specific micro-optimizations surfaced into API primitives: they explicitly call out RoPE precomputation, layer fusion, and context-length-based RAM trade-offs, indicating they expose or exploit lower-level runtime invariants to trade memory vs compute on Apple Silicon — not just 'quantize-and-run'.
Same-session hybrid on-device/cloud model routing: they provide a unified API so you can run simple requests on-device and escalate complex ones to a cloud model without changing the session semantics. That requires careful state sync, prompt/context consistency, and deterministic sampling controls to avoid user-visible divergence.
If Mirai achieves its technical roadmap, it could become foundational infrastructure for the next generation of AI applications. Success here would accelerate the timeline for downstream companies to build reliable, production-grade AI products. Failure or pivot would signal continued fragmentation in the AI tooling landscape.
“Mirai builds the fastest on-device inference engine for Apple Silicon.”
“Sometimes you want to create a complex pipeline where some requests are processed on-device and the more complex ones are handled in the cloud using a larger model.”
“In under a year, a 14-person team built a full stack, from model optimization to a proprietary runtime, outperforming MLX and llama.cpp on supported models.”
“We’re making local inference practical, fast, and reliable for real products.”
“The main entry point to the SDK. Use it to refresh the registry, download models, and create inference sessions.”
“Conversion and optimization toolkit Convert and optimize your model for iPhone, iPad and Mac.”
