No, Jev has only been popular for a week—how did they already create a local version under 1GB??
A developer has ported the open-source alternative Laya native to Apple MLX, creating Laya-MLX—
The 421M parameter version peaks at only 943.6 MiB of MLX memory usage.
Switching to the 322M multilingual version dropped it further to 687.6 MiB.
No need for cloud APIs, PyTorch, or Transformers runtime—just download the model to your Mac and run it directly.
And although it's small, it's surprisingly fast.
On the M3 Max, Laya-MLX achieves a fastest decision time of just 7.39 ms for the short decision P50; the 421M version is only 13.42 ms.
Even when feeding in 50 questions at once, the throughput of the 322M multilingual version can still reach 395 questions/s.
Open-source Jev
Everyone should be familiar with Jev by now.
Its biggest difference from a standard LLM is that it eliminates the most time-consuming task—text generation—so it doesn't chat, write code, or produce articles... it focuses solely on making decisions.
Bert: Huh? I feel like I’ve heard this job description before.
Without the step of generating tokens one by one, the inference speed naturally differs from that of traditional LLMs.
It was precisely through this straightforward and aggressive approach that Jev quickly went viral after its release.
Immediately afterward, an open-source alternative appeared.
Laya, a multilingual, non-autoregressive System 1 decision model.
Similar in approach to Jev, it does not expend computational power on generating text but directly outputs decision results for structured problems, having already garnered 10.4k stars on GitHub.
It currently primarily handles three types of tasks:
choice: select one from several options; score: assign a rating based on given criteria; noul: determine the probability that something is True.
Feed in a state along with the question to be evaluated, and obtain the result in a single forward pass.
Laya currently has three main checkpoints.
The English version is based on ModernBERT-large, with 421 million parameters;
The multilingual version is based on mmBERT-base, with only 322 million parameters and supports over 100 languages;
There is also a 421M version tailored for the typed-decisions workflow.
The model itself has only a few hundred million parameters and is already very fast.
The original Laya takes approximately 33 ms per question on a Tesla T4; when processing in batches, it averages 7.2 ms per question, achieving a single-card throughput of up to hundreds of questions per second.
The third-party Jev P50 latency referenced in the project is between 236 and 276 ms, while Laya's own measurement for a single query is 32.8 ms—approximately 7 to 8 times faster.
In 2,000 decision tests on yped-decisions, the specially fine-tuned Laya version achieved an accuracy rate of 76.6% (previously, Jev’s public result was 72.7%).
The open-source version is available, and it's not slow.
Immediately after, people began wondering if it could be made even smaller.
Then Laya-MLX arrived.
Can run locally with 1GB
Laya-MLX powers Laya's inference stack.
Developers have been able to deploy the original Laya themselves, but primarily run it within the PyTorch and Transformers environment.
mizorewww reimplemented the full neural network architecture of Laya on the Apple MLX framework, enabling the model to run natively on Apple Silicon.
PyTorch and the Transformers runtime can both be removed, and there is no need to request cloud APIs.
Moreover, after switching the inference stack, the original results must still match.
Laya-MLX validated the three checkpoints using both FP32 and FP16, and matched the original Laya's selected answers on all 63 validation questions.
Three models, two precisions, with a total of 378/378 comparisons passed, and no inference drift occurred.
The test machine is an M3 Max with a 40-core GPU and 128GB of unified memory.
When handling a short query, the 421M English version peaked at 943.6 MiB of MLX allocation; the 322M multilingual version further reduced this to 687.6 MiB.
But the speed hasn't slowed down.
On the same M3 Max, the 421M version processes a short query with a P50 latency of 13.42 ms; the 322M multilingual version has a P50 latency of 7.39 ms.
The P95 values are also only 13.92 ms and 7.79 ms, respectively.
If you batch 50 questions at once, the multilingual throughput can directly reach 395 questions/s.
Moreover, these times are not the bare latency of the model's core operators.
The project's timing range includes prompt preparation, tokenization, tensor construction, synchronized inference, calibration, and final result formatting, excluding only model loading time.
In other words, the 7.39 ms measures the end-to-end latency of a short decision from entry to result.
Laya-MLX can make decisions at a speed of 60 times per second while playing Snake.
At each step here, Laya is actually invoked to make a new decision based on the current state, while an external cycle safety layer corrects any dangerous actions.
After enabling the compiled and prefix reuse optimizations tested in the project, run 2,400 steps consecutively on the same M3 Max—
75.40 moves/s, 0 deaths.
Throughout the process, the security layer intervened twice, with action adjustments made only in minimal scenarios.
Three-line operation
Interested users can deploy locally on Apple Silicon Macs.
The project currently requires Python 3.11+ and macOS 14+.
After installing the dependencies, you can directly import the library and load the pre-converted checkpoint:
pip install laya-mlx import laya_mlx as laya agent = laya.load("aac6fef/laya-mlx")
Next, just pass the state and the issue to agent.predict().
The first run will automatically download the checkpoint; subsequent inferences can be performed entirely locally.
Of course, there are costs to having less.
421M English version context has only 512 tokens;
The 322M multilingual version and the 421M typed-decisions version each have only 1024 tokens.
With only 1k context, it must simultaneously accommodate state, instructions, and options, making it better suited for short inputs, high-frequency calls, and decision tasks with well-defined outputs.
The shortcomings objectively exist, but they can't stop open-source experts from jumping on it overnight—yes, we're already working on the next version of Dream!
Jev: I've only been popular for a week...
Project address: https://github.com/mizorewww/laya-mlx
This article is from the WeChat public account "Quantum Bit," authored by Wen Le.