Today | News | Books | Recipes
Notes | QuickTake | Wiki | Browse
Maps | Reference | Reddit | YouTube
Chat | Games | About Back to HN Laya (OS Jev) on Mac M4 CoreML Offline (45 decisions per second)by putna | 71 points | 9 comments | 2026-09-20 10:58:52 Central Open Source Link | Read Source Here Open on Hacker News Commentsaltano
How much memory does this use of the test machine's (M3
Max) 128 GB unified memory?
| > WASDx
The model is only 0.3B params so probably not much.
| PaulRobinson
Local LLMs are the future, and one of the reasons I think
the data centre furore is just going to end in a market
crash.LLMs that can reliably be used for control problems
are the future, and I think classic/deep RL has generally
been overlooked for years for a whole host of problems by
wider industry because it felt inaccessible. The first
thing I thought of when I saw Jev (and then Laya), was
"this might move the needle in a really, really
interesting way".Local LLMs that can reliably be used for
control problems smash through a lot of barriers I'm
interested in, and this intrigues me a lot. Guess I'm
about to become a big Laya fan if it can run on this kind
of hardware to this performance.
| > ipsi
The future for whom? The general public? Not a chance,
no way, not unless it's able to run on a phone
(anywhere from 20-40% of internet users, world-wide,
are phone-only).For companies? I think that's a lot
more plausible, as that's mostly just a question of
money - is it cheaper to run and administrate our own
models, or outsource that?For technically inclined
users? I think that's unlikely unless they're able to
operate on relatively cheap hardware while still being
just as good as the hosted models. And by that I don't
mean "a mac studio," that's far more money than I
think is reasonable. A single RTX 5080, maybe, once
memory prices start to drop.
| > frag
that's not a local LLM. If it's local, it doesn't
matter in this case. Laya is a System 1 "AI", namely
works like a classifier, given a state and questions,
it shoots probabilities for each. I publish an episode
tomorrow about Laya and Typesafe AI on
https://www.youtube.com/@DataScienceatHomeStay tuned
;)
| > bigyabai
It won't. Laya is a finetuned version of Google's BeRT
model, which is almost 10 years old right now.If BeRT
had any potential to disrupt the datacenter buildout,
it already would have.
| frag
Great job!
Did you finetune your own Laya for the snake game or what?
| tentacleuno
This looks like a local AI model playing Snake -- is that
correct? The article offers no explanation.
| > ryuuseijin
The linked github project [1] contains more
information.[1]:
https://github.com/mizorewww/laya-coreml
|
|