2026-11-01 23:55 UTC
I’m running gpt4all on a thinkpad T470, and although is slow I rather use my own hardware for running LLMs
Replies (2)
-
@Gyrcas@piefed.zip 2026-11-02 00:52
Please don’t run LLMs at all…
-
@DaTingGoBrrr@lemmy.ml 2026-11-02 05:33
If you have enough regular ram and an nvme ssd you can run huge models on modest hardware using Colibrí. I got a Lenovo mini PC with 64GB of ram and a 500GB nvme and a 38B model runs with 11 tokens per second on my machine. The 38B model fits in my ram but if I had used a 120B parameter model then it would stream the rest of the model from my nvme disk.