If you have enough regular ram and an nvme ssd you can run huge models on modest hardware using Colibrí. I got a Lenovo mini PC with 64GB of ram and a 500GB nvme and a 38B model runs with 11 tokens per second on my machine. The 38B model fits in my ram but if I had used a 120B parameter model then it would stream the rest of the model from my nvme disk.