With only 8 GB of memory, you're going to be running a really small quant, and i...

		yjftsjthsd-h 64 days ago \| parent \| context \| favorite \| on: Ggml.ai joins Hugging Face to ensure the long-term... With only 8 GB of memory, you're going to be running a really small quant, and it's going to be slow and lower quality. But yes, it should be doable. In the worst case, find a tiny gguf and run it on CPU with llamafile.