Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

Reply to @anemone@ebiverse.social
lain@lain@fediffusion.art (2026-09-10 15:27:39)
@anemone yes, on a strix halo. at the moment you kind of need to hunt down patched llama.cpp repos to get good performance, but when it works it works really well (qwen 3.8 flash next, that is)
---Reply---
Trogdor the Burninator :trogdor:@trogdor@rayci.st (2026-09-10 15:33:13)
@lain @anemone I've found that the atomicchat quants of 3.8 flash is the most performant for me. I'm using the main branch of llama.cpp though. I'm using a 16 GB GPU with 64 GB DDR5 with the ngrams offloaded to nvme though. 120K context

---Replies---

lain@lain@fediffusion.art (2026-09-10 15:34:25)
@trogdor @anemone what kind of performance do you get with that?