amenome@anemone@ebiverse.social: @lain@fediffusion.art you running it yourself?

Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

lain@lain@fediffusion.art (2026-09-10 15:19:20)
It really is a “big model”. And it’s actually smaller than the old llama 70b models where it counts (VRAM) and much faster too.
---Attachments---
image: https://fediffusion.art/media/75/94/3a/75943a7be5d4ace1c9d8320d27b87b1a1b9ed0f541ecd17ee4353b82bd1fc6a5.jpg
---Reply---
amenome@anemone@ebiverse.social (2026-09-10 15:23:21)
@lain@fediffusion.art you running it yourself?


---Replies---

lain@lain@fediffusion.art (2026-09-10 15:27:39)
@anemone yes, on a strix halo. at the moment you kind of need to hunt down patched llama.cpp repos to get good performance, but when it works it works really well (qwen 3.8 flash next, that is)