Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

kaiaskutes@kaiaskutes@fediffusion.art (2026-10-08 19:12:54)
I asked Muse 1.3 to help me setup Qwen 3.8 in OpenCode locally, which it did immediately, with llama-cuda.

now I'm trying out local code generation, since I have a use-case where I cannot use remote models. it does kinda work, but the context window is limited and it is quite slow.
---Reply---
underproofed ganbold@ganbold (2026-10-08 21:24:26)
woah running 27b locally, nice! what is your graphic card? for a reference.

idk with the quality, maybe try the 35b-a3b as well. I assume it should be giving more breathing room to the card rather than using dense 27b, for a slight faster speed. 

but yeah context size is my issue as well with bigger model. I am surprised that 64k-128k is barely enough for myself