Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

Reply to @lain@fediffusion.art
𝒲𝒜𝐿𝒫𝒰𝑅𝒢𝐼𝒮 𝒥𝒰𝐸𝒱𝐸𝒮@s8n@posting.lolicon.rocks (2026-09-28 01:55:16)
@lain qwen-3.8-flash in 4bit quant is down to 138GB of ram if I read this correctly. We are well on the way for this being entirely doable locally before 2030
---Reply---
lain@lain@fediffusion.art (2026-09-28 13:06:06)
@s8n this was done locally, half of that data is the engram table which does not need to live in RAM. It's running on my px13, strix halo laptop with 128gb unified ram.

---Replies---

𝒲𝒜𝐿𝒫𝒰𝑅𝒢𝐼𝒮 𝒥𝒰𝐸𝒱𝐸𝒮@s8n@posting.lolicon.rocks (2026-09-28 13:10:48)
@lain damn that's pretty good, thanks for the info. I doubt I can get up to 128gb but maybe I should consider it. Getting to 64 will be 400 bux anyway because my RAM has very low CAS latency and I'd need a matched set. Maybe I should just spring for a cheaper 128 set and ditch the 32gb I have now entirely