Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

Reply to @dielan@shitposter.world
Blurry Moon@sun@shitposter.world (2026-10-10 08:43:16)
@dielan @mrsaturday I'm gonna try to get hermes working with a local model again today, perhaps qwen3.8 flash
---Reply---
:lazer_D::lazer_I::lazer_E::lazer_L::lazer_A::lazer_N:@dielan@shitposter.world (2026-10-10 08:44:48)
@sun @mrsaturday if I knew this AI stuff was coming like this in 2024 i would have gotten a slightly better card. I'm stuck at 20 GB VRAM with my 7900XT. So i can run 27B models with like no context window, or i can run smaller models one might recommend for a 16GB card with a big dick context window. It's not a bad life but damn i wish i had 24GB VRAM

---Replies---

Blurry Moon@sun@shitposter.world (2026-10-10 08:45:21)
@dielan @mrsaturday I have a 24GiB 4090 and it's not enough for most cool models now :sadge: