Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

Reply to @mrsaturday@shitposter.world
:lazer_D::lazer_I::lazer_E::lazer_L::lazer_A::lazer_N:@dielan@shitposter.world (2026-10-10 08:40:55)
@mrsaturday @sun I heard a fun theory the other day on a podcast that suggested the reason all the frontier model companies are calling for a slowdown - aside from obvious IPO collusion shenanigans - is the fact that the latest models aren't substantially smarter than the previous. Which is why Anthropic specifically lobotomizes it's old versions to set high expectations of the newest models when they release. 

Meaning - local models will eventually catch up, and all the development most recently has been in harnesses rather than the model. Doing more with what they have and putting it behind a subscription. The frontier model companies are trying to get their shit together so harnesses like Hermes, Pi, and Opencode for example don't overtake them. I would expect some time in the future they'll try to lock you into their own harness and stop letting you use an API key in a different harness soon.
---Reply---
Blurry Moon@sun@shitposter.world (2026-10-10 08:43:16)
@dielan @mrsaturday I'm gonna try to get hermes working with a local model again today, perhaps qwen3.8 flash

---Replies---

:lazer_D::lazer_I::lazer_E::lazer_L::lazer_A::lazer_N:@dielan@shitposter.world (2026-10-10 08:44:48)
@sun @mrsaturday if I knew this AI stuff was coming like this in 2024 i would have gotten a slightly better card. I'm stuck at 20 GB VRAM with my 7900XT. So i can run 27B models with like no context window, or i can run smaller models one might recommend for a 16GB card with a big dick context window. It's not a bad life but damn i wish i had 24GB VRAM