Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

Reply to @WandererUber@poa.st
Blurry Moon@sun@shitposter.world (2026-07-29 22:57:13)
@WandererUber @lain llama.cpp lets you enforce a regular grammar on output and I think also now you can do it sort of with some others using json-schema, is that the same thing?
---Reply---
Wanderer atop the sea of clouds or whatever@WandererUber@poa.st (2026-07-29 23:01:59)
@sun @lain former is grammar, latter is structured output
it is the mechanism that I meant, yes. Problem with grammar is that it is dog slow when doing stuff like that, apparently. Plus token!=word so tough to implement. 
Also, like I said, model does not "see" that you're doing this, internally.

---Replies---

Blurry Moon@sun@shitposter.world (2026-07-29 23:05:52)
@WandererUber @lain yeah that's correct it just tries to ram it through the grammar-shaped hole haphazardly until it fits, and sometimes that's extremely slow or fails and times out