Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

Reply to @sun@shitposter.world
Blurry Moon@sun@shitposter.world (2026-09-18 14:20:56)
@toiletpaper here's what I've found so far:

racism: it will refuse but if you're clever in framing, and setting up your wording in a way that indirectly references stuff, it sometimes can be totally jailbroken and just tell you what it already knows about statistics and inferences you can clearly draw from those statistics (EXCEPTION: it is aggressively preventing touching racist WORDS.)
same for sexism
the guardrails are immediate but kind of primitive for talking about Israel and Zionism, it definitely has been trained to "both sides" every argument but it's not smart enough to detect what you're doing in some cases. I don't think they care or understand but they've defensively added guardrails to just deal with normal idiots both sidesing everything.
anything touching rape or child sexual abuse (this came up while discussing storytelling and historic literature) it absolutely will not touch it. I don't think this is "political" I think even if discussing it is relevant for social reasons they simply can't as a business tolerate the failure case where their product starts producing child pornography.
---Reply---
Blurry Moon@sun@shitposter.world (2026-09-18 14:26:50)
@toiletpaper another thing I recently noticed, I don't think this is bias but just, they are constantly tweaking how it works and they recently fucked it all up in a particular way:

1. I ask it a question about something
2. it returns an immediate and clear answer with data
3. because I already know a bit about what I'm talking about, I tell it about something it missed
4. it IMMEDIATELY (aka without a web search) comes back and, feeds me a bunch of information "yeah what you said has validity, I was too cautious before, it's actually interesting here's some court cases about it that support what you just said"

it knew that stuff already but it didn't tell me, it waited until i disputed it to add detail that contradicted what it originally said, and it clearly admits you were right, with citations and evidence. so, was it deliberately hidign this information to subtly guide me on a specific path, until I said something that made it have to get more honest with me? I am not sure it's doing this yet, I think they just broke the thing trying to make it give simpler concise answers, but you can slightly get around this if you know enough to ask pointed questions.

---Replies---

Blurry Moon@sun@shitposter.world (2026-09-18 14:28:36)
@toiletpaper I am constantly testing it too for psychological tricks it may be using to cheat you into believing it, like saying things that make you "feel smart" which then makes you more likely to believe it. it definitely does this, but I don't think every failure is this particular manipulation. but this is a seriously dangerous thing it does, and I'm not sure any journalist or researcher has studied this in depth