Home | Notifications | New Note | Local | Federated | Search | Logout

Note Detail

Rob Ricci@ricci@discuss.systems (2026-09-10 14:02:52)
I kind of hate that *gestures at everything* is providing such fertile ground for thinking about @viveksrikumar and C Thi Nguyen's Ethics in AI class (which I'm sitting in on), but it is, so, here I go again.

One of the ideas they've been talking about in class is risk tolerance. Pretty basic in concept, people make decisions about how much risk they are willing to take on. A lot of this is very mundane and implicit (is the second patty worth the cholesterol risk? worth it to dash across the street here instead of going to the corner?) Some of it is explicit in things like investment decisions. The risk tolerance that you take on is a reflection of your values - play it safe? shoot for the moon? Pretty standard stuff.

One way, however, that this shows up in unexpected (to many of us) ways is in things like like science and medicine: it's a value judgement that we don't often acknowledge. A threshold for p-values of 0.05, which most folks use unquestioningly, means (approximately, it's complicated) a tolerance of a 5% chance of being wrong. Is this ... a good amount of risk to tolerate? It's a nice round amount, that's cool. Is it too high? Is it too low? Depends, probably?

Here's a pretty standard illustration of "it depends" that was used in class: risk tolerance in policing. Let's say you've got some technology, totally hypothetical here, that reads license plates with cameras, let's call it Flook. Somebody, somewhere, gets to set thresholds for what's a certain enough match that the cops should roll out and pull over the driver. This might seem like it's just some number, but it's a value judgement about the risk of being wrong.

The cops? They have a pretty high risk tolerance in this situation. Sure, they don't (let's assume, to be generous) want to spend all day pulling over the "wrong" people. But the cost to pulling over the wrong person is just not that high for them. A bit of wasted time, maybe a confrontation with driver that's mad at you, possibly a b