Thought Bubble 3: The Tension Between Safe and Useful

The Value of Healthy Skepticism

Just as most of you, I’ve been thinking a lot about generative AI lately. I’ve been trying to take a healthy, skeptical look at it as I try to figure out where best to integrate it into my daily life — writing code, writing blog posts, summarizing piles of documents I don’t have the time (or desire) to read, planning my wedding, and more. I’m looking at my tolerance for error with a given task (am I saving lives or saving data that’s non-critical to a database?).

I’m also trying to understand how an LLM understands the problems I give it and why it might decide to solve it that way. Why does this one generate an image of a man when I asked for one of a woman? And why do they seem so confident in their approach to a problem (like the time I tried to use ChatGPT to solve the problem of hanging three equally spaced pieces of art on a wall) even when I tell them they are wrong. Earlier this week, I stumbled upon this fun piece of writing from AWS Community Hero, Brian Tarbox, comparing LLMs to teenagers and discussing this same healthy skepticism I refer to. Go take a read of his post and give it a thumbs up!

It struck me that Brian was onto something. We have more patience for kids — who have equal measures incredible promise and staggering hubris. We allow ourselves to be surprised by their capabilities, but we also don’t expect them to shoulder the full responsibilities of adults and professionals.

-Jenna

The Tension Between Safe and Useful

I love these thoughts, Jenna, and I agree that Brian’s piece was on the money. Reading outputs from LLMs, I’ve had flashbacks to my teaching days, when I was grading the essays of college freshmen. Those kids, just like LLMs, could be surprisingly insightful in one sentence, and confidently wrong in the next.

Speaking of being confidently wrong, Google’s Gemini model made waves in the last couple of weeks by answering questions and requests in absurd ways, often with the apparent goal of NOT offending people. The idea was, images of people shouldn’t simply default to White folks, so Gemini, when asked to produce images of German soldiers in WWII, spat out pictures of Black and Asian men and women in German uniforms.

Not exactly historically accurate.

The blunder was bad enough that Google disabled Gemini’s image-generating capabilities, and CEO Sundar Pichai called the fiasco “unacceptable.” But countless people have continued posting on social media about the absurd answers they get even from the text-only version of the model — particularly when it comes to more politicized topics.

I don’t want to wade into the politics here, but what’s fascinating to me is the extent to which our desire for safety — that is, for tools that don’t demonstrate significant bias, that don’t enable users to generate hateful content, and so on — is often at odds with usefulness.

Think about it like this: if you want to design a shovel, but you’re worried about shovel-users bludgeoning each other over the heads with your shovels, you may consider making the blade of the shovel blunter and lighter. But at some point, changing the shape and materials also makes it a less effective shovel.

Google’s not new to this challenge. As the New York Times reported, Google Photos raised ire in 2015 when it labeled a photo of two Black people as displaying gorillas. The answer: Google Photos stopped labeling anything at all as gorillas, including gorillas.

I want to be clear here: I don’t think there’s a simple solution to these problems. All true innovation gets used in scary ways. Just look at the combustion engine or nuclear fusion or the internet.

But while we’re thinking about the engineers behind Gemini and other AI models, it’s worth remembering how difficult it actually is to create a tool that’s both useful and completely safe.

-Dave