Integrity is not a binary and environmental incentives matter

I’ve been thinking a lot about integrity recently and finding that it’s a more difficult subject than I thought at first.

Be honest, don’t try to deceive or trick anyone, easy right? Welll I don’t think it is. The problem is that our incentives can distort our thinking & actions.

The conclusion I’ve come to is that acting with integrity takes a lot of work. It takes a lot of questioning your own thinking & looking for motivated reasoning, and especially it takes good friends to hold you accountable / help you see your blind spots.

An example from the security profession. We’re incentivized to overemphasize the risks presented by attackers. It’s easy for me to tell a story about a scary exploit to convince someone to implement a particular security control, even if that exploit is not commonly exploited.

There are real attacks out there worth protecting against, but I sometimes catch myself forgetting to convey my real beliefs about the likelihood of a particular attack – or worse yet, leaning into a belief that an attack might occur because it’s convenient to believe that!

Thinking through some of this has really driven home the connection between integrity and epistemics. To really seek the truth, I think it’s important to notice where you’re rewarded for twisting your beliefs in subtle ways.

It feels especially hard to track truth when in social domains with personal stakes. It’s hard to see through annoyance to really understand why a friend may have done the thing I’m annoyed with. Or why they might be annoyed with me.

The low integrity route is to blame other people when it’s convenient and only listen to their story when you have to. To take the path of higher integrity is to seek out others stories when you notice your perspective has some gaps. Works best when your friends are trustworthy!

I don’t think acting with integrity means abandoning your position if people disagree. If you have justified confidence, it means holding your position even when it’s socially inconvenient. Judgement of friends can be hard but sometimes that judgment contains something important.

Coming into a conflict, it may be tempting to try to make your perspective the dominant one or trying to front the perspective of your friend. But I think it’s better to try to find the truth in both and weight it according to the evidence there.

It can be hard to know what’s true in a social conflict. I think it’s the way of high integrity to notice where you’re really confused and acknowledge you’re not confident. And when you’re actually sure of something, don’t adopt false humility!

Lots of caveats here, I think there are times to project confidence and times to be gentle bringing forth your beliefs. But in general, being able to speak your beliefs with your real level of confidence is a route to higher integrity.

Which brings up a really important thing that this year drove home for me. Your environment, especially the people around you and the norms they hold, have a big impact on your ability to maintain integrity. Environment can make it much easier or difficult!

I left the biosecurity field this year because I couldn’t find a way to work in it and maintain my commitment to integrity. 😢 Hard to tell the whole story but I found myself facing pressures that prevented me from speaking on and acting on things I cared about.

In the past, I’ve stayed in environments that made integrity difficult to maintain for too long. Sometimes it’s better to leave. Sometimes it’s better to stay and try to change the environment. Not always obvious what the right call is.

But I think the more common failure is to stay in an environment which incentivizes dishonesty and bending to the pressure. I think it’s hard to avoid sometimes!

And that’s a lot of what I’ve learned. Choices about integrity show up in big and small ways. The big ones can be hard if big things are at stake. But the small ones are hard in a whole different way. They’re hard because small compromises can seem innocuous, but add up.

I’m not an integrity absolutist. There are other values and other virtues worth embodying. Sometimes you have to trade a little integrity to do some other good, because there are real tradeoffs. But I do think it is pretty damn important.

A key reason integrity is important to me is that I think it’s required for people to coordinate on projects with unusually hard epistemic difficulty. Like the kind with long feedback loops or lack of precedent, e.g. AI alignment.

So I think it’s worth accepting some of the costs that go along with striving for high integrity. I think it takes a lot of attention & discussion to improve at the skill. And I think it makes you worse at advertising, worse at propaganda, and other things. Acceptable losses.

But like I said before, I don’t think it’s binary. That you have integrity or you don’t. I think we can all strive to improve, to keep ourselves and our friends a little more honest. And I think that could help a great deal in aggregate.