Aumann’s agreement theorem

Aumann’s agreement theorem, roughly speaking, says that two agents acting rationally (in a certain precise sense) and with common knowledge of each other’s beliefs cannot agree to disagree.

LessWrong Wiki

In other words, when two truth-seeking individuals share information, they approach the same beliefs.

Sometimes, when I’m in a discussion with others, I remember Aumann's agreement theorem. To me, it’s an ideal: Whenever we discuss facts about the world, we should approach agreement. Otherwise, we're failing. This ideal assumes that we are:

  1. trying to approach truth, not to e.g. signal our virtues; and
  2. not discussing mere preferences, such as whether chocolate or vanilla ice cream tastes better.

An example

Imagine that we’re going for a drive, and we’re considering two different routes to our destination. You’re 65 percent certain that route A will be faster, and I’m 55 percent certain that route B will be faster. This is our starting point—our priors.

Then, you tell me that you’ve driven these routes many times before—in your experience, route A usually takes about 25 minutes, and route B 40 minutes. I simply had a weak hunch that route B will be faster, so I update to agree with you that the probability that route A will be faster is 65 percent. Now, we’re in agreement.

An ideal

As I mentioned above, I hold Aumann’s agreement theorem as an ideal. Specifically, I hold it as an ideal for epistemic rationality, defined by Eliezer Yudkowski as “systematically improving the accuracy of your beliefs.”

In practice

In practice, it is a difficult ideal to achieve, even when the two assumptions above hold. For example, our beliefs are often informed by intution, in the sense of internalized experience. In our route choice example, I could have an internalized experience that less popular routes tend to be slower. If route A was less popular, I might not agree with your assessment. But I would have no conscious awareness of why I deemed route A to be slower than route B. Consequently, there would be no way for me to share this information with you, and we would—sadly—have to agree to disagree.

Regardless of the difficulties, I still find it useful to keep the theorem as a lofty goal.