What's wrong with Max Tegmark's river metaphor

It is a metaphysical disposition that puts the wrong framing around the debate

In the recent Munk debate, Max Tegmark proposed a powerful metaphor within which he explains the various positions on the runaway AGI scenario. We are on this big ship sailing down the Niagara river. Max and Yoshua are warning everyone that there is a waterfall down there, which maybe isn’t safe. The waterfall is representing the existential danger of catastrophic runaway AGI. Melanie is not convinced that there is a waterfall, and even if there is one, we need to first deal with the dangerous rocks between here and then. Yann says that yes, there is a waterfall, but we will be dealing with the waterfall when we get there.

Yoshua repeated the metaphor in an interview later, showing that he found it a good way of representing catastrophic runaway AGI.

The tree and the river. DALL-E

What I find the most interesting is that nobody in the debate is questioning the framing, everybody seems to accept that we are indeed sitting in a boat on a river. This is a powerful picture. It is not an argument, but a framing of an argument, focusing the debate on something that everybody accepts as granted. If we accept it, we must answer: what do we think about the looming waterfall?

The point I raise in this podcast with Bogdan is that it is not obvious to me that we are in a river because the river represents a deterministic, highly teleological metaphysics. The river is like a tree but upside down, so there are a lot of ways to get in the river, but then we all go in the same direction, and the only choice we have is to stop the boat or let it drift towards the unavoidable waterfall.

But I think the world doesn’t work like this. We are not in a river. We have free will, both individual and collective, and our decisions shape how the world unfolds. If anything, it is a tree that grows indefinitely. At every branching point, we have choices to make, more precisely, our decisions are creating the branches. There are some imaginable branches where there is a catastrophic existential threat, but a lot of the other branches are safe. I think that this framing would generate a more meaningful debate. Within this framing, Yoshua and Max are saying that there are no more branching points between now and runaway AGI, whereas Yann and Melanie are saying that there are, moreover, there are also important decisions to make about other dangers that we will miss if looking too far ahead.

This is further related to what is your general disposition in life, how optimistic or pessimistic you are. Whether you believe that the world is designed in a way where the most important thing is to align yourself, to live a life of meaning at every level, personal, family, smaller and larger community, and trusting that if everyone does so, we avoid existential catastrophes.

Trust is important here because there are no guarantees. This is where the debate gets hard, because the argument for usually goes: here is my plausible scenario of catastrophic AGI, you cannot prove that it will not happen, therefore we should focus all our energy to avoid it. What I think is that the scenario is not a forecast, it’s a prophecy. And prophecies are dangerous, because they tend to realize themselves even though we are working to avoid them, or precisely because we are working to avoid them. This is, of course, non-scientific truth, but we are all in the theological realm once we have started prophesizing.

And this is where John Vervaeke’s positive program becomes very important, because it shows an objective to work towards, not to work against.