Naturalizing relevance realization

Notes on the Jaeger-Riedl-Djedovic-Vervaeke-Walsh paper

The paper is deep and beautifully written. Their claim, that natural agency, cognition, and consciousness, at their very core, are not computational (but physical!), is thought-provoking and controversial, but, quite paradoxically, a lot of what they propose is highly useful even within the purely computational (extended) reinforcement learning paradigm. So if you feel a metaphysical repulsion reading the claim, I urge you to put it aside for now, and delve into the paper anyway.

My notes are not meant to replace or even abridge the paper. They serve me as cognitive stepping stones, which I thought could be useful also for others hiking through this complex landscape. I use italic to mark my own thoughts, the rest is my understanding of the paper.

Naturalizing relevance realization by DALL-E.

Abstract

• organismic agents (vs algorithms): autopoietic, anticipatory, and adaptive organization of living beings

• relevance realization is pre-inferential, beyond formalization

• adaptive, emergent, metabolic, co-constructive dialectic

• grip on the arena

• making sense of one's world: embodied ecological rationality

Introduction

• Algorithmic agents: automated computational procedures in small (I would call closed) world. Predefined formalized ontology. Well-defined problems. Everything and nothing is relevant at the same time (flat saliency landscape). Cannot deal with unexpected situations, only mimic improvisational behavior.

• Organismic agents: limited beings coupled to an open-ended world overflowing with potential meaning, profoundly exceeding our grasp; important cues are scarce, ambiguous, fragmentary. Problems are ill-defined. To survive, we need to solve the problem of relevance. 

• Before inference, organisms need to turn ill-defined problems into well-defined problems, open worlds to closed worlds, semantics to syntax (right to left hemisphere): relevance realization. Brings forth a world of meaning.

• Self-referential, self-creating (autopoietic) systems. Degree of self-determination, limited but autonomous actions, anticipation of the consequences of their actions.

• Opponent processing is a dialectic: two reciprocally constituting dynamic processes (not simply feedback). Support each other's existence by mutual co-construction. Competing and complementary behaviors that mediate the organism's interactions with the environment. Mutually co-creating and thus collectively co-emergent interrelation between the agent's goals, actions, and the affordances that arise from its interactions with its experienced environment. An internal and external dance. Also works between individual, for example, ina couple.

• Main claim: the dialectic dynamic of relevance realization is not an algorithmic process. Therefore, natural agency, cognition, and consciousness, at their very core, not computational.

• Relevance and evolutionary fitness are similar. So relevance should be understood to be evolutionary, ecological, economic. 

• Relevance realization is a prerequisite of cognition at all levels. Including populations?

Agential emergentism

• Computationalism: physical reality is explained by algorithmic computation based on lower-level mechanisms.

• Algorithmically simulate != processes are intrinsically a form of computation.

• Agential emergentism postulates that all organisms are intrinsically agential, and agency is natural and fundamental. Algorithms cannot act on their own behalves, cannot pursue their own goals.

• Natural agency: living system acts according to its own internal norms. Self-manufacturing (autopoietic) system rule 1: keep myself alive. Algorithms can only possess extrinsic purpose. (One form of alienation, symptom of the meaning crisis, when we start seeing ourselves as such algorithms.)

• The ability to solve the problem of relevance is connected to the possession of intrinsic goals. P9 first paragraph. Hm. I'm somewhat lost here. A WiFi antenna whose goal is to serve its users with bandwidth has an external goal, yet relevance realization can be important for it. The paperclip maximizer derives the goal of self-preservation from its external goal: if it dies, it cannot make paperclip. So I don't see this, and I'm not sure we need this argument at all.

Relevance realization

• RR is transjective. Organism enacts its meaning/value. Grounded in agent/arena relationship.

• Computationalism: RL framework, general problem-solving, initial and goal states, sequence of actions.

• Large search spaces, combinatorial explosion -> heuristics. Satisficing instead of optimizing. Embodied bounded rationality: link heuristics to problem. Identifying relevant cues still a problem.

  • Collection of possibly relevant features is indefinite.

  • Relevance is situation-dependent.

  • Infinite regress: features have to be identified before solving the problem, if RR is cast into a problem-solving optimization, we have circularity.

• RR is not completely formalizable. RR is the act of formalization. OK but isn't it a jump from there that it cannot be algorithmic?

• Consequence: behavior and evolution of organisms also escape algorithmic frameworks. Simulation is possible but it will never be complete. Semantic residue: unknown unknowns.

• Opponent processing is a set of meta-heuristics, often in pairs of antagonistic objectives, played against each other and adjusted constantly. Efficiency vs resilience, generality vs specialization, exploration vs exploitation, focusing versus diversifying. Listening to inside vs outside. Problem solving vs reframing. Constant deialectic re-assessment leads to a situated and temporary adaptive fit between agent and arena. Predictive processing (paying attention to and trying to eliminate prediction errors) is a compatible meta framework, but PP is also opponent processing: projecting vs adjusting. (In RL: planning using the predictive model vs adjusting after executing the selected action and observing the real consequences.)

• But isn't this algorithmic? In my view semantic residue requires/is solved by self-transcendence, when meta-heuristics are extended. A finite set of opponent processes is always algorithmic, though, and this is exciting, very different from what we do in RL today (trying to summarize everything in a single reward).

• Presupposes intrinsic goals. We will ground RR in the basic organization of living beings and the kind of agent-arena relationship. It is an evolutionary view of RR.

Biological organization and natural agency

• We need to show that organisms can define and pursue intrinsic goals without requiring intentionality, cognitive capacities, or consciousness (why? because these are derived?)

• Aristotle's four causes:

  • material: the marble

  • formal: what makes it the sculpture of the sculptor’s (and not anyone else's)

  • efficient: sculptor wielding their tools

  • final: sculptor's intention to make their statue

• Autopoiesis (self-manufacturing). Life is what life does. Local entropy decrease can exist in non-living (e.g. tornado) but it can't maintain and create itself. Pragmatically I understand, but metaphysically it is very hard to draw lines, clearly delineating these categories, for example, looking at the beginning of life on Earth (Nick Lane’s work) or the beginning of a given life, see e.g. Ciaunica et al. I understand that one can draw a line between a tornado and a cell, but on the way the cell becoming a cell from a tornado-like self-organizing phenomenon, probably around hot wells deep in the ocean, it would be hard to draw a line between non-living "thing" and organism.

  • Not attached to physical structure, on the contrary, it's dynamically changing its components.

  • Organizational closure of constraints: collective dependence between the functional components. Organism-level pattern of constraints restricts and channels the constitutive dynamics in a way that preserves the constraints. Causally circular, it's a form of self-constraint. Organisation becomes the cause of its own stability. -> identity and individuality. Requires thermodynamic openness to harvest entropy from the environment. Maintenance requires work. Closed under efficient causation, but not material causation. More than mere cybernetic feedback (=material causation, no hierarchy, flat). Immanent causation is full of cycles that preserve themselves over space and time. (the timescale issue, Levin’s cognitive cones). Coincides with final causation: autopoiesis, keep producing itself. (death and self-sacrifice?)

• The starting problem of biological systems. All components must exist and be in place at the same time.

• Rosen's definition of complex system: at least one hierarchical cycle. Incompleteness argument, similar to Godel. vs Church-Turing. But where does it start and end at the bottom of the hierarchy?, from Michael Levin. Claim: it doesn't even make sense to ask if living systems are computable if they are not formalizable. Computability focuses on predictability, this limitation is about explainability.

• Intrinsic goals make it autonomous. It relies on the environment, but it has its own existence as intrinsic goal. Future states of the system are dynamically presupposed by its own inherent organization. When it dies, it becomes simple (hm, again, what about its components? BTW, death is the proof of the impossibility of perfect RR) Complexity originates from within, opaque to the observer.

• Next: interactive autonomy: organizational and ecological dimensions.

Basic biological anticipation

• Agency has effect beyond the organism -> ecology

• If it can't die, it has no reason to act. That's why an algorithm needs an outside executor.

• Anticipation needed. This is not controversial: computational systems can also be anticipatory, within their small world.

• Projective perception in a dialectic adaptive dynamic.

• Internal models, including self.

Affordance, goal, action

• Situated Darwinism, an evolutionary account of RR

  • intrinsic goals

  • available actions

  • affordances

• Affordances are relational, transjective (like relevance). They constitute an (even the!) arena.

• Organisms experience their environment as an affordance landscape. Arena depends on the environment and intrinsic goals. Organism senses. Landscape is laden with value, meaning wrt the goal.

• Goals can be contradictory.

• Dynamic process: constant coordinated codependent collapse and reconstruction of the three sets (goals, action set, affordances). Leads to a firmer or broader grip on the arena, if works well -> evolution. If not: reciprocal collapse, agent disconnects or alienates from the arena, narrowing and death. Constant co-evolution and co-construction.

• Works both at the individual and population level.

To live is to know

• Three dialectic processes:

  • autopoiesis: collective co-constitution of internal milieu and regulated selective cross-boundary transport. Enables the agent to autonomously set its own intrinsic goals.

  • anticipation: internal predictive model of both internal and external states, projective about the environment (constructive-generative?), available actions, planning. Enables the agent to pursue intrinsic goal.

  • adaptation: transjective between the agent and the arena, established through co-constitution of intrinsic goals, set of actions, and relevance realization resulting in affordance/salience landscape, tightening the agent-arena relationship and the organism's grip on reality. TAME point of view: the arena itself is the ecosystem from the agent’s point of view, but in fact it is also a living organism. The agent is part of a bigger living being.

• Let's join them: a naturalistic account. Requires no mysterious new forces or laws of physics, just a change of focus from small-worldish predictive models that are good approximative models for local (in time and space) processes, to the open world with its unpredictable historical succession of dynamic constraints.

• They share the same kind of dialectic dynamics.

• They can only be simulated (see starting and halting problems).

• Anticipatory rather than reactive (also creative/generative/hierarchical-predictive vs surface predictive?, forecast vs prophecy, generative?)

• Open-ended: the adjacent possible. Open affordance landscape, the transjective dance that make affordances and the agent/arena evolve. That also requires a creative/generative model. Deep understanding means I can mentally build the scene. The adjacent possible cannot be pre-stated, generated dynamically as evolution/RR advances. Possible future affordances, goals, and actions cannot be pre-stated as well-defined sets before they are actually actualized. Kauffman: radical emergence. That also supposes a special kind of model.

• What unites the three: the continuous collective co-constitution of constraints.

• Due to their openness of formal cause, they constantly explore new structural variants of within their closure of efficient causation -> unprecedented behavior.

• Constitutive frame: the three processes are interdependent, they emerge from each other and they actualize together.

• Constraint frame (Platonistic, what is not there): the emergence of the coherent overall dynamics reduces the degrees of freedom of the subprocesses. It’s a general law: processes part of a larger organism are less free than when they “check out” - e.g., see cancer.

• The constitutive frame shapes the likelihood landscape of potential futures, whereas the constraint frame determins what is accessible/permissive. Hm, in principle, likelihood may include accessibility (non-accesible = zero likelihood) - I’m missing an argument here.

• Work (using up free energy) is needed to maintain the constraints that keep the organism alive (constraints that, reciprocally, afford the channeling of the free energy into work). Orderliness = low entropy restricts the space of the dynamic system so it can maintain itself. The organism needs to tap into the entropy gradient in the environment, so it is thermodynamically open.

• Non-living self-organizing systems (like a tornado) '“eats through” its source of free energy as fast as possible while destroying it at the same time. Living organisms maximize both dissipation and the path length to the (inevitable) depletion of the source. Here is where I don’t see the quantitative difference. See cancer, which is between the two. I prefer Levin’s cognitive cones that explcitely model the time-scale. Here is where death is important. Death is always recycling part of the whole system. There is definitely more here to think about.

• All this leads to multiple, potentially new, levels of organization, as long as the new level is a self-sustaining dialectic dynamic. It is driven by a metabolism (harvesting entropy gradient) of the constituting subsystems.

• A higher-level organization is for example an agent with a cognitive system. Bacteria move towards nutrition gradient and can die if there is a discontinuity or danger towards the maximum. But they don’t model the environment. An agent with a cognitive system actively models its environment, therefore can be deluded. It can self-transcend: revise its world model based on experience (if it doesn’t die). Again, I see the difference between these two kinds of systems, but I don’t see the sudden qualitative jump. It’s gray scale. In addition, as the authors stated before, relevance realization (and its self-transcending nature) and evolution are similar dynamics. Something always dies when models diverge from reality, either the individual, or the lineage.

• Speculative: intentionality, awareness, and consciousness may arise as higher levels of dialectic dynamics, when the agent starts modelling other living organisms and itself.

• Consciousness and cognition are not comutations. They are elaborations of natural agency based on relevance realization. Agency, consciousness, and cognition are ways by which organisms come to know the world. “To live is to know.” (Maturana). At the heart of this is the ability to pick what is relevant, to delimit the arena in a large (open) world. It is not formalizable as an algorithmic process, it is the process of formalization itself. (Relevance realization is the map explaining the mapmaking process itself). We do not create meaning through computation. We generate meaning through living and acting, which is how we get a grip on our reality.

Conclusion

• Aristotle’s three levels of soul: nutritive, sensitive, rational.

• Fully naturalized animating principle of living systems: autopoiesis, anticipation, adaptation. Constraints are dialectically created and maintained through physical work consuming and metabolizing free energy in far-from-equilibrium thermodynamical systems. This is compatible but not reducible to known laws of physics.

• Agential emergentism. Dynamics is contingent and partly generated from the organization itself. Autonomy is neither due to lack of causal determination, nor driven by randomness, but it is also not determined by automated reactions to environment. Autonomy comes from the dialectic interplay of impredicative, continuous, and concurrent subprocesses, constantly requiring the other subprocesses to be present and work properly. This dynamic and emergent self-constraint leads to agency: the ability of self-manufacture, set and pursue intrinsic goals, interact with the arena emanated from inside.

• Living organisms are teleological: they want to create and maintain themselves. They do not just behave as if they had agency (Dennett), but they actually do have it.

• Scientific causal explanation answer to the how, the (equally scientific, naturalistic) teleological explanation answer to the why.

• Do not suffer from the usual problems with teleological explanations:

  • Future does not causally generate present. The agent pulls the future in the present using anticipative predictive models. These models and their expectations are fully actualized in the present. Requires the learnability/intelligibility of the arena.

  • No circularity: intentionality, cognition, predictive models can evolve from simple habituation.

  • No predefined normativity: other than staying alive, all goals emerge intrinsically.

• No need for global teleology, on the contrary, life is radically open-ended. Unpredictable, not prestatable, not formalizable.

• Living is infinite play. The space of possibilities is evolving, and we the agents have a say as we co-construct the arena.

• Levels are open to further evolution.

• Relevance realization is the process of formalization, cannot be formalized. All living systems realize RR. Algorithms don’t encounter it since they live in small worlds (already relevance realized).

• Physiological, behavioral, and evolutionary adaptation implement RR in a non-algorithmic fashion, through opponent processing: various dynamic processes, often contradictory, “fight it out”.

• Predictive processing is limited:

  • Variables, framing, small world has been already selected (realized).

  • Priors needed.

• Computationalist approaches (eg Levin) can lead to empirical success but fail to address RR. Here I am torn, in any case, RR and opponent processing seems useful even within the computationalist frame. Actually: “allowing for computationalist approaches as a part of its wider outlook” - seems to mean that we can have good mechanistic models for certain subsystems.

• RR exists in low level organisms, like bacteria.

• RR and embodied heuristics (Gigerenzer, 2021) and embodied bounded rationality (Mastrogiorgio & Petracca, 2016; Gallese et al., 2020; Petracca,

  2021; Petracca & Grayot, 2023). Rationality is doing the appropriate thing to attain one’s goal. In a small world it can be formalized, but in a large world it needs to include RR through opponent processing, so != logic and != intelligence.