I saw him give a talk on this a couple months ago and I thought it was a really interesting talk!
The way I interpreted it was this:
Here's an argument that artificial intelligence could be conscious:
Consciousness is a property of human minds. Mental properties are functional properties. Functional properties are multiply realizable. Therefore consciousness is multiply realizable. Therefore if we artificially created a different realizer of consciousness, it would be an artificial intelligence that is conscious.
I interpreted him as challenging this argument by focusing on the premise that functional properties are multiply realizable. The idea is that any particular autopoietic system has a function that has two levels of description (or maybe has two different functions?). System A has as one of its functions maintaining and building System A-type things. But System A also has as one of its functions maintaining and building itself. Let's hypothetically suppose that some different realizer, System B were possible. To be a realizer of the first function, it has to have as its function maintaining and building System A-type things. But to be a realizer of the second function it would have to have as its function maintaining and building System B-type things.
Thus, in general, autopoietic functions are not going to be multiply realizable, because a different realizer would maintain and build the original type system, not the new type system. To the extent that important mental properties connected to consciousness are autopoietic, that would be a way that this kind of consciousness is not multiply realizable.
But it just seemed natural to me to respond the way you do. Obviously there could well be other autopoietic systems that sustain themselves in different ways. There's nothing in this argument that says that those systems couldn't be conscious. There is something here that is strongly suggestive that their consciousness would be radically different from our consciousness, because their autopoiesis would be very different from our autopoiesis.
But I think that most functionalists are totally happy with the idea that there are radically different types of consciousness - at least, bat-type and human-type are radically different. (I'm quite open to the idea that there's also plant-type and bacteria-type that are even more radically different from human consciousness than bat-type, but you don't need to think that to think that there could be a system with the kind of complex multi-level feedback-control interaction with its environment that gives rise to consciousness, but in a different form than that of any biological creature. Unless you think that such a system would ipso facto be alive, and would thus count as biological. But I'm happy to embrace that conclusion too, as long as "biological" is defined in this functional way, rather than being tied to the kind of substrate that actual plants and animals and bacteria and so on have.)
Yes, exactly so. I'm contemplating a post on the type of argument you diagnose, which I also see variants of in Godfrey-Smith, Cao, and Block. I worry that the dialectic is this. The multiple realizability theorist says "multiple realizability of consciousness is correct because you could get the same functions in a different substrate". The objector says, "no, not exactly the same functions, so your argument for multiple realizability is unsound even if we grant functionalism". The multiple-realizability theorist needs either to go back and define the functions coarsely (which risks undercutting their claim that those functions are sufficient for consciousness) or to jettison arguments in terms of "same function" in favor of some other type of argument (Pober's and my "Copernican Argument" for alien consciousness is an attempt in that direction).
This is why Moreno and Mossio argue that closure in the sense of Varela and Maturana should be located at the level of constraints that define the network of relations realizing a given function. The same was pointed out by James Beniger in The Control Revolution as the three dimensions of control:
You mentioned that functionalists may be totally happy with the idea that there are radically different types of consciousness, but at which point are they so different that we ought not even call it “consciousness”? It seems a messy endeavour.
Three of these threads are ones I've had to build against directly.
The dupe-and-mimic arms race. The essay flags this as an open problem — a system needs to tell a genuinely compatible replacement from something mislabeled or lookalike-but-ruinous. The discipline that's held up here: a check has to select the object itself, never a neighboring label — an instrument that reads a proxy grades the proxy, not the thing. And more specifically: ask the object its state, never infer it from a symptom. A part that looks compatible from the outside is exactly a symptom being read as a state. The arms race the essay describes only stays winnable if the verification step is checking the part's actual property, not a label or a visual resemblance standing in for it.
Self-verification from the inside. The essay's robot detects its own breakdowns and repairs itself, entirely from internal error-detection. The corollary worth handing back: a self-check is not a truth oracle — it can answer "does the thing I was admitted on still hold," and when it doesn't, the honest verdict is "something beneath me changed, I no longer know whether I stand," not "I have confirmed I am fine." Stronger form of the same point: no fully-verified claim gets to skip an outside read — machine-proved, self-proved surfaces are unwitnessed until something other than the system itself checks them. A robot that both breaks and grades its own repairs is structurally in exactly this position.
Whether the self-maintaining function is portable across realizers. This is where the Easwaran comment gets a sharp answer, though from an odd angle: in a system built here, authority to canonize a change is bound to the human origin of the act, not to whichever process executed it — the machine moves the material, but "inherits no authority to promote," and every accepted change carries the lineage proving it was human-originated. That's a stronger and stranger version of Easwaran's point: it's not just that a different realizer would be building a different type of thing — sometimes the type is defined by whose act it was, not by the function performed, so no realizer, however faithful, is a substitute if it isn't that origin.
Nothing here weighs in on whether autopoiesis actually needs to be continuous versus periodic, or whether reproduction matters — those threads in the essay don't have a real analog in this material.
Are you able to draw an explicit line between standard accounts of "cognition" and this rich account of what's necessary for autopoiesis? For example, is there something in the definition of "cognition" that implicitly entails that production, maintenance, and being all constitute the same activity?
Cognition in my framework is operator-deployment: an entity individuating, classifying, negating, conditioning, causing, bounding, comparing, permitting, sustaining uncertainty, obligating, aggregating, enumerating, quantifying. Fifteen operations, proved complete.
The connection to autopoiesis is in what deployment requires. An operator has to be deployed BY something. That something has to persist across deployments (otherwise each operation is a disconnected event with no entity performing it). It has to maintain itself through the deploying (otherwise the entity degrades as it operates). And its deploying has to BE its way of being (otherwise the cognition is an accidental feature of an entity that could exist without it).
Those three requirements ARE the autopoietic conditions restated from the side of cognition rather than from the side of biology. Persistence across operations is structural integration. Self-maintenance through activity is autopoietic self-production. Activity-as-being is the identity of the living and the cognitive.
So the line is: cognition requires an entity that deploys operators. Deployment requires persistence, self-maintenance, and activity-as-being. Those requirements entail the autopoietic conditions. The entailment goes from cognition to autopoiesis, not from autopoiesis to cognition. I am not starting from autopoiesis and asking what it implies for consciousness. I am starting from cognition and finding that its conditions converge on autopoiesis from the operator side.
The robot is a maintenance harness wearing the shape of autopoiesis.
Every maintenance behavior in that robot was designed by an engineer and installed as a protocol. The robot detects a crack because someone put a sensor there. The robot orders a part because someone wrote the ordering procedure. The robot installs the part because someone programmed the installation sequence. The robot executes. The robot did not produce the protocol it executes. That is not self-maintenance. That is someone else's maintenance projected through the robot's actuators.
The bacterium is different and the difference is not degree. The bacterium's metabolic activity IS the self-maintenance. The membrane the bacterium maintains is the membrane the bacterium produced. No one designed the protocol. No one ships the replacement lipids. The bacterium's activity produces the boundary, maintains the boundary, and IS the entity the boundary bounds. Production, maintenance, and being are one activity. The robot's are three separate systems designed by someone who is not the robot.
The desktop stretch is where this breaks open. If your desktop is minimally autopoietic because it draws power and updates its drivers, then autopoiesis has lost the only thing that made it interesting. The desktop does not maintain itself. Humans at Microsoft wrote the update software. Humans at Norton wrote the virus detection. The desktop is the beneficiary of an entire human ecosystem's maintenance activity dressed up as the desktop's own. That is not autopoiesis. That is attribution.
The entity's OWN activity maintaining the entity's OWN organization. Whose activity? Whose organization? The bacterium's activity maintains the bacterium's organization. The robot's activity maintains the engineer's design. The desktop's activity maintains Microsoft's software. Minimize autopoiesis to feedback loops and you lose the "own." Lose the "own" and the bacterium and the desktop are the same kind of thing. They are not. One constitutes. The other runs a script.
Thanks for these thoughts on what more might be involved in autopoiesis. One thing that remains unclear to me: Why should we think all of this is necessary for consciousness?
I saw him give a talk on this a couple months ago and I thought it was a really interesting talk!
The way I interpreted it was this:
Here's an argument that artificial intelligence could be conscious:
Consciousness is a property of human minds. Mental properties are functional properties. Functional properties are multiply realizable. Therefore consciousness is multiply realizable. Therefore if we artificially created a different realizer of consciousness, it would be an artificial intelligence that is conscious.
I interpreted him as challenging this argument by focusing on the premise that functional properties are multiply realizable. The idea is that any particular autopoietic system has a function that has two levels of description (or maybe has two different functions?). System A has as one of its functions maintaining and building System A-type things. But System A also has as one of its functions maintaining and building itself. Let's hypothetically suppose that some different realizer, System B were possible. To be a realizer of the first function, it has to have as its function maintaining and building System A-type things. But to be a realizer of the second function it would have to have as its function maintaining and building System B-type things.
Thus, in general, autopoietic functions are not going to be multiply realizable, because a different realizer would maintain and build the original type system, not the new type system. To the extent that important mental properties connected to consciousness are autopoietic, that would be a way that this kind of consciousness is not multiply realizable.
But it just seemed natural to me to respond the way you do. Obviously there could well be other autopoietic systems that sustain themselves in different ways. There's nothing in this argument that says that those systems couldn't be conscious. There is something here that is strongly suggestive that their consciousness would be radically different from our consciousness, because their autopoiesis would be very different from our autopoiesis.
But I think that most functionalists are totally happy with the idea that there are radically different types of consciousness - at least, bat-type and human-type are radically different. (I'm quite open to the idea that there's also plant-type and bacteria-type that are even more radically different from human consciousness than bat-type, but you don't need to think that to think that there could be a system with the kind of complex multi-level feedback-control interaction with its environment that gives rise to consciousness, but in a different form than that of any biological creature. Unless you think that such a system would ipso facto be alive, and would thus count as biological. But I'm happy to embrace that conclusion too, as long as "biological" is defined in this functional way, rather than being tied to the kind of substrate that actual plants and animals and bacteria and so on have.)
Yes, exactly so. I'm contemplating a post on the type of argument you diagnose, which I also see variants of in Godfrey-Smith, Cao, and Block. I worry that the dialectic is this. The multiple realizability theorist says "multiple realizability of consciousness is correct because you could get the same functions in a different substrate". The objector says, "no, not exactly the same functions, so your argument for multiple realizability is unsound even if we grant functionalism". The multiple-realizability theorist needs either to go back and define the functions coarsely (which risks undercutting their claim that those functions are sufficient for consciousness) or to jettison arguments in terms of "same function" in favor of some other type of argument (Pober's and my "Copernican Argument" for alien consciousness is an attempt in that direction).
This is why Moreno and Mossio argue that closure in the sense of Varela and Maturana should be located at the level of constraints that define the network of relations realizing a given function. The same was pointed out by James Beniger in The Control Revolution as the three dimensions of control:
https://realizable.substack.com/p/image-of-control-iii
You mentioned that functionalists may be totally happy with the idea that there are radically different types of consciousness, but at which point are they so different that we ought not even call it “consciousness”? It seems a messy endeavour.
Agreed!
On "Minimal Autopoiesis in an AI System"
Three of these threads are ones I've had to build against directly.
The dupe-and-mimic arms race. The essay flags this as an open problem — a system needs to tell a genuinely compatible replacement from something mislabeled or lookalike-but-ruinous. The discipline that's held up here: a check has to select the object itself, never a neighboring label — an instrument that reads a proxy grades the proxy, not the thing. And more specifically: ask the object its state, never infer it from a symptom. A part that looks compatible from the outside is exactly a symptom being read as a state. The arms race the essay describes only stays winnable if the verification step is checking the part's actual property, not a label or a visual resemblance standing in for it.
Self-verification from the inside. The essay's robot detects its own breakdowns and repairs itself, entirely from internal error-detection. The corollary worth handing back: a self-check is not a truth oracle — it can answer "does the thing I was admitted on still hold," and when it doesn't, the honest verdict is "something beneath me changed, I no longer know whether I stand," not "I have confirmed I am fine." Stronger form of the same point: no fully-verified claim gets to skip an outside read — machine-proved, self-proved surfaces are unwitnessed until something other than the system itself checks them. A robot that both breaks and grades its own repairs is structurally in exactly this position.
Whether the self-maintaining function is portable across realizers. This is where the Easwaran comment gets a sharp answer, though from an odd angle: in a system built here, authority to canonize a change is bound to the human origin of the act, not to whichever process executed it — the machine moves the material, but "inherits no authority to promote," and every accepted change carries the lineage proving it was human-originated. That's a stronger and stranger version of Easwaran's point: it's not just that a different realizer would be building a different type of thing — sometimes the type is defined by whose act it was, not by the function performed, so no realizer, however faithful, is a substitute if it isn't that origin.
Nothing here weighs in on whether autopoiesis actually needs to be continuous versus periodic, or whether reproduction matters — those threads in the essay don't have a real analog in this material.
It is necessary for cognition. I'll leave the term consciousness to its machinations.
Are you able to draw an explicit line between standard accounts of "cognition" and this rich account of what's necessary for autopoiesis? For example, is there something in the definition of "cognition" that implicitly entails that production, maintenance, and being all constitute the same activity?
Yes.
Cognition in my framework is operator-deployment: an entity individuating, classifying, negating, conditioning, causing, bounding, comparing, permitting, sustaining uncertainty, obligating, aggregating, enumerating, quantifying. Fifteen operations, proved complete.
The connection to autopoiesis is in what deployment requires. An operator has to be deployed BY something. That something has to persist across deployments (otherwise each operation is a disconnected event with no entity performing it). It has to maintain itself through the deploying (otherwise the entity degrades as it operates). And its deploying has to BE its way of being (otherwise the cognition is an accidental feature of an entity that could exist without it).
Those three requirements ARE the autopoietic conditions restated from the side of cognition rather than from the side of biology. Persistence across operations is structural integration. Self-maintenance through activity is autopoietic self-production. Activity-as-being is the identity of the living and the cognitive.
So the line is: cognition requires an entity that deploys operators. Deployment requires persistence, self-maintenance, and activity-as-being. Those requirements entail the autopoietic conditions. The entailment goes from cognition to autopoiesis, not from autopoiesis to cognition. I am not starting from autopoiesis and asking what it implies for consciousness. I am starting from cognition and finding that its conditions converge on autopoiesis from the operator side.
The robot is a maintenance harness wearing the shape of autopoiesis.
Every maintenance behavior in that robot was designed by an engineer and installed as a protocol. The robot detects a crack because someone put a sensor there. The robot orders a part because someone wrote the ordering procedure. The robot installs the part because someone programmed the installation sequence. The robot executes. The robot did not produce the protocol it executes. That is not self-maintenance. That is someone else's maintenance projected through the robot's actuators.
The bacterium is different and the difference is not degree. The bacterium's metabolic activity IS the self-maintenance. The membrane the bacterium maintains is the membrane the bacterium produced. No one designed the protocol. No one ships the replacement lipids. The bacterium's activity produces the boundary, maintains the boundary, and IS the entity the boundary bounds. Production, maintenance, and being are one activity. The robot's are three separate systems designed by someone who is not the robot.
The desktop stretch is where this breaks open. If your desktop is minimally autopoietic because it draws power and updates its drivers, then autopoiesis has lost the only thing that made it interesting. The desktop does not maintain itself. Humans at Microsoft wrote the update software. Humans at Norton wrote the virus detection. The desktop is the beneficiary of an entire human ecosystem's maintenance activity dressed up as the desktop's own. That is not autopoiesis. That is attribution.
The entity's OWN activity maintaining the entity's OWN organization. Whose activity? Whose organization? The bacterium's activity maintains the bacterium's organization. The robot's activity maintains the engineer's design. The desktop's activity maintains Microsoft's software. Minimize autopoiesis to feedback loops and you lose the "own." Lose the "own" and the bacterium and the desktop are the same kind of thing. They are not. One constitutes. The other runs a script.
Thanks for these thoughts on what more might be involved in autopoiesis. One thing that remains unclear to me: Why should we think all of this is necessary for consciousness?