How is it that sodium and potassium and calcium ions shuttling back and forth through holes and neural membranes—or hundreds of neural membranes, or thousands, or millions; whatever it might be—how do you go from that description of the brain in a principled way to: I’m now experiencing the taste of chocolate? Without waving your hands, pulling a rabbit out of the hat—no magic. We want a mathematical, precise theory.
One of the interesting implications of that definition—I discovered as I was playing with it—is that when I took two conscious agents and had them interact, and looked at the system of two conscious agents, that system actually satisfied the definition of being a single conscious agent. And so this continues ad infinitum: you can keep taking pairs of agents and combine them. And so this suggests a really interesting structure.