By Leonardo Andrade
[SPOILER WARNING: SOMA]
"From the moment I understood the weakness of my flesh, it disgusted me. I craved the strength and certainty of steel. I aspired to the purity of the blessed machine. Your kind cling to your flesh as if it will not decay and fail you. One day, the crude biomass you call a temple will wither and you will beg my kind to save you. But I am already saved. For the Machine is Immortal."
The creed of the Warhammer 40k tech-priest Magos Mecanicus Faustinius isn’t too far off from the beliefs espoused by some wealthy, influential tech oligarchs. Elon Musk is on record saying humanity must merge with computers to “stay relevant” in a future where AI far exceeds our capabilities. The choice is between symbiosis and being permanently ejected from the driver's seat of our lives and collective destiny. That guy whose main claim to fame is stealing self-driving car tech from Google has been trying to make an actual AI church happen for a decade, falling over himself to give up control to the machine-god he sees as inevitable.
Meanwhile, the Carbon Copies Foundation has as its mission statement freeing us from weak, forgetful flesh “ that evolved to fit a niche in the early Pleistocene, 2.5 million years ago” and enabling our minds to inhabit constructed substrates. “The machines we design are able to upgrade and advance. Outcomes and experience with a computer program can improve simply by running it on a better processor. Unfortunately, the biology of a brain is not equipped for back-ups or fundamental improvements,” the Foundation states.
Adherents to these beliefs reductively view the body as a simple “substrate” they unfavorably compare to machines based on arbitrary metrics like the “output rates” of typing versus uploading. This viewpoint is clearly in conversation with science fiction, both new and old. Many recent, successful pieces of media–such as CD Projekt RED’s Cyberpunk 2077 videogame and AMC’s animated series Pantheon–emphasize freeing the mind from the supposed weakness of flesh via cybernetics (some of which are consumer products, even!). But the best fiction about the subject, regardless of medium, never presents its scenarios as blueprints. Instead, these works interrogate the possibilities and the issues of said technology. Rushing to turn these fictional imaginings into reality without the same wary skepticism is short-sighted at best and denotes a degree of media illiteracy. The Adeptus Mechanicus are not supposed to be inspirational. At this point, cautioning Big Tech against building the Torment Nexus has become a cliche.
Some other popular sci-fi games with less gonzo, satirical roots include stark drawbacks to the wholesale rejection of the human body. The entirety of 2015 survival horror game SOMA, for instance, deals with the fallout of ripping a mind from its original body. The legendary Cyberpunk games feature Cyber-Psychosis since the first edition of the original tabletop RPG. Not even sci-fantasy is safe: getting much too cybered up the Shadowrun—originally a tabletop RPG like Cyberpunk,and likewise currently more well-known for its videogame adaptations—universe can dull or even sever your connection to your soul.
My preferred reading of these is that they represent the tension between a computational theory of mind and an embodied one. If the base assumption this type of tech people make is that the mind is just software running on a fat-based computer (the brain) and the body is nothing more than a set of peripherals, there would not be any complications to trading its parts or running the mind on a better, perhaps silicon-based computer. These fictions make more sense from the standpoint that human cognition is embodied rather than exclusively computational. This doesn’t necessarily mean that no computation takes place as part of the cognitive process, but that it is only one small part of the whole. In other words, the brain does some things that computers also do, but it also does much more than that in ways that are informed by the body the brain is in, so something important might be lost when a mind is “ported” to a cybernetic platform.
Embodied cognitive science is a school of thought and research program that posits that the body’s sensorimotor capacities and its interactions with the environment shape cognition. To quote the thesis statement of one of the most influential books on the topic, Lakoff and Johnson’s Philosophy in The Flesh:
“Reason is not disembodied, as the tradition has largely held, but arises from the nature of our brains, bodies, and bodily experience. This is not just the innocuous and obvious claim that we need a body to reason; rather, it is the striking claim that the very structure of reason itself comes from the details of our embodiment. The same neural and cognitive mechanisms that allow us to perceive and move around also create our conceptual systems and modes of reason.” (page 3)
I find it very convincing, but it is far from a settled debate, especially when we add technology that does not (yet?) exist to the mix. It is impossible to make hard, definitive statements about this in honesty. We are left in the realm of speculation. As usual, science fiction treads where science has yet to triumph. Let’s focus on SOMA specifically.
In SOMA, you play as a man called Simon Jarret in the prologue, going to a doctor’s office to get an experimental total brain scan that could help find a treatment to save Simon’s life from severe brain damage sustained in a car crash. When the scan is over, you play as an instance of his digitized brain (called a neurograph), fully conscious, running on a computer chip jammed into the spinal cord of a beheaded body that has been reanimated with futuristic “structure gel”. You don’t know this, though. For quite a while, you think you are Simon, because the memories up to the moment of the scanning are the ones the digitized brain has when it comes to. In truth, however, Simon has been dead for a century, and in this future, humanity as you knew it no longer exists.
A meteor destroyed all life on the surface, and the few people living in the underwater research station PATHOS-II, where the game takes place, have all died or mutated beyond recognition due to misuse of structure gel. Many of them had their brains digitized before, and copies of their neurographs were stored in different substrates by PATHOS-II’s AI life-support system, WAU (Warden Unit), as it attempted to keep humanity alive. These neurographs have robots for bodies, except for Simon’s—you, the player. This made them all lose touch with reality or become violently insane (it is a horror game, after all). The only two stable post-humans are you and the neurograph of Catherine Chun, which you carry in your pocket computer (omnitool) after finding her in a damaged robot—fan theories abound as to why she remains stable.
The main goal you pursue was her goal before you showed up and joined forces: To get instances of all the neurographs into a single simulated world—where they get to have simulated human bodies in a simulated earthly environment—powered by solar panels on a tiny space satellite, which you have to fire into orbit, too.
This is as close as it gets to “saving humanity” in this game.
That’s a lot to chew on if you haven’t played the game. Although spoilers abound, I didn’t give away everything, and I encourage you to play this glorious—and genuinely quite moving—sci-fi mindfuck of a game. It handles its subject matter with care and depth that the average tech bro mind-upload pusher can’t dream of.
Typebar Magazine contacted SOMA writer Mikael Hedberg about how much, if at all, the philosophy of the mind influenced him in writing the game, and how he arrived at the central tensions of the game’s story.
“How the creative director pitched it was that he wanted us to start inside the head of a robot at the bottom of the sea, and then the game was that we got to explore the world around us. It was more about the premise of the mystery, rather than the philosophy, when we started. It felt natural that this would become the main point of the story,” Hedberg told Typebar via LinkedIn of all places. “We had already somehow become this robot, so the question was ‘what did we lose when we transferred bodies?’”
SOMA is powered by a mix of philosophies, rather than strict adherence to a given philosophy. The computational aspect is necessary for any upload to be possible, so it’s fundamental to the premise, but discard the body’s role and you lose the narrative tension that gives the story its weight.
In Hedberg’s words, “when does the human stop being human? When do we lose so much that we are unrecognizable? Basically the Ship of Theseus paradox, but with humans instead of a boat.
[...]
We were well into production when I floated the idea that we would switch bodies again and finally load our consciousness into a virtual world. Which, if we were to dissect the story now, would be considered the base of the whole story.”
The second body switch and the heavy question of what it entails—a mere transfer, as Simon wants to believe, or copying and pasting, only to ditch the first neurograph in a hopeless situation—put this on the shortlist for most underrated hard-hitting moments in gaming. It is necessary for progression, rather than a choice with varied outcomes, but it had me and many others waiting for a long time, feeling torn up about it, before we decided to do the terrible deed in order to see the rest of the game.
Hedberg has an idea of why it works so well:
“The most striking part of the philosophy of mind that I encountered when making the game came very late. Even though I sort of knew it intellectually, I didn’t fully see the impact until I saw other people stream the game. The reason why the story works and why I think it is quite special in a way that most mediums can’t replicate is perspective. I find it almost difficult to put into words how important it is that the story is experienced through playing the game in a first-person view. If I were a philosopher myself, I might find better words to explain, but I think being conscious involves a lot of unexamined acceptance of what we happen to be at every moment. This is achieved by the fact that we as Simon can’t really study or reflect on our state of being visually. It helps us forget any inconsistencies and simply accept that we still are who we’ve always been.”
It’s an interesting idea to drill down further into. A way to look at this could be that the mind may protect itself by plastering the known over the unknown and refusing to engage with the disquieting change. Simon is helped by the fact that he is operating out of a human, flesh-based body plan, to keep this illusion going. And it is actually addressed in-game. When riding the zeppelin to Theta, Simon and Catherine have a conversation about it. “Maybe I was more in the know than I remembered :D,” Hedberg muses. Typebar did show up in his inbox a literal decade after the game’s release, so it is entirely possible.
It could mean that the suffering of SOMA’s monsters, and indeed their change into something inhuman, comes from the inability to do as Simon does. He has unsettling intrusions from reality into his useful delusion before he understands what is happening. Eases into it. Whereas other uploaded neurographs and structure-gelled conscious minds in bodies are dropped into reality. Could it be that the resulting trauma breaks them?
“Yes, exactly,” is Mikael’s answer.” Catherine mentions something about this when they discover that Simon has the body of her dead friend Reed and the brain of a robot. WAU tries everything to save humanity, and Simon was the first time it made a stabile [sic] ‘human’ with enough presence in the real world and an intact sense of self. [...] Unlike Simon, the monsters just had too many flaws, or were too different from their previous existence for their human mind to accept their new form, so it corrupts their consciousness to different degrees.”
Catherine was interesting to me in this rabbit hole I've gone down because she seems stable enough, even though she keeps popping in and out of existence and never in a familiar body. There is fan-theorizing about the possibility of some neurologically atypical trait contributes to making her as she is and allows her to handle the transition better. It could also just be that she was the one person who had explored the implications and acquainted herself with the philosophy the most, so her mind was more prepared. Neat thing about human minds is that we have the power to self-edit, to some extent. Despite what the Carbon Copies Foundation says, the brain changes itself all the time, improves, even—just not in the exact same way we can improve a computer.
Mikael likes all the theories on Catherine, and has no desire to influence anyone when it comes to trying to figure her out on why she is doing so well. But he has this to say, as a final thought: “How do we know that Catherine or even Simon are unaffected by their new selves? They seem to be doing better than the rest in the sense that they are stabile and are able to evaluate reality correctly. But it could be that some parts of their behavior have changed significantly, and Catherine and Simon are both so trapped in their perspective that they can't reflect on that change. Like we talked about before, we all suffer from an unexamined acceptance of our state of mind. We simply accept what we are in every moment of existence.”
Would a mind changing into a new category of mind even realize it? And how much change would tip one over to the realm of the post-human? Video games are a great medium to explore these questions. In the case of SOMA, with the first-person perspective and real-time gameplay, it certainly uses our perceptions of embodiment to do it. It’s what makes it, in my opinion, an important piece of work.
How does the embodied cognitive science school of thought handle its questions? In its most extreme form, ECG can extend to include the very environment as a component of cognition, instead of just a peripheral. In this framework, bodily interaction with the environment can make it function as an extension of the body, and therefore, play a similar role in shaping cognition. Especially in the case of humans. Humanity’s key features separating us from most other animals are:
- Our ability to outsource cognitive tasks to the environment—we write to store information off-brain AND to aid the brain in retaining knowledge; most people have an easier time doing maths by hand than in their heads, etc.
- Our ability to alter the environment and then use it to augment ourselves: By conceiving pointy sticks, sharp rocks, pens, steering wheels, forks, wrenches, and so on as extensions of ourselves, we can do so much more than a bare human body. Some other animals have tool use—monkeys, birds, octopi (now imagine how having their neurons mostly distributed in ganglia in their tentacles shapes their cognition! Freaky stuff)—but none supposedly as refined as ours. Note that superior tool use doesn’t mean a guarantee of the best possible outcome. For all the good it has done, it has also had the downstream consequence of wrecking the planet.
Eventually, these two lead to glasses, hearing aids, prosthetics, motorbikes, the omnipresent cellphone, and…mantis blades?
One could make the argument, and indeed, metaphysicist Andy Clark does, that humans are Natural Born Cyborgs. In the book, he goes over how the traits listed above, our brain plasticity, and modern technology interact to create hybrid minds. He argues that we are already undergoing cyborg-ification.
Do you have all your appointments, passwords, professional knowledge, and to-dos in your head at all times? I don’t. I have them in computers. I also wear lenses over my eyes whenever I’m awake and a gizmo that blasts the amplified sounds of the world right into my faulty auditory ossicles, and I’m learning how to drive a car. A lot of it is feeling the machine, as many of you reading this know.
All of this is generally done by addition rather than replacement, however, and what does get replaced is usually faulty or nonfunctional. Replacing the body as a means for upgrading capabilities remains an unexplored frontier. What, then, is the problem with jamming the internet right into our brains without middleware like phones and laptops, becoming Adam Smasher, or living forever by uploading your mind?
The trauma angle discussed with Hedberg can conceivably happen within the confines of ECG. A total bypass or wholesale discarding of the body could be a very traumatic experience, even if not in a single, spectacular monster-making fashion. The real world does not operate based on genre conventions, but stories often do.
More interestingly, however, the resulting cognition of an Adam Smasher or fully disembodied mind-upload might cross some threshold into no longer being recognizably human. If there is such a threshold, what makes it so that it must reside in one of these extremes and not in, say, someone who’s only 50% cyberpunk chrome? I cannot imagine the physical and emotional sensations that could come from replacing a lot of my skin, muscle, and nerves with metal and circuitry. I’m also not too eager to risk having spam beamed directly into my brain or getting hooked on doomscrolling at the speed of thought, even if it does let me pick up new skills faster or never ever be bored again. The symbiotic relationship with the smartphone is already a very mixed bag.
…
Some of you might be uncomfortable by now with how this discussion of loss of humanity, or fundamental change into something Other, can potentially veer into ableist and even queerphobic terrain. I know some people in my circles did. Typebar reached out to queer game writers Albert Bassili (Cairo: Otherscape) and Ellis Devereux (GIRLKILLER) over Discord for comments on this issue. Both of them argue that, in the future, such technologies as we discuss in this essay might be empowering and life-affirming for the queer and disabled communities, like current versions already are. It has been a theme in Albert’s work. Cairo: Otherscape features a heavily cybered-up faction that chose to be this way in defiance of traditional views of religion and gender in the setting.
SOMA does not argue the inherent inhumanity of a pacemaker, prosthetic limbs, or even the Neuralink technology, nor do the other two series I mentioned earlier—Cyberpunk and Shadowrun. But they all busy themselves with worst-case scenarios of futures in which cyberpunk-style augmentations exist. It is important to use nuanced language when discussing these topics due to the risk of reproducing damaging patterns of thought. In cyberpunk, where the threshold makes itself known through the phenomenon of cyberpsychosis, there certainly has been a qualitative jump in this regard between Cyberpunk 2013 (1988):
“Something happens when you start adding metal and plastic to people. They start to change. And it isn't pretty. In 2013, we call this cyberpsychosis; a mental disease in which the addition of cybernetics causes an already unstable personality to fragment. At first, the victim begins to relate more to machines than to humans. Soon, he starts to ignore people-parents, friends, lovers. Eating, sleeping all become less important. Finally, human interactions begin to irritate, culminating in a terrifying rage that consumes the victim entirely.
So, how do I get cyberpsychosis?
Every character in Cyberpunk has an Empathy Stat (EMP). This Stat is a measure of how well the character relates to other people, and is the basis of such skills as Leadership, Lying, Convincing and Seduction. Likewise, every piece of cybernetic equipment has a corresponding Humanity Cost, which is added together to get an overall Humanity Cost of your enhancements. For every ten points of Humanity Cost, the character loses one point of Empathy. Example: I add four new cybernetic devices for a total Humanity Cost of 36. I will lose 3 points of Empathy.” (pages 19-20)
And Cyberpunk RED (2020):
“Humanity Loss is defined (for this purpose) as a loss of empathy for others and a corresponding loss of self-regard or sense of self preservation. Subjects with low Humanity have trouble emphasizing with themselves or others as "real." Instead, they start to see themselves or others as collections of parts instead of living, breathing organisms. This is basically a form of dissociative disorder.
[...]
Cyberpsychosis comes about when the subject begins to compulsively alter the body beyond the human baseline. Seeing the body as a thing—a form of Dissociative Personality Disorder—they change it without thought.
Why this doesn't count for people who have non-voluntarily needed cyberware:
Replacing a lost or damaged body part with a new cloned part or Medical-Grade Cyberware (only 50eb to purchase separately, cost is included in the cost of a hospital visit in the rare circumstance when a cloned limb isn't available) will not increase dissociation. This is because the replacement of the body part makes the person feel "whole" again, increasing their level of body awareness. Now, if they replaced that limb with a cyberarm with knives in the knuckles—that choice was voluntary because it was excessive augmentation, and will thus come with Humanity Loss.” (Pages 230-232)
RED defines the issue of cyberpsychosis a little better and addresses the disability and medical need angle in one fell swoop. The same logic of using cyberware to achieve wholeness can, in my opinion, extend to gender-affirming care (which is healthcare). Ellis had one grain of salt to add:
“I suppose my only qualm with it in that case is that it's hard to know where the line is. Like, I have a friend who is "nonbinary" in that they are essentially dissatisfied with having a human body at all and would probably, if they could, replace themselves with machine parts - and there are many people who probably feel similarly, of wanting to e.g. have wings or horns or things that are not normally considered gender affirming care because they are not possible/not considered in the standard understanding of gender”
Gender is a fuzzy thing at the intersection of biology—the actual brain and body—and social constructs (environment?), and “how much is too much” is often a very hard question to answer with precision.
The expanded definition of cyberpsychosis in RED also clearly states that cyberpsychosis is not necessarily violent, and that loss of the Humanity resource may also come as a result of trauma. Albert, without referencing this, did muse that perhaps cyberpsychosis being its own thing wouldn’t work in the real world, and it may well be simply a variant of PTSD. It does harken back to trauma-broken WAU-bots from SOMA
Abrupt, radical body change, even without cyberware, can be challenging for the mind to handle, even if it doesn’t obliterate the “humanness” of the subject. Human-essentialism is not needed as a starting point for caution about technoutopian promises to do away with the body, mostly or totally.
The insights from ECG and the speculative points drawn from these recent sci-fi games should provide ample grounds for questioning bold, sweeping statements from tech-billionaires about the nature of the mind and its relationship to the body. It should make you wary of their sales pitches for the future of humanity. This wariness should be tempered by critiques made with disability and queerness in mind, but not outright discarded like the billionaires want you to do. The cybered-up body and the disembodied mind might seem like a faraway concern, but there are powerful forces at work who want to shorten this timeline. The question of “do you want what they are selling?” could become pressing within our lifetime.
