Reification and the Consciousness of the Tech Bro
Demystifying the Singularity
Henry Farrell and Cosma Rohilla Shalizi begin their new paper, “AI as Social Technology,” by noting that “debates about ‘AI’ grow out of 1990s science fiction”—specifically, that of Vernor Vinge, the mathematician and novelist whose writings birthed the version of the “Singularity” that still haunts our collective imagination. The “sweeping visions of how information technology will completely transform society, economy, politics, or all three” that dominate public discourse are to a large degree derived from texts that, as Farrell and Shalizi point out, predate the rise of the currently prevalent technology of AI (Large Language Models) by decades. “It is,” they say, “a scandal that this dream of the ’90s is still alive and shaping debate.”
The paper goes on to argue that
it is a category error to think of ‘large models’ as self-motivated agents in the making. Instead, they are better understood as ‘cultural’ and ‘social’ technologies, resembling libraries and languages on the one hand and markets and bureaucracies on the other.
Put simply, Farrell and Shalizi argue, “LLMs create social relations between their users and the authors of the text in their training corpora.” What appears, through the prism of the Singularity myth, to be the birth of a divine entity that will supersede and replace humanity is in fact the latest phase in the mediation of human social relations by complex, partly self-regulating systems, a phenomenon as old as human society itself whose evolution has been accelerated by the technological explosion of the past few centuries.
Farrell and Shalizi’s effort at demystifying the Singularity seems to me both correct and helpful. The “actual implications” of Large Language Models, as they say, “are obscured by the mythic garb it is swaddled in.” The question, however, is why this technology appears to us cloaked in this particular “mythic garb.” The authors gesture at the “complicated social and cultural reasons” why “the progenitors and funders of modern generative AI bought heavily into this mythology,” but don’t elaborate. I have a few thoughts.
“Social relations swaddled in mythic garb”: Anyone conversant with the standard Marxian account of commodity fetishism will recognize this as a familiar scenario. Here is Georg Lukács’s formulation of it in his “Reification and the Consciousness of the Proletariat”: “The essence of commodity-structure … is that a relation between people takes on the character of a thing and thus acquires a ‘phantom objectivity,’ an autonomy that seems so strictly rational and all-embracing as to conceal every trace of its fundamental nature: the relation between people.” LLMs offer a straightforward exhibit of this fetishistic logic: artifacts of human social labor, these technologies are perceived as autonomous “agents” pursuing a self-propelled trajectory to which we must acquiesce. The Singularity myth is simply the specific narrative form this mystification takes today.
Although versions of the Singularity can be traced back to midcentury computer science pioneers like John von Neumann and I.J. Good, its locus classicus is, as Farrell and Shalizi say, Vernor Vinge’s 1993 paper “The Coming Technological Singularity.” What is striking about this text today is that the certainty of Vinge’s prophecy—“We are on the edge of change comparable to the rise of human life on Earth”—is in tension with the utter vagueness in his account of how exactly the change will occur. Not only is there no inkling of the advent of Large Language Models, “superhumanly intelligent” computers are only one of the four scenarios he lays out for a possible Singularity.
The other three Singularity scenarios considered by Vinge are the following:
Large computer networks and their associated users may “wake up” as superhumanly intelligent entities.
Computer/human interfaces may become so intimate that users may reasonably be considered superhumanly intelligent.
Biological science may provide means to improve natural human intellect.
Needless to say, these visions differ dramatically from the most prevalent one today, which inevitably involves the total technological obsolescence of humanity. In the latter two versions proposed by Vinge, humans are not merely “in the loop”: The Singularity is defined in them as a drastic form of human cognitive enhancement. The first is fuzzier, but still includes human “associated users” in the “superhumanly intelligent” collective.
It is worth adding that other influential accounts of Singularity elaborated in the same period were far closer to Vinge’s forgotten versions than to the one that eventually prevailed. For instance, Manuel DeLanda’s War in the Age of Intelligent Machines (1991) discusses the concept at length, but without any inkling of the teleology of absolute technological supersession that later eclipsed the other possible prognoses. DeLanda’s account hews closer to the definitions of the concept previously deployed in physics and systems theory, as a “critical point” at which “a set of previously disconnected elements converges into a synergistic whole.” “Singularities,” writes DeLanda, “are involved in self-organizing processes at many different physical scales and levels of complexity.” This account converges partly with Vinge’s three hybrid models, as well as with Farrell and Shalizi’s view that the Singularity already happened two centuries ago with industrialization, which irreversibly fused human social collectives with the operations of complex machinery.
So why did Vinge’s fourth and most extreme scenario, the one centered on “computers that are ‘awake’ and superhumanly intelligent,” become the exclusive definition of the Singularity in the ensuing decades? The answer is certainly not that it turned out to be the one that most closely resembled what actually happened during that period. In terms of what has actually occurred since 1993, the enhancement of human/computer interfaces clearly corresponds most closely to reality. And today, even the most fanatical enthusiasts of LLMs still talk about superhuman intelligence as something they will achieve in the future, not something already accomplished.
The better explanation of why the superhuman machines scenario prevailed is that it is, in Marxian terms, the most fetishized one: the one in which, as Lukács puts it, “man’s own activity, his own labour, becomes something objective and independent of him, something that controls him by virtue of an autonomy alien to man.” Given the nature of our political and economic order, it is only natural that this vision has come to the fore. However, it is also notable that those most in thrall to the fetishism of the Singularity are the leaders of the tech industry themselves, who seem to passionately believe in AI as, in Farrell and Shalizi’s words, an “alkahest that will dissolve the dross and cruft of human institutions, leaving only pure, undiluted progress.” By contrast, the public at large seems unenthusiastic about this ersatz utopia. This wariness, if it can be mobilized politically (a big if), offers the most likely path for shifting debate away from “the countdown to infinity” and towards “what is actually happening to human society and institutions right now.”
This week in Compact:
Me on Ivan Illich and Pope Leo’s encyclical
John Byron Kuhner on postliteracy and Nolan’s Odyssey
Julia Yost on tradsploitation
Scott Yenor on Title IX
Helen Andrews on yuppies