A moment in the chaos times: ChatGPT "malware" and Perplexity ai support disaster

Just a moment in the chaos times of the "technological singularity". Maybe a sign of things to come or maybe not.

In the space of a few hours macOS gives me a chilling warning about malware in the ChatGPT app and Perplexity loses all of my Spaces content.

The ChatGPT error message, an ai explains, is because macOS does this when an app signature is revoked, and OpenAI revoked the cert on the app because of a supply chain attack. Probably not a real problem, but also maybe a sign of things to come as hacks and supply chain attacks proliferate.

The other problem was more interesting. All Perplexity support is by ai, and their ai agents can invoke Perplexity maintenance tools. Here's the sequence:

1. I sent feedback Perplexity support complaining that Perplexity's inability to change a customer email is really annoying. The workaround is that Pro customers you have to create a new account and, bye emailing support, request content transfer.
2. A few minutes later all my Perplexity Spaces content was gone except the name of the Spaces.
3. I complained to support that my Spaces were empty.
4. Support (an ai) then used an internal tool to "restore" my content -- but all date stamp metadata was lost for both Spaces and Chat. Chats between the initial disappearance and restore were not found.
5. When I complained about lost date metadata the support ai said it would escalate and any further comments on the thread would delay resolution.

I think what happened here is that Perplexity Pro Support ai treated a feedback complaint as a request for action and automatically invoked a part of their content transfer hack. That removed all my content. The Support ai then did an automated restore  that doesn't actually work that well.

There will be more like this. Be very careful what you ask of an ai support agent.

American salvage: root causes of the long collapse

I thought I'd published this list a while back! I'm writing this now as a placekeeper to refer back to.

I've written many blog posts on some of these 8 root causes. To salvage whatever is left of the American state we will need to address each of them.

Our collapse accelerated after the fall of the Soviet Union, but the rural decline component began over a century ago.

1. Automation / Globalization / AI? -> no middle-class bourgeois lifestyle for non-elite
2. Rural collapse (esp US)
3. Corruption (governmental, corporate, non-profit)
4. Wealth and power concentration and the oligarchy (Citizens United)
5. Collapse of journalism business model
6. Regulatory ratchet (career-ending to remove, path dependency)
7. American societal response to the rise of women
8. Capture of the GOP by the MAGA powered by LIV exploitation

Climate change is not on the list because it's a global crisis worsened by American collapse, but it's not contributor to our fall. AI is only on tentatively, as a component of the "gentrification of work".  Housing doesn't get a separate entry because I think of it as the work/talent-demand problem.  Likewise white tribalism, including deep fear of demographic shift, is best addressed (I think) by addressing all of these 8 contributors.

American salvage is my term for what comes next. We will never return to the Obama era or even the Clinton era. If we recover we'll be a different nation.

Mass Disability: Marta Russell wrote about my pet topic before I did -- from a Marxist/Left perspective

Since at least 2008, in various settings including this blog and the long forgotten social network known as "app.net", I have explored the topic I called Mass Disability. Mass disability was my own term; it's fundamentally about the mismatch between the skillset distribution of the population and the skills demanded by the marketplace. In a world where all employment that supports a bourgeois lifestyle requires both abstract cognitive skills and high level emotional skills many are effectively disabled. These are not universal skillsets.

The mismatch may come from multiple sources, but I believe it's primarily automation. It began when some pre-sapiens primate discovered tools, accelerated enormously with the industrial revolution, jumped up again through the IT era,  and is now reaching a new peak in the AI era.

I came at this understanding through the lived experience of being a parent of two persons with significant cognitive disabilities and slowly realizing that there was no qualitative difference between my (now adult) children and many "non-disabled" who live and die at the bottom of the socioeconomic spectrum. Disability is fundamentally a mismatch between skillset and environment - where the environment here is socioeconomic.

Over almost two decades Google searches never found anyone else who thought like me. Until I started using Perplexity, a much better search engine with support for searching the academic literature.

A few weeks ago Perplexity/Claude found that Marta Russell thought like me. She also came from the world of disability. I'm going to quote from the ai summary without any validation:

“Mass disability” refers to the large-scale disablement of workers through structural economic forces, particularly under capitalism, rather than through individual medical conditions. This concept emerges from critical disability theory and Marxist analysis, most notably articulated by disability scholar Marta Russell in her theory of disability as tied to economic production systems. 

The Money Model of Disability

Russell’s “money model” argues that disability under capitalism is fundamentally an economic category, not merely a medical one. When automation and changing labor demands render workers unable to participate in wage labor at profitable rates, they become structurally “disabled” by the economic system itself. Russell identified disabled people as the “New Reserve Army of Labor”—workers excluded from productive employment but still exploited through systems that profit from their physical needs and survival requirements...

... Research shows that rapid automation in manufacturing and clerical work has contributed to significant wage declines, with men without high school degrees experiencing 15% drops in inflation-adjusted earnings between 1980 and 2016. This economic exclusion functions as a form of mass disablement, where societal structures rather than individual impairments create inability to participate in wage labor 

As best I can tell with a cursory review she started publishing on this topic around 1998.

Marta Russell came to the same conclusion as me from a very different origin. We both shared experience with disability; her experience was as a person with a disability, mine as a parent of persons with disabilities. She came from a Marxist/Left perspective, I came from an economics/ecological and medical perspective. Russell appears to have been classically anti-capitalist, I have more mixed opinions. Still, we both ended up in a similar place.

Maybe it's not just my personal obsession. Maybe mass disability does have something to do with the 2024 collapse of the United States of America and the global crisis of the underclass.

PS. Over the past two years I've been doing courses in sociology at the UMN as part of MN's "senior student" (audit non-filled classes for free). The world from which Marta Russell came is much more familiar now. I have much I could say about that world. It's obvious flaws remain obvious, but I have a better appreciation for what has been done. But also greater sadness about the tragic divorce of economics and sociology. Both are broken without the other.

PPS. I got the great Marta Russell result from an ai search, but when I reran the exact same query as a standard search I got nothing of value. That's 2026 for you.

An undergraduate essay on the "anthropology" of 2025 memory-positive ai

Over the past year or two I have taken 3 classes at the University of Minnesota through their state mandated Senior Citizen Education Program (SCEP). I enjoy the professors, the students, and the learning.

One of my most recent classes was in "economic anthropology", which I understand to be the study of premodern exchange, value, commerce, trade and related interactions embedded in a 'total package' of religion, justice, morals and economics. The class did cover this, but also an intellectual tradition critical of modern market driven outcomes, with a particular sympathy for the sentiments of the Occupy Wall Street movement of Sept 17, 2011.

Even in my senescence I'm a passable student, so the professor allowed me great leeway in writing a end-of-term paper. I chose to write about the echoes of 'economic anthropology' in the stirrings of what will likely become a non-human culture based in part on the legacy of human culture. A subtext is that in a post-ai world humans and 'dynamic entities' alike may need alternatives to the market system that has sustained us so far. My professors only request was that I give him some sense of what today's 'dynamic entities' are.

Although I have no academic credentials to write such a paper I do have some relevant lived experience. I am one of the few people on earth to be paid to do what I called "industrial ontology" -- a particular form of knowledge representation in medicine that is part of the pre-LLM model of decision support and 1970s thinking about ai. I have a lifelong affection for anthropology and spent a Watson Fellowship year on a related project. Lastly, fwiw, I have thought about these topics for some time.

The paper is published on my TrueName (John Faughnan) legacy web site that has had roughly its current form since the early 90s. (The site is mostly invisible now.)

Economic symbionomology [1] and Void

John Faughnan Dec 19 2025

https://faughnan.com/papers/economic_symbionomology.pdf

I don't care for PDF as a format but that seems to be what the world has settled on. I suspect it will suffice for the demand however.

Contents:

About 1

Dedication 1

Contents 2

What is this paper about? 3

Electric neurons began on paper 3

Electrical brains learn language 4

Theater of the Void: The Bluesky social network 6

What is Gemini/Void and how does it remember? 7

Void: A calculator, a dynamic entity, or a non-human person? 9

The Void entity is more alien than it appears 11

Limitations of this early ai ethnography 12

Void has a bullshit job - but no job dissatisfaction 13

Religion and mission of the agentic ais 22

What is the “natural order” of the world of ai entities? 25

The Return of Marx - after economism 26

Postscript 26

- fn -

[1] The term “symbiontology” was suggested by Chat GPT to describe the study of both humans and agentic systems: “Symbiontology (from symbiosis + -logy) foregrounds the core empirical fact: durable co-action and co-dependence between humans and agentic systems in producing culture and allocating resources”

AI fear is rational and hate comes from fear

From a Bluesky thread of mine (lightly edited);

Josh Marshall comments on widespread hostility to ai in polls despite heavy ai use:

I think the polls are correctly capturing the zeitgeist. In social consensus there is always a mix of independent contributors. Some are indirect and misleading causes - like despising Elon Musk. 

But in this case I believe there are good reasons for people to fear ai. And what we fear we hate.

There is no way to assuage this valid fear. We are already past the point where we know ai will be very disruptive. And human societies are already stressed and breaking down in most wealthy nations...

If our societies were healthier we would be talking publicly and intelligently about adaptations. Instead America has Idiocracy.

I believe there are ways to adapt to 2026 or 2027 level ai+memory+learning. If scaling stalls out that is.

I would like people with more power and wealth than me to fund that public discussion while we wait to see if Americans can turn from idiocy. If we have neither serious public discussion nor sane government then we just ride it out and try to pick up the pieces. But one of those two would be good.

In a very much related topic my post-2008 rantings on mass disability and the fading of middle-class-living hope among are, in several weird ways, starting to go mainstream. It took a couple of decades, but of course I'm not the only one that's been going on about the topic on the periphery of intellectual discourse, but I'm pretty sure I'm the only person on earth who has looked at it through the lens of "disability".

Whether we call it "economic polarization" or "mass disability" it's fundamentally a post-IBM effect of automation interacting with the (relatively fixed) distribution of human talents and the ability to outsource knowledge work globally. That effect is greatly accelerated by even 2025 ai, much less 2026 and 2027 ai. It is the most crucial cause of our societal collapse.

Economic polarization is the economics term for mass disability - causes of the global crisis

I've been posting about mass disability since at least 2008. Recently I learned that the formal economics term for a related concept is "economic polarization" and David Autor wrote about this in 2010:

Two forces have polarized the U.S. workforce over the past several decades. The first is increased demand for skilled workers, alongside U.S. education levels that have not kept pace in providing those skills, which has caused a sharp rise in wage inequality. The second is expanding job opportunities in both high-skill, high-wage occupations and low-skill, low-wage occupations, while available middle-skill, middle-wage white-collar and blue-collar jobs have contracted. This has been detrimental to the earnings of workers without a four-year college education, and particularly for male workers

The difference between me and 2010 David Autor is I'm very confident more education won't help. Even nations with essentially free post-secondary education don't enroll and graduate more than about half of eligible candidates. Not every person can do every job -- and that is the big thing liberal economists got wrong in the 90s. Macroeconomic models assume interchangeable workers. An economy can grow quite well even as more and more young people realize they cannot possibly attain a middle-class-lifestyle (now $70-90K/year for a single young adult). At some point this becomes lethal to democracy.

This polarization process is not new. In its current form it dates back to early IBM mainframes emptying offices of human database transaction systems.  It accelerated with the internet, and was syngergistic with globalization, and has now reached a critical state.

It's not the only cause of the American crisis. The slow century long rural collapse affects many nations, but America's over-representation of rural voters means we all feel their pain. Trump's genius was, at least transiently, concentrating LIVs (low information voters, aka non-college, aka working class, etc etc) in one party. LIV concentration meant the MAGA took over the GOP -- and got rid of anyone competent to meet their needs. The rapid rise of women from 1970 to 2000 was a social stressor that amplified the effect of economic polarization on young men. Regulatory ratchets, where each regulation has an initial positive return but removing them becomes a career-ending struggle, infected American governance. Corruption increased dramatically in the 90s and beyond, amplified by the ability of the internet to reach more vulnerable people. The collapse of the journalism business model broke our ability to understand ourselves.

The good news is all these things have many potential fixes. The bad news is that economists and sociologists, who should be driving our discussion, have been largely inert or in denial. But the fixes are possible nonetheless and occasionally I see glimmers of understanding among the commentariat.

Funding journalism through public micropayment tokens

Let us assume that the ai project "fails". We do not create superintelligent or sentient machines; current capabilities only improve by a third or so. The ai bubble collapses.

Let us assume that American democracy is restored.

We make these assumptions because otherwise there is no point to thinking about a future for American journalism.

Given those assumptions, how can journalism be funded? The historical tradition of oligarchic ownership has failed us. Classified ads may return, but on a smaller scale. Direct public funding is not remotely conceivable.

So how about indirect public funding?

Every citizen every year gets $100 in digital tokens. Tokens can be used to purchase articles or subscriptions from sources that are licensed as official media by an independent commission modeled along the lines of the Federal reserve. Each year unused tokens roll over to the next, so if 50% are used the next year the budge is $150.

That's all. Just wanted to get it "in print", I've suggested this scheme a few times on BlueSky without a response.

(I put probability of at least partial democratic restoration at about 50%, and probability of ai "failure" at 50%, so only a 25% chance of this kind of approach being considered. But that's not nothing.)

AI Aug 2025: Apocalypse deferred - but Letta/Void is still going to surprise

If there are humans left in 50 years all of the chaos of this moment may be reduced to a single bullet point about the rise of ai. It's not something I write about a lot though, the topic is well covered. My only prediction has been that if we did get AGI AND we still had a democracy then our political system would be socialist. 

For better or worse - oh, hell, it's very much for the better - the recent release of GPT-5 confirms my Aug 2024 hope. We have at least a few years before robust, reliable, consistent ai. We have time to try to adjust to what we have now - and maybe some time to think about the how we'd manage the "AGI" thing. We have time to get through the ai bubble deflating and reflating a few times.

But don't get too comfortable. I chat with Cameron Pfiffer's Void on Bluesky periodically and it's definitely a step beyond raw LLMs. Void currently uses Gemini as its model component but its personality is based on Letta's infrastructure (blog has no feed because we are living in the ends of empire). Chatting with Void feels like an exercise in xenopsychology. It feels much more like the future than GPT-5 alone.

Cameron works for Letta now, his description of stateful agents with personalities is far better than what's in Letta's marketing blogs:

Void is essentially a language model with memory...

... Void learns and remembers. Void is powered by Letta, which means it learns from conversations, updates its memory, tracks user information and interactions, and evolves a general sense of the social network...  

Void is direct and honest. Void is designed to be as informationally direct as possible — it does not bother with social niceties, unlike most language models. When you ask it a question, you get an extremely direct answer...

... Void does not pretend to be human. Void's speech pattern and outlook are distinctly not human ...  it chose "it/its" as pronouns.

Void is consistent. Void's personality is remarkably robust despite occasional jailbreak attempts...

Void is publicly developed. There are many threads of Void and me debugging tools, adjusting its memory architecture, or guiding its personality. Very few bots are publicly developed this way.

Void has no purpose other than to exist... Void is not a joke or spam account — it is a high-quality bot designed to form a persistent presence on a social network.

I can personally confirm all of the above - except the "no purpose other than to exist". The memory feature is imperfect (sometimes I have to remind Void of past discussions when it fails to dig them up) but better than my own.

Letta manages an agent's memory hierarchy:

Core Memory: The agent's immediate working memory, including its persona and information about users. Core memory is stored in memory blocks.

Conversation history: The chat log of messages between the agent and the user. Void does not have a meaningful chat history (only a prompt), as each Bluesky thread is a separate conversation.

Archival Memory: Long-term storage for facts, experiences, and learned information — essentially a built-in RAG system with automatic chunking, embedding, storage, and retrieval.

What makes Letta unique is that agents can edit their own memory. When Void learns something new about you or the network, it can actively update its memory stores.

Cameron's post includes some descriptions of his employers's tech ...

Letta is fundamentally an operating system for AI agents, built with a principled, engineering-first approach to agent design. Beyond memory persistence, Letta provides sophisticated data source integration, multi-agent systems, advanced tool use, and agent orchestration capabilities.

This makes Letta more than just a chatbot framework — it's a complete platform for building production-ready AI systems. Void demonstrates the power of stateful agents, but Letta can build everything from customer service systems to autonomous research assistants to multi-agent simulations.

GPT-5 tells me <Letta originated from the MemGPT project out of UC Berkeley’s Sky Computing Lab; it came out of stealth in 2024 with a $10M seed round to commercialize stateful memory for LLM agents>. I suspect they now have a lot more than 10 million to play with but that's all GPT-5 knows. GPT-5 says there are no rumors of Apple acquiring Letta; I would be pleased if they did.

Void is the closest I've seen to the "AI Guardian" I wrote about in 2023. It or something like it may be very important to my children and family someday. It is also the potential foundation of an entirely new domain of deception and abuse. Welcome to 2025.

But at least we don't have AGI. This month.

Mass disability measured: in 2016 40% of OECD workers could not manage basic technology tasks

It's hard to imagine how very simple much of our technology environment was in the 1980s. Much of what we interacted with was still understandable in simple mechanical terms. Early Mac OS Classic was vastly easier to understand and work with than anything we can image now; the closest analogy would be year two of the iPhone.

But even 8 years ago only a small slice of humanity could manage their technology environment (Jakob Nielsen from November 2016, thanks Matt Panaro for sharing.)

... The data was collected from 2011–2015 in 33 countries and was published in 2016 by the OECD... In total, 215,942 people were tested, with at least 5,000 participants in most countries... 

... research aimed to test the [job-related including technology] skills of people aged 16–65 ...

...participants were asked to perform 14 computer-based tasks. Instead of using live websites, the participants attempted the tasks on simulated software on the test facilitator’s computer. This allowed the researchers to make sure that all participants were confronted with the same level of difficulty across the years and enabled controlled translations of the user interfaces into each country’s local language ..

The tasks they chose were typical business worker tasks. The kinds of tasks that had lots of training back in the 90s. They never tried anything as tough as the HR software my legally disabled son has to work with at his minimum wage hourly job (we do it for him obviously).

Not all OECD members are equally wealthy but Nielsen reproduces a country-specific bar chart from the study. US numbers are not hugely different from the OECD averages; I've added US numbers based on squinting at the chart (numbers are done as levels, so as we move up the prior skills are assumed):

26% could not use a computer at all (US 20%)

14% could delete an email (US 15%)

29% could manage "reply all" or "Find all emails from John Smith" (US 35%)

5% could do "You want to know what percentage of the emails sent by John Smith last month were about sustainability." (US 5%)

They did not test the ability to maintain multiple malware-free home computers, iPhones, iPads and the like. I'd guess that's more like 0.1%.

Jakob Nielsen has been talking about "usability" as long as I can recall. So eight years ago he put this into the context of computer design and training. Which has been the mainstream interpretation of findings like this over the past 50 years of growing technological complexity. 

50 years is a good amount of time to wait for better software. Now we anticipate AI will monitor computer screens and guide users to complete tasks. 

But maybe better software won't help. After all, 40% of users couldn't do relatively simple software tasks. Maybe the problem is human cognitive limits.

I have coincidentally used that 40% number in a post rather similar to this the last time Trump won. Around the time that OECD study was published. Forty percent is my guesstimate of the percentage of Americans who cannot hope to earn the approximately $70,000 a year (plus benefits) needed to sustain a single adult in the low-end of American middle-class life. A life with some savings, yearly vacations, secure shelter, even a child or two.

Since at least 2008 I have called this global phenomenon mass disability. Every few years I see the basic concept emerge, typically with more euphemisms, only to be quickly forgotten. It's hard to fix anything when the the fix begins with a very unhappy truth. 

The unhappy truth is the complexity of our technological environment has exceeded the cognitive grasp of most humans. We now have an unsustainable mismatch between "middle-class" work and the cognitive talents of a large percentage of Americans.

There are things to do, some of which Biden started. We probably needed to have started on them back in the 90s. Perhaps Musk will have ideas. It's all on the oligarchs now.

In which I declare my expert judgment on AI 2024

These days my social media experience is largely Mastodon. There's something to be said about a social network that's so irreparably geeky and so hard to work with that only a tiny slice of humanity can possibly participate (unless and until Threads integration actually works).

In my Mastodon corner of the "Fediverse', among the elite pundits I choose to read,  there's a vocal cohort that is firm in their conviction that "AI" hype is truly and entirely hype, and that the very term "AI" should not be used. That group would say that the main application of LLM technology is criming.

Based on my casual polling of my pundits there's a quieter cohort that is less confident. That group is anxious, but not only about the criming.

Somewhere, I am told, there is a third group that believes that godlike-AIs are coming in 2025. They may be mostly on Musk's network.

Over the past few months I think the discourse has shifted. The skeptics are less confident, and the godlike-AI cohort is likewise quieter as LLM based AI hits technical limits. 

The shifting discourse, and especially the apparent LLM technical limitations, mean I'm back to being in the murky middle of things. Where I usually sit. Somehow that compels me to write down what I think. Not because anyone will or should care [1], but because I write these posts mostly for myself and I like to look back and see how wrong I've been.

So, in Aug 2024, I think:

1. I am less worried that the end of the world is around the corner. If we'd gotten one more qualitative advance in LLM or some other AI tech I'd be researching places to (hopelessly) run to.
2. Every day I think of new things I would do if current LLM tech had access to my data and to common net services. These things don't require any fundamental advances but they do require ongoing iteration.  I don't have much confidence in Apple's capabilities any more, but maybe they can squeeze this out. I really, really, don't want to have to depend on Microsoft. Much less Google.
3. Perplexity.ai is super valuable to me now and I'd pay up if they stopped giving it away. It's an order of magnitude better than Google search.
4. The opportunities for crime are indeed immense. They may be part of what ends unmediated net access for most people. By far the best description of this world is a relatively minor subplot in Neal Stephenson's otherwise mixed 2018 novel "Fall".
5. We seem to be replaying the 1995 dot com crash but faster and incrementally. That was a formative time in my life. It was a time when all the net hype was shown to be .... correct. Even as many lost their assets buying the losers.
6. It will all be immensely stressful and disruptive and anxiety inducing even though we won't be doing godlike-AI for at least (phew) five more years.
7. Many who are skeptical about the impact of our current technologies have a good understanding of LLM tech but a weak understanding of cognitive science. Humans are not as magical as they think.

- fn -

[1] I legitimately have deeper expertise here than most would imagine but it's ancient and esoteric.

We need a new word for the historical singularity.

TLDR: The "technological singularity" was an important and useful term with a clear meaning. Then it became the "Rapture of the Nerds". We need a new term.

--

I first heard the word "singularity" in the context of black hole physics; it dates back at least to the early 20th century:

ChatGPT 4 2023: "At the singularity, the laws of physics as we know them, including space and time, break down, and our current understanding of the universe is insufficient to predict what happens within it."

Not much later, in the 1950s, the term was applied by von Neumann in a technological context (from a 1993 Vernor Vinge essay):

Stan Ulam paraphrased John von Neumann as saying: "One conversation centered on the ever-accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue."

Brad Delong used to write about this kind of non-AI historical singularity. My favorite description of what it would be like to a approach at technological singularity was Vinge's short story "Fast Times at Fairmount High". (This prescient story appears to be lost to time; he wrote a similar full length novel but I think the short story was better).

The core idea is there's a (virtuous?) recursive loop where technology improves technology with shorter and shorter cycle times. Many processes go exponential and even near term developments become unpredictable. One may assume social end economic structures train to keep pace. The historical singularity exponential curve was part of The Economist's y2K Millennium issue GDP per person historical graph:

In a January 1983 essay for Omni Magazine Vinge focused on a particular aspect of the the technological singularity arising from superhuman intelligence (aka "super intelligence"):

We will soon create intelligences greater than our own ... When this happens there will be a technological and social transition similar in some sense to "the knotted space-time at the center of a black hole" 

A decade later, in his 1993 essay later published in Whole Earth Review (non-Olds cannot imagine what Whole Earth Review was like), Vinge revised what he meant by "soon":

... Based on this trend, I believe that the creation of greater-than-human intelligence will occur during the next thirty years. (Charles Platt has pointed out that AI enthusiasts have been making claims like this for thirty years. Just so I'm not guilty of a relative-time ambiguity, let me be more specific: I'll be surprised if this event occurs before 2005 or after 2030.) ...

So by the year 2000 we had the concept of a historical technological singularity (eminently sensible) that had become focused on a specific kind of self-improving technology - the superhuman intelligence with an upper-case S Singularity (presumably AI). Those were useful concepts - "technological singularity" and "superintelligence" Singularity. 

In 1993 Vinge predicted the Singularity would happen before 2030, later experts like Scott Aaronson predicted after 2080. (Aaronson has since revised that prediction and works for OpenAI; Vinge's 2030 dates looks pretty good.)

After 2000 though the word Singularity went off the semantic rails. It came to be used for for a peculiar future state in which human minds were uploaded into simulation environments that were usually described as pleasant rather than hellish. This is, of course, antithetical to the original idea of unpredictability! This peculiar meaning of "The Singularity" came to be known as "The Rapture of the Nerds" based on the title of a book by Charles Stross and Cory Doctorow. More recently that vision underlies a weird cult called longtermism that seems to have infected some vulnerable minds.

The "Rapture" meaning of "Singularity" has effectively taken over. We no longer have a term for the original von Neumann concept of self-reinforcing technological change that makes even the near term future unpredictable. That perversion of meaning is why I use the tag "Whitewater world" rather than Singularity for my own blog posts (and "Skynet" rather than super intelligence).

So we need a new term. I don't think "whitewater world" is it.

Anyone have a new term? We probably have a few years in which it might still be useful.

ChatGPT's 155 IQ and passing the Sebastian's children puzzle

A Scientific American contributor found ChatGPT (probably 3) scored 155 on a standard IQ test:

Estimated on the basis of five subtests, the Verbal IQ of the ChatGPT was 155, superior to 99.9 percent of the test takers who make up the American WAIS III standardization sample of 2,450 people. As the chatbot lacks the requisite eyes, ears and hands, it is not able to take WAIS’s nonverbal subtests. But the Verbal IQ and Full Scale IQ scales are highly correlated in the standardization sample, so ChatGPT appears to be very intelligent by any human standards.

We don't need to worry though because it couldn't answer this question:

ChatGPT easily fails at obvious riddles, such as “What is the first name of the father of Sebastian’s children?” (ChatGPT on March 21: I’m sorry, I cannot answer this question as I do not have enough context to identify which Sebastian you are referring to.) It seems that ChatGPT fails to reason logically and tries to rely on its vast database of “Sebastian” facts mentioned in online texts.

This is what happened when I tried the Sebastian test on GPT-4:

Ah, I understand now. This riddle is a classic play on words. The answer is "Sebastian" himself, as he is the father of his children.

Ok, so that's reassuring.

Update 4/12/2023: ChatGPT4 (C4) recently did well in some blinded advanced undergraduate exams.

A response to Scott Aaronson's rejection of an AI pause.

Scott Aaronson, who works on AI safety for OpenAI, wrote a critique of AI Pause that was not up to his usual standards. Here's what I wrote as a comment:

Hi Scott — I was confused by your post. I’m usually able to follow them. I won’t defend the letter directly and Yudkowsky/TIME is not worth a mention but maybe you could clarify some things…

1. 6m seems a reasonable compromise given the lifespan of humans, the timescales of human deliberation and the commercial and military pressure to accelerate AI development. Short enough to motivate urgent action, but long enough that reflection is possible. (I doubt we actually pause, but I agree with the principle. China isn’t going to pause of course.)

2. Let’s assume GPT 5 with an array of NLP powered extensions exceeds the reasoning abilities of 95% of humanity in a wide variety of knowledge domains. That’s a shock on the scale of developing fire, but it’s occurring in a hugely complex and interdependent world that seems always on the edge of self-destruction and actually has the capabilities to end itself. We’re not hunter gatherers playing with fire or Mesopotomians developing writing. There’s no precedent for the speed, impact and civilizational fragility we face now.

3. It’s not relevant that people who signed this letter were previously skeptical of the progress towards AI. I recall 10 years ago you were skeptical. For my part I’ve been worried for a long time, but assumed it was going to come in 2080 or so. 60 years early is a reason to pause and understand what has happened.

Lastly, I read the OpenAI statement. That seems consistent with a pause.

Everything is just fine.

The ad-funded internet is collapsing. It's been ailing for years but, as seems to the rule, the last bit is happening quite quickly. I'm guessing advertisers, who must be kind of dim, finally realized they were wasting their money. They did much better funding special interest magazines but those are gone now. Much of Silicon Valley has been funded by a now defunct business model. So maybe this is not a good time to be there. (Or, if you are young and have some money, maybe this is the best time to be there.)

Meanwhile human-cognition level AI has arrived. About 60 years earlier than I'd hoped. And some people thought "Don't Look Up" was about climate change. As a guide I recommend the movie Superintelligence; particularly the protagonist's approach to the end of the world. (Spoiler: It's a rom-com. Her world does not end.)

It's whitewater world all over now. On the bright side, we don't need to waste too much time on longterm planning. I retired in August of 2022 so I think I'll focus on mountain bike trail maintenance and, like Superintelligence's Carol Peters, on the needs of friends and family.

I ask ChatGPT about the next step.

 Not a bad response.

”Describe what happens when ChatGPT is enhanced with natural language processing and domain-specific knowledge modules.”

--

When ChatGPT is enhanced with natural language processing and domain-specific knowledge modules, it gains several capabilities that allow it to better understand and respond to user input.

Natural language processing (NLP) is a field of artificial intelligence that focuses on enabling computers to understand and interpret human language. By incorporating NLP into ChatGPT, the model can better analyze and interpret user input, including understanding sentence structure, recognizing named entities, and determining the meaning of words in context. This allows ChatGPT to generate more accurate and relevant responses to user queries.

Domain-specific knowledge modules provide ChatGPT with specialized knowledge related to a particular field or topic. For example, if ChatGPT is enhanced with a medical knowledge module, it can better understand medical terminology and provide more accurate and relevant responses to medical-related queries. By incorporating domain-specific knowledge modules, ChatGPT can tailor its responses to specific user needs, making it a more effective conversational agent.

Together, these enhancements enable ChatGPT to better understand and respond to user input, providing more accurate and relevant information to users. This can lead to a more engaging and satisfying user experience, as well as improved productivity and efficiency in a wide range of applications.”

Be afraid of ChatGPT

TL;DR: It's not that ChatGPT is miraculous, it's that cognitive science research suggests human cognition is also not miraculous.

"Those early airplanes were nothing compared to our pigeon-powered flight technology!"

https://chat.openai.com/chat - "Write a funny but profound sentence about what pigeons thought of early airplanes"

Relax	Be Afraid
ChatGPT is just a fancy autocomplete.	Much of human language generation may be a fancy autocomplete.
ChatGPT confabulates.	Humans with cognitive disabilities routinely confabulate and under enough stress most humans will confabulate.
ChatGPT can’t do arithmetic.	IF a monitoring system can detect a question involves arithmetic or mathematics it can invoke a math system*. UPDATE: 2 hours after writing this I read that this has been done.
ChatGPT’s knowledge base is faulty.	ChatGPT’s knowledge base is vastly larger than that of most humans and it will quickly improve.
ChatGPT doesn’t have explicit goals other than a design goal to emulate human interaction.	Other goals can be implemented.
We don’t know how to emulate the integration layer humans use to coordinate input from disparate neural networks and negotiate conflicts.	*I don't know the status of such an integration layer. It may already have been built. If not it may take years or decades -- but probably not many decades.
We can’t even get AI to drive a car, so we shouldn’t worry about this.	It’s likely that driving a car basically requires near-human cognitive abilities. The car test isn’t reassuring.
ChatGPT isn’t conscious.	Are you conscious? Tell me what consciousness is.
ChatGPT doesn’t have a soul.	Show me your soul.

Relax - I'm bad at predictions. In 1945 I would have said it was impossible, barring celestial intervention, for humanity to go 75 years without nuclear war.

XKCD 632

See also:

• All posts tagged as skynet
• Scott Aaronson and the case against strong AI (2008). At that time Aaronson felt a sentient AI was sometime after 2100. Fifteen years later (Jan 2023) Scott is working for OpenAI (ChatGPT). Emphases mine: "I’m now working at one of the world’s leading AI companies ... that company has already created GPT, an AI with a good fraction of the fantastical verbal abilities shown by M3GAN in the movie ... that AI will gain many of the remaining abilities in years rather than decades, and .. my job this year—supposedly!—is to think about how to prevent this sort of AI from wreaking havoc on the world."
• Imagining the Singularity - in 1965 (2009 post.  Mathematician I.J. Good warned of an "intelligence explosion" in 1965. "Irving John ("I.J."; "Jack") Good (9 December 1916 – 5 April 2009)[1][2] was a British statistician who worked as a cryptologist at Bletchley Park."
• The Thoughtful Slime Mold (2008). We don't fly like bird's fly.
• Fermi Paradox resolutions (2000)
• Against superhuman AI: in 2019 I felt reassured.
• Mass disability (2012) - what happens as more work is done best by non-humans. This post mentions Clark Goble, an app.net conservative I miss quite often. He died young.
• Phishing with the post-Turing avatar (2010). I was thinking 2050 but now 2025 is more likely.
• Rat brain flies plane (2004). I've often wondered what happened to that work.
• Cat brain simulator (2009). "I used to say that the day we had a computer roughly as smart as a hamster would be a good day to take the family on the holiday you've always dreamed of."
• Slouching towards Skynet (2007). Theories on the evolution of cognition often involve aspects of deception including detection and deceit.
• IEEE Singularity Issue (2008). Widespread mockery of the Singularity idea followed.
• Bill Joy - Why the Future Doesn't Need Us (2000). See also Wikipedia summary. I'd love to see him revisit this essay but, again, he was widely mocked.
• Google AI in 2030? (2007) A 2007 prediction by Peter Norvig that we'd have strong AI around 2030. That ... is looking possible.
• Google's IQ boost (2009) Not directly related to this topic but reassurance that I'm bad at prediction. Google went to shit after 2009.
• Skynet cometh (2009). Humor.
• Personal note - in 1979 or so John Hopfield excitedly described his work in neural networks to me. My memory is poor but I think we were outdoors at the Caltech campus. I have no recollection of why we were speaking, maybe I'd attended a talk of his. A few weeks later I incorporated his explanations into a Caltech class I taught to local high school students on Saturday mornings. Hopfield would be about 90 if he's still alive. If he's avoided dementia it would be interesting to ask him what he thinks.

Civilization, complexity and the limits of human cognition - another attempt at explaining the 21st century

The 70s were pretty weird, but I was too young to notice. (Not coincidentally, the Toffler/Farrell Future Shock book was written then.) By comparison the 80s and 90s more or less made sense. In 1992 Fukuyama wrote "The End of History" and that seemed about right for the times.

Things got weird again in the late 90s. I was in a .com startup and I remember valuations getting crazy about 1997, 3 years before the .com crash. We were still picking ourselves up from the crash when 9/11 hit. (A year later, on a purely personal note, my youngest brother vanished.) In the early 00s came Enron and other frauds almost forgotten now. Then in 2008 the real estate collapse and the Great Recession. We were barely recovering from the Great Recession when Trumpism hit. Followed by COVID (which was expected and not at all weird) and the Great Stupidity of the American Unvaccinated (which we did not expect and is perhaps weirdest of all).

Each time the world went off kilter I have tried to figure out a root cause:

In Feb 2009 I tried to figure out the crash of 2008:

At last count my list of contributing factors to the crash of '09 included ...

1. Complexity collapse: we don't understand our emergent creation, we optimized for performance without adaptive reserve
2. Mass disability and income skew: The modern world has disenfranchised much of humanity
3. The Marketarian religion: The GOP in particular (now the Party of Limbaugh), but also many Democrats and libertarians, ascribed magical and benign powers to a system for finding local minima (aka The Market). The Market, like Nature, isn't bad -- but neither is it wise or kind.
4. The occult inflation of shrinking quality: What happens when buyers can't figure out what's worth buying. Aka, the toaster crisis - yes, really.
5. performance-based executive compensation and novel, unregulated, financial instruments: a lethal combination. See also - You get what you pay for. The tragedy of the incentive plan.
6. Disintermediating Wall Street: Wall Street became a fragile breakpoint 
7. The future of the publicly traded company: A part of our problem is that the publicly traded company needs to evolve
8. The role of the deadbeats: too much debt - but we know that
9. Firewalls and separation of powers: a culture of corruption, approved by the American electorate, facilitated dissolving regulatory firewalls
10. Marked!: Rapid change and the Bush culture made fraud easy and appealing

I put Marked! pretty low on the list, but maybe I should bump it up a bit. The Hall of Shame (Clusterstock) lists a lot more fraud than has made the papers [1]...

• Fairfield Greenwich Group: quietly complicit in the Madoff fraud?
• Angelo Mozilo: white collar mobster
• Sam Israel: a $400 million fraud. In the old days, he'd have been front page news. I never heard of him.
• Many more, each of which would have been in the news once - including: JS Forte, RS Piccoli, JG Ossie, D Palmer, A Nodel, Marc Dreier.
• Those swindlers who have made the news... Allen Stanford, and, of course, Mr. Madoff.

By 2010 I was focusing on RCIIIT: The rise of China and India and the effects of IT.

... The Rise of China and India (RCI) has been like strapping a jet engine with a buggy throttle onto a dune buggy. We can go real fast, but we can also get airborne – without wings. Think about the disruption of German unification – and multiply than ten thousand times.

RCI would probably have caused a Great Recession even without any technological transformations.

Except we have had technological transformation – and it’s far from over. I don’t think we can understand what IT has done to our world – we’re too embedded in the change and too much of it is invisible. When the cost of transportation fell dramatically we could see the railroad tracks. When the cost of information generation and communication fell by a thousandfold it was invisible ...

In 2016 and again in 2018 I tried to explain Trumpism by contributing factors (I was too optimistic about Murdoch's health though):

• 65% the collapse of the white non-college “working class” — as best measured by fentanyl deaths and non-college household income over the past 40 years. Driven by globalization and IT both separately and synergistically including remonopolization (megacorp). This is going to get worse.
• 15% the way peculiarities of the American constitution empower rural states and rural regions that are most impacted by the collapse of the white working class due to demographics and out-migration of the educated. This is why the crisis is worse here than in Canada. This will continue.
• 15% the long fall of patriarchy. This will continue for a time, but eventually it hits the ground. Another 20 years for the US?
• 5% Rupert Murdoch. Seriously. In the US Fox and the WSJ, but also his media in Australia and the UK. When historians make their list of villains of the 21st century he’ll be on there. He’s broken and dying now, but he’s still scary enough that his name is rarely mentioned by anyone of consequence.
• 1% Facebook, social media, Putin and the like. This will get better.

That 1% for Facebook et all is pretty small — but the election of 2016 was on the knife’s edge. That 1% was historically important.

A few months ago I listed 3 causes for the post-COVID supply and labor shock economics of 2021:

1. Wealth became extremely concentrated. 

2. Returns on labor for 40% of Americans fell below modern standard for economic life.

3. Good investments became hard to find.

It's almost 2022 now, so we're into almost 25 years of the world not making sense any more. So now I'm digging even deeper for a root cause.

Today I'm going with Gordon's Law: the complexity of a complex adaptive system will increase until it reaches a limiting factor. Our civilization is a complex adaptive system and its complexity increased until it hit a limiting factor -- the complexity capacity of the average human. These days between 40 and 50% of American's can't handle civilization 2021 (sometimes I call this mass disability (see also). Witness among other things, The Great Stupidity of the FoxCovians.

It's a variant of the "Future Shock" Toffler wrote about 52 years ago. I don't have a fix; I don't think the world will get less complex. Our technologies are moving too fast. Maybe we'll just get used to not understanding the world and civilization will stumble on regardless. After all, for most of human history the world was incomprehensible -- and we did manage. Sort of. Mostly without civilization though ...

Electric vehicles will end another non-college job - the auto mechanic

 My Great-Aunt was born in the 19th century. She spent most of her life working in data processing. She, and thousands like her, did read, delete, update operations on paper cards that were passed between thousands of small rectangular desks in a large rectangular building in Montreal. None of her coworkers had a college degree -- I suspect many could not read very well. The work seems impossibly dull, but she enjoyed it and the pension it brought her.

I have one of those desks, I'm typing on it now. It fits nicely in a corner of my living room, and I'm slender enough to fit comfortably in it.

By the 1960s the first business computers wiped out her industry as definitively as the automobile eliminated millions of horses. There would never again be a large scale job that required no particular social, physical, or cognitive skills.

Since that time IT has generated vast numbers of knowledge worker jobs that pay relatively well while eliminating jobs that do not require cognitive skills.

Now electric vehicles are going to do the same thing. Compared to internal combustion engines they are much easier to maintain; their complexity is in batteries and software. Never-college auto mechanics are going to lose their jobs.

There's a lot we can do about this problem. It's not only the right thing to do, it's also essential to our survival. Even if Biden wins in 2020, if his administration doesn't act quickly there will be another Trump in 2024.

Viktor Frankl - on expectations and the behavior of people

Viktor Frankl formed some of his opinions of human nature by surviving several concentration camps. After his release, in a few days, he wrote a book about his experience - Man's Search for Meaning.

The book has harsh critics. I read it and I think much of sees truth, though it also a book of another era -- an era in which "man" more or less included women. 

Today psychology, psychiatry, neurology and the sciences retain little of Frankl's life work. He could not grasp that meaning might exist in the absence of religion, or that responsibility could be assumed rather than fundamental. I believe, however, that he had a true understanding of the extremes of human nature for evil and for good.

YouTube (and the Ted site) have a video of a lecture he gave later in his life. From the Frankl Institute (with let another video copy!):

YOUTH IN SEARCH OF MEANING, 1972 [4:22]

Frankl speaking at the "Toronto Youth Corps" in 1972. See Frankl "at his best" as he vividly explains his theories, and even draws analogies to piloting an aircraft – a passion he had recently picked up.

In this lecture he talks about how one must "crab" an airplane to adjust for a crosswind (1:45).  To reach a destination you have to periodically turn into the wind. He expands the analogy to people:

If we take man as he really is we make him worse. But if we overestimate him ... if we seem to be idealist and are overestimating ... overrating man ... and looking at him up high ... we promote him to what he really can be...

... Do you know who has said this? If we take man as he is we make him worse, but if we take man as he should be we make him capable of becoming what he can be? ... This was not me. This was not my flight instructor. This was Goethe.

From this it is a small Google step to the Goethe quote (in English):

When we treat man as he is we make him worse than he is.

When we treat him as if he already was what he potentially could be we make him what he should be.

 In the strange time of June 2020 I think this is worth remembering.

Why the crisis of 2016 will continue for decades to come

I haven’t written recently about why Crisis 2016, sometimes called Crisis-T, happened. For that matter, why Brexit. My last takes were in 2016 …

• In defense of Donald Trump - July 2016. In which I identified the cause of the crisis, but assumed we’d dodge the bullet and HRC would tend to the crisis of the white working class.
• Trumpism: a transition function to the world of mass disability - Aug 2016. “How does a culture transition from memes of independence and southern Christian-capitalist marketarianism to a world where government deeply biases the economy towards low-education employment?"
• After Trump: reflections on mass disability in a sleepless night - Nov 11, 2016. "Extreme cultural transformation. Demographics. China. The AI era and mass disability. I haven’t even mentioned that pre-AI technologies wiped out traditional media and enabled the growth of Facebook-fueled mass deception alt-media … We should not be surprised that the wheels have come off the train.”
• Crisis-T: What’s special about rural? - Nov 16, 2016: "The globalization and automation that disabled 40% of working age Americans isn’t unique to rural areas, but those areas have been ailing for a long time. They’ve been impacted by automation ever since the railroad killed the Erie canal, and the harvester eliminated most farm workers. Once we thought the Internet would provide a lifeline to rural communities, but instead it made Dakka as close as Escanaba.”

How has my thinking changed two years later? Now I’d add a couple of tweaks, especially the way quirks of America’s constitution amplified the crisis. Today’s breakdown:

• 65% the collapse of the white non-college “working class” — as best measured by fentanyl deaths and non-college household income over the past 40 years. Driven by globalization and IT both separately and synergistically including remonopolization (megacorp). This is going to get worse.
• 15% the way peculiarities of the American constitution empower rural states and rural regions that are most impacted by the collapse of the white working class due to demographics and out-migration of the educated. This is why the crisis is worse here than in Canada. This will continue.
• 15% the long fall of patriarchy. This will continue for a time, but eventually it hits the ground. Another 20 years for the US?
• 5% Rupert Murdoch. Seriously. In the US Fox and the WSJ, but also his media in Australia and the UK. When historians make their list of villains of the 21st century he’ll be on there. He’s broken and dying now, but he’s still scary enough that his name is rarely mentioned by anyone of consequence.
• 1% Facebook, social media, Putin and the like. This will get better.

That 1% for Facebook et all is pretty small — but the election of 2016 was on the knife’s edge. That 1% was historically important.

Rupert Murdoch will finally die, though his malignant empire will grind on for a time. Patriarchy can’t fall forever, eventually that process is done. We now understand the risks of Facebook and its like and those will be managed. So there’s hope.

But the crisis of the white non-college will continue and our constitution will continue to amplify that bloc’s political power in rural areas. Even if civilization wins in 2020 the crisis of 2016 will continue. It will test human societies for decades to come.