Crosspost from my substack: AI for Cultural Enlightenment - by Jonah Wilberg
I was recently invited to give a talk at the London Initiative for Safe AI with the following brief:
The core question for the evening is: “If AI safety achieves one thing before 2027, what should it be?” The second half of 2026 feels like a particularly consequential moment for people working in this space. Given the pace of technological and political change, we want the format to encourage speakers to respond boldly, candidly, and concretely about what matters most now. What interventions must the field focus on (and what should be dropped); what does the field need to secure the best outcomes for humanity?
Here is a lightly edited transcript of my talk on ‘AI for Cultural Enlightenment’, along with the slides I used.
I’m going to be talking about what I call ‘AI for Cultural Enlightenment’.
Don’t worry - I’m not proposing that we try to achieve Cultural Enlightenment by 2027.
But I do think AI for Cultural Enlightenment deserves to be a high priority cause area this year.
And in this talk I will briefly sketch out how this approach differs from and builds on existing work on ‘AI for epistemics’.
Let me start with a brief summary of AI for epistemics - and here I’m particularly drawing on recent work by Forethought, in particular their design sketches for a more sensible world.
AI for Epistemics is basically AI that improves our ability to figure out what’s true.
And there are basically four kinds of AI tools that could do this in ways that support AI safety.
-
Tools that support individual decision-making - so-called angels on the shoulder that draw on the individual’s context - are potentially useful in addressing all the main kinds of AI risks that I’ve listed on the slide, which depend on the quality of decisions by powerful people.
-
Tools for collective epistemics, like reliability tracking and provenance tracing, aim to improve the general quality of political debate, which seems clearly useful to address AI risk.
-
Likewise tools for strategic awareness, such as superforecasting and scenario planning tools, also help ensure people have the right information for managing risks from AI.
-
Finally coordination tech like automated negotiation, monitoring and verification systems, would help address coordination failures like race dynamics that are central to AI governance challenges.
So what’s the problem here? Well what I want to focus on here is the major adoption challenges that these tools face.
It’s not enough to simply design good tools, there’s also the challenge of getting people to actually use them.
So what makes this so challenging?
It’s partly a problem of circularity, which has been discussed in the writing from Forethought: in order to tell that a tool will improve your epistemics, you first need epistemic norms that are good enough to allow you to see that.
But I suggest we also need to think beyond epistemics in a narrow sense to the social and psychological factors that prevent adoption of such tools.
On the social side, there’s the fact that people often need social proof, evidence that other people are doing something, before they do it themselves. And this leads to so-called adoption thresholds, and tipping points where a critical mass has to be reached before adoption can become widespread.
And on the psychological side, things like humility, psychological safety and tolerance for uncertainty are really important in being willing to take advice from a good AI tool - and there have actually been some empirical studies of this.
So what I’m proposing is that in response to these challenges we need to shift from AI for epistemics to AI for cultural enlightenment.
-
The core idea is to go beyond epistemics in a narrow sense to looking at the social and psychological conditions for adopting good epistemic norms.
-
And instead of the standard design and marketing approach, we go deeper into the social norms that can support or block adoption of these (and this is arguably one of the things Google failed to do with Google Glass for example).
-
AI for epistemics sometimes focuses on the elite or powerful decision-makers, whereas the shift I’m proposing puts more of a focus on changing widespread cultural norms, because of that awareness of social norms and tipping points.
-
And all these points mean that we need to broaden the research base to include disciplines like sociology, political science, psychology and history - and with history I’m particularly thinking of the study of previous enlightenments, and the role of technologies like the printing press, and the spread of social norms around scientific methods, in those cultural transitions.
Using the idea of cultural enlightenment in this context is not new - it’s similar to terms used by Yudkowski (raising the sanity waterline), Macaskill (second Enlightenment) and Pollock (Second Renaissance).
But what I’m suggesting is relatively new is the idea of ‘AI for cultural enlightenment’ as a coherent theory of change in which cultural enlightenment and AI tools form a self-reinforcing feedback loop that supports AI safety and existential security more broadly.