At this Friday’s research call I’ll share some work-in-progress ideas on how the complexity science paradigm for the social sciences helps us understand societal crisis and transformation.
Register and join here:
2 Likes
I’m on Dave Snowden’s mailing list (from a 2024 workshop), and the attached document just now came to my inbox. It seemed pretty relevant to tomorrow’s research topic!
Where Machines Work Best and Why Humans Remain Indispensable_.pdf (1.1 MB)
1 Like
Is there any recording available?
Here is @JonahW ‘s Substack article on the same theme. It’s generally similar to what was presented in the talk.
will make the recording available in the coming weeks yes!
A few questions to keep in mind about the application of mathematical complexity to human affairs:
-
how many variables? Henri Poincaré was a transitional figure in the move from linear systems to non-linear complexity, demonstrating how straight-forward calculations become intractable at the level of even three astronomical bodies in gravitational relationship. If just three bodies, modeled only through relatively simple terms like mass and momentum, are subject to computational uncertainly due to barely detectable variances around empirical observations, how much more so must any human dynamical system be subject what Taleb calls Black Swans? The more variables involved, the more such computational models should really hang in the Musée d’Orsay alongside the works of all the other impressionists!
-
With this sort of thing in mind, Turchin and his team cut cliodynamics as close to the bone as possible. The models they present involve about 3-4 variables at most, and they fully understand these patterns are at best suggestive about how the matters they treat may play out pragmatically in any given situation or cycle. Which of the many available variables to model is quite a central question.
-
The question of scale is also extremely pertinent. This includes both geographic and temporal scales. There is much overheated rhetoric lately about the impending doom of “civilization” or “humanity” or the “planet”. That sort of hyperbole is de rigueur for superhero screenplay authoring. But those who wish to be constructive in the face of the “metacrisis” (or pick your own preferred impending apocalypse) will want to be quite a bit more targeted and analytical before settling on any of the thousands of available action plans.
1 Like
Folding in a WhatsApp post by @michaelgarfield
“David Wolpert has somebody to say about this
Wolpert et al are dense enough that I wanted to LLM it to see if closer inspection is worth it. I’d say from the full Gemini results below, worth it. A deeper dive into the maths will take some effort, however.
Prompt: Compare the model in the open tab to Turchin’s cliodynamics.
Gemini said
The research in the open tab presents a framework that is both a direct extension of and a methodological refinement to Peter Turchin’s Cliodynamics. While both aim to transform history into a predictive, quantitative science, they differ in their mathematical treatment of “chance” and “structure.”
Key Similarities
-
Scientific History: Both projects share the goal of “Cliometrics 2.0”—moving beyond qualitative narratives to treat historical change as a dynamical system.
-
State-Space Modeling: Like Turchin’s work in Historical Dynamics, this paper uses a state-space approach, tracking macro-variables like population, territorial extent, and institutional complexity (Scale and Computation) to map the “rise and fall” of civilizations.
-
Data Source: The authors explicitly use the Seshat: Global History Databank, which was co-founded by Turchin, as a primary dataset for their Application 2.
Core Differences in Approach
| Feature | Turchin’s Cliodynamics | This Paper’s SDE Framework |
|---|---|---|
| Primary Lens | Deterministic Cycles: Focuses on structural-demographic cycles and secular trends (e.g., the “double-helix” of integration and collapse). | Stochastic Processes: Views history as a continuous-time “random walk” influenced by both systematic drift and irreducible noise. |
| Mathematical Tool | Differential Equations: Often uses coupled ODEs (like predator-prey models) to find equilibrium points and periodic oscillations. | Stochastic Differential Equations (SDEs): Uses Langevin dynamics to decompose history into Drift (trend) and Diffusion (uncertainty). |
| Role of Chance | Often treated as a “residual” or external shock to a deterministic system. | Constitutive: Randomness is a core feature; the model quantifies how much a path is “typical” vs. “exceptional.” |
| Key Metric | Cycle Phase: Determining where a society sits in a predictable 200–300 year cycle. | Irreversibility: Quantifying the “Arrow of History” to see if a change (like a bureaucratic shift) can be easily undone. |
Methodological Advancement
The paper’s authors (including David Wolpert) argue that Turchin’s original methods often rely on “deterministic snapshots” that overlook the inherently uncertain nature of historical trajectories.
By using Langevin Bayesian Networks and nonparametric Gaussian processes, they introduce two capabilities that go beyond standard Cliodynamics:
-
Exogenous Perturbation Detection: A formal way to identify “Black Swan” events (like the 9/11 attacks or the 1968 movements) by measuring their statistical “surprisal” relative to the system’s internal noise.
-
Probabilistic Imputation: A more rigorous way to fill in the “missing years” in fragmentary historical records by simulating potential paths between known data points, rather than just using linear interpolation.
Good point about variables. I’ve heard David Krakauer make similar points in terms of ‘dimensions’. Mainstream social science often ends up being one-dimensional, e/g. with economic analyses revolving around GDP, inflation, prices etc, one at a time. Complexity science aims to provide a multi-dimensional view of the same phenomena - but without adding so many dimensions that it becomes opaque and intractable. There’s a sweet spot determined partly by human cognitive capacities - a goldilocks zone of optimum understandability between the inaccuracy of one dimension and the opacity of a thousand.
Not to get fixated just on Peter Turchin, here is another example. In Why the West Rules - for Now, Ian Morris set out essentially to determine if geography is destiny. That question can’t really be answered with snapshots of things like European vs Chinese GPD over spans of mere decades. Morris therefore took on the entire historical record, not to mention prehistory. That involved quite a bit of selectivity, of course. Morris, however, documented every assumption and every modeling decision about which data to use and which not to use. (There is an entire companion volume just on methods - The Measure of Civilization.)
To get a rough and ready East vs West metric, Morris constructed an index of social development involving four dimensions: Energy Capture Per Capita, Urbanization, Information Technology, War-Making Capacity. There were many other candidate variables. But Morris shows in some detail how other metrics really don’t have much independent impact on the overall scale, which is an admittedly crude yardstick for comparing civilizational development across wide temporal and geographic spans.
The most interesting takeaway from all this was that although the West led the East (including Egypt and Mesopotamia as “West”) for most of history, China around the Song dynasty had a superior run for several centuries, outperforming both imperial Rome and the prior Chinese Han. Europe did not pull fully level until well into the early modern era. Clearly, no one should be resting on their laurels just now!
Another takeaway is that although Morris’s four dimensions make independent contributions in various historical circumstances, if we had to pick one metric to rule them all, it’s not any sort of close decision. Energy capture by a country mile! Which totally supports any number of energy-centric theorists currently focusing on complexity as anti-entropic energy capture.
Morris’s motivational theory is similarly parsimonious: greed, fear, sloth (in no particular order). We don’t really need terribly sophisticated psychological models to understand why peoples the world over attacked their neighbors, enslaved them, and put them to them to work in the fields, and grew corpulent consuming the resulting surplus.
In reading the Bae et al. paper now, I realized I need to devise a post-graduate course for myself to even understand what the paper is talking about. I then used Gemini to suggest the curriculum (that’s below). If anyone wishes to join that tutorial with me, feel free to engage here or through DM channels!
Prompt: Create a set of learning objectives for a graduate course on quantitative methods in historiography based on the paper in the tab.
Gemini said
Based on the framework presented in the paper “Unveiling hidden features of social evolution by inferring Langevin dynamics from data”, here are the proposed learning objectives for a graduate-level course on quantitative methods in historiography.
These objectives shift the focus from traditional linear regressions toward stochastic dynamical modeling and complexity science.
Course Learning Objectives
1. Theoretical Foundations of Stochastic History
-
Contrast traditional deterministic and static “snapshot” methodologies in cliometrics with continuous-time stochastic processes.
-
Evaluate the philosophical implications of treating history as a distribution of counterfactual trajectories rather than a single realized path.
-
Differentiate between aleatoric uncertainty (intrinsic randomness) and epistemic uncertainty (fragmentary records/measurement error) within historical datasets.
2. Modeling Social Evolution via SDEs
-
Construct a latent state space from heterogeneous historical observations (e.g., mapping categorical Seshat data into continuous variables).
-
Formulate and interpret Stochastic Differential Equations (SDEs), specifically identifying the role of the drift term (structural trends) and the diffusion term (stochastic fluctuations).
-
Apply the Euler-Maruyama approximation to bridge the gap between continuous-time theory and discrete, irregularly sampled historical records.
3. Advanced Diagnostic Metrics
-
Quantify the “Arrow of History” using measures of time irreversibility based on likelihood ratios of forward and time-reversed trajectories.
-
Implement Exogenous Perturbation Detection to statistically distinguish “Normal” internal volatility from “Great Levelers” (e.g., wars, pandemics, or revolutions) using information-theoretic surprisal.
-
Execute Probabilistic Imputation to reconstruct missing historical data points by sampling from bridge distributions governed by learned dynamics.
4. Computational Inference Methodologies
-
Compare the utility of Langevin Bayesian Networks (LBN) for data-rich regimes (like modern political economy) versus nonparametric Gaussian process SDEs (npSDE) for sparse, ancient historical data.
-
Validate the Markovian assumption in social dynamics by analyzing the temporal correlation of standardized residuals (white noise testing).
5. Critical Synthesis and Application
-
Analyze case studies such as the “institutional takeoff” of Latium or the “administrative capacity” of the Paris Basin to identify where structural constraints end and contingency begins.
-
Critically assess the limitations of SDE modeling, including the risks of homogeneity assumptions across diverse geographic regions and the challenges of causal interpretability.
Would you like me to draft a sample syllabus or a weekly reading list based on these objectives?
1 Like
For all the fancy maths in Bae et al. (which I do intend to pursue, because what’s not to love about fancy maths?), the summary below is pretty consistent with Turchin’s cruder (but effective) conclusions from the same underlying data set.
”This implies that while high inequality may not trigger a rapid deterministic collapse in wealthy societies, it leads to a loss of structural resilience. In this regime, the restoring forces that typically sustain democracy effectively vanish, leaving the system structurally unanchored and weakly but consistently biased toward the low-democratic region.”
For those wishing to check out the cool graphs in the midst of Bae et al., but wanting plainer language about what all the numbers mean, here is a quick and dirty translation:
- State space - everything that ever might have happened
- Scale - additional territory under a consistent social system
- Computation - higher level complexity in government, bureaucracy, information systems, etc.
- Forward translation log density - the quality of being unexpected or Black Swan-ish.
- Irreversibly - alignment with long term structural trends.
here is the recording:
Love the second Substack installment on power laws, fat tails, etc. Running very parallel to my current reading of all things Nassim Taleb!
1 Like
Great, here is the link in case that’s useful for others
In a recent Discord thread @dvdjsph , my grad students and I got into theoretical weeds about fat tails, etc. in the interests of prototyping David’s Promise Protocol. To give a flavor, I’ll repost here a couple sections posted by me in the context of much back-and-forth discussion.
”This from a technical footnote in: Nassim Nicholas Taleb. (2022). The Black Swan : the impact of the highly improbable. Random House Trade Paperbacks. (p. 353) "With probability simply, a metaprobability assigns degrees of credence to every probability. … " The full note is long, but all this is from a technical paper referencing, among others the work of Russell and Tarski on why sentences cannot contain their own truth predicates. (“This sentence is true” results in paradoxes). So every language needs a metalanguage. From that, Taleb derives that every probability (i. e. Credence) needs its own probability (i. e. Confidence). Bottom line - all of this is deep in the weeds white paper fodder allowing us to frame PP in very sophisticated terms.”
”Your final paragraph is Taleb in a nutshell. I came to a similar conclusion last night - for risk analysis and/or decision support, there has to be a payoff term. Most of Taleb’s energy goes into what you call k. He criticizes what he calls the “ludic fallacy”, which is applying Gaussian statistical analysis to p, assuming we really understand the rules of the game (as if life were as tidy as a casino). The Black Swan question is “what really is the game”? Long Term Capital Management, Lehman Brothers, etc. were optimizing for what they thought the game was. Taleb was hedging that the “game” was beyond comprehension and subject to adverse outcomes for unexpected reasons. So there is the probability. And there is also the probability about how right you are about the probability. And finally, the payoff (or penalty) attached to being wrong about either”
All that clearly relates, @JonahW to your latest. For any question that matters about social analysis, given some variable, is the distribution Gaussian (bell curve) or Power Law? In Taleb’s quantitative technical work, he and coauthors show very interestingly that the Power Law can derive from a Gaussian of a Gaussian. Any given data set can fit any number of bell curves! Are we all clustered around some regular center? Or we really off more in some chunking fat tail? Taleb’s point - supported rather elegantly by Promise Protocol, is for any given probability analysis, we also ought to be factoring in the probability that we may be wrong about the probability space we think we are in!
1 Like
Very interesting, I might have look more closely at Taleb. Can you remind me what promise protocol is in a nutshell? Curious as to how these questions of probability fit in with that.
Probabilities of probabilities reminds me of what I called ‘model uncertainty’ in this piece. Wise Altruism: Doing Good under Radical Uncertainty
Here’s PP in a nutshell - PROMISE Platform
Credence and Confidence are both their own Gaussians, so the mapping to Taleb’s analysis (which allows us to model fat tails and Black Swans) is rather uncanny.
In a nutshell, Promise Protocol is a trust-building protocol that gives us a way to answer the question how likely is a given promise to be kept, as a general way of operationalizing trust.
It has two sides, according to the two ways of answering this question: before committing trust (forecasting) and after committing trust (record-keeping).
The pre-commit process is where the topic of meta-probability enters the picture. Promise Protocol’s pre-commit/forecasting process (Abductio) can be thought of as a black box that takes propositions and assigns a credence and confidence (analogous to probability/meta-probability respectively). While confidence is insufficient, the process continues recursively. (The process keeps track of the cost of evaluation, each operation costing a pre-defined amount of ‘credits’, so it will always terminate.). The explanation on the page @RobertBunge linked to contains an interactive illustration of this process applied to investigating a marketing claim; the following images show two different steps of an Abductio process.
At a deeper level, Abductio is meant to be a more precise way of defining Charles Saunders Peirce’s Pragmatic Maxim, i.e.
Consider what effects, that might conceivably have practical bearings, we conceive the object of our conception to have. Then, our conception of these effects is the whole of our conception of the object.
Applying this to truth values, it can be thought of like this: something is as true as all of its implications are true.
1 Like