THUNK – 150. Mutual & Common Knowledge

Mutual & common knowledge: because simulating someone else simulating you is confusing…perhaps usefully so….

-Links for the Curious-

Common Knowledge & Aumann’s Agreement Theorem, by Scott Aaronson –

Common Knowledge (Vanderschraaf & Sillari, 2014) –

The Psychology of Common Knowledge & Coordination (Thomas et al, 2014) –

Bystander “Apathy” (Latane & Darley, 1969) –

The logic of indirect speech (Pinker et al, 2008) –


THUNK – 149. Solution Space & Tiling Trees

Challenge: Examine enough potential solutions to a problem to be confident in one. Answer: Tiling trees!

-Links for the Curious-

Rationally Speaking, Episode 212: Ed Boyden –


“Get Lucky,” by Bob Holmes –

Text available here –

The Philosophy of Problem Space and Solution Space, by Karen Bachmann –

Binary Search Tree (Wikipedia) –


THUNK – 148. Aristotle’s Infinite Regress & Intuition

Aristotle found a problem at the foundations of epistemology: an infinite regress of justification.

-Links for the Curious-

Reason & Intuition (Parsons, 1998) –

The Stanford Encyclopedia of Philosophy
Intuition –
Aristotle’s Logic –
Foundationalist Theories of Epistemic Justification –
Coherentist Theories of Epistemic Justification –

“Regress Argument,” on Wikipedia –


THUNK – 147. Grice’s Maxims of Conversation

How exactly does implication work? Philosopher Paul Grice might have had a couple decent ideas.

-Links for the Curious-

Logic & Conversation, lecture by Paul Grice –

“What we say vs what we mean: what is conversational implicature?” by Maria Kasmirli –

Paul Grice, by the Stanford Encyclopedia of Philosophy –

The Logical Syntax of Language, by Rudolf Carnap –

THUNK – 146. Mental Illness & Creativity

Everyone knows you have to be a little bit crazy to be a great artist, right? …right?

-Links for the Curious-

Mental illness, suicide and creativity: 40-Year prospective total population study (Kyaga et al, 2013) –

Mental Illness Statistics from the NIH –

“The Madness of Sir Isaac Newton,” by Marina Jones –

“11 Historical Geniuses and Their Possible Mental Disorders,” by Kathy Benjamin –

Wikipedia entry on creativity & mental illness –

“Creativity and Mental Illness,” by Arline Kaplan –

“What Neuroscience Has To Say About The ‘Tortured Genius’,” by Sarah Klein –

THUNK – 145. Tit for Tat & Zero Determinant Strategies

Tit for Tat: the king of Prisoner’s Dilemma strategies. Its simplicity masks its depth…but it *can* be beaten!

-Links for the Curious-

A fantastic interview with Press about his findings –

An overview of the various algorithms pitted against each other in Axelrod & Press/Dyson’s battle royale –

A great summary for PNAS of the ZD strategies & their implications –

A summary of ZD strategy extortion –

Forbes’s (now dated) piece on the Generous Tit for Tat strategy –

THUNK – 144. Thinking Like a Bayesian

Prediction is a lot like playing poker…so why do we treat it like it was a coin toss? Enter the Bayesians!

-Links for the Curious-

Bayes’ Theorem Illustrated (My Way), by komponisto –

Conditional probability explained visually (Bayes’ Theorem), by Khan Academy –

Critical Thinking Fundamentals – Bayes’s Theorem, by Wireless Philosophy –

A visual guide to Bayesian thinking, by Julia Galef –

Newcomb’s Problem: it’s like “Is a hotdog a sandwich?,” but WAY nerdier, with deep implications for rationality.

NOTE: Thanks also to Rob Dufau for his help with this week’s episode!

-Links for the Curious-

Newcomb’s Problem And Two Principles Of Choice (Nozick, 1969) –

“Causal Decision Theory,” Stanford Encyclopedia of Philosophy, 2016 –

“Causation in Decision Theory,” by Jim Joyce –

Some Counterexamples to Causal Decision Theory (Egan, 2006) –

“Newcomb’s problem, expected utility, and dominance,” by Jeff Speaks –

Newcomb’s Problem & the tragedy of rationality, by Julia Galef –

“Decision Theory FAQ,” by lukeprog & crazy88 via LessWrong –

Prisoner’s Dilemma is a Newcomb Problem (Lewis, 1979) –

THUNK – 142. “Rogue AI’s” of the Present Day

Many people are worried about rogue AI’s of the future, but what about the ones that already exist?

-Links for the Curious-

“Dude, you broke the future!” by Charlie Stross –

“Artificial Personhood is the Root Cause Why A.I. is Dangerous to Society,” by Carlos E. Perez –

“AI Has Already Taken Over. It’s Called the Corporation,” by Jeremy Lent –

“A Mod for Civilization Makes Rogue AI More Dangerous Than Gandhi’s Nukes,” by Rhett Jones –

Predicting Depression via Social Media (De Choudhury et al, 2013) –

Online Social Networking and Addiction—A Review of the Psychological Literature (Kuss & Griffiths, 2011) –

Algorithmic Entities (LoPucki, 2017) –

“Elon Musk’s Freak-Out Over Killer Robots Distracts From Our Real AI Problems,” by Tom Simonite –

THUNK – 141. Von Neumann Probes

-Links for the Curious-

“Theory of Self-Reproducing Automata,” by John Von Neumann, 1966 –

:Advanced Automation for Space Missions,” edited by Robert A. Freitas, Jr. & William P. Gilbreath, 1980 –

Self-Replicating Robots and Galactic Domination | Space Time | PBS Digital Studios –

Additive Construction using Basalt Regolith Fines (Mueller et al, 2015) –

Michio Kaku: The von Neumann Probe (A Nano Ship to the Stars) –

“Desktop 3D printer presages the future of multi-layer circuit board design,” by Lucas Mearian –


THUNK – 138. Misinformation

What’s the harm in a little white lie? Well, if you tell enough of them, you may be able to steer an electorate.

-Links for the Curious-


“Cognitive Ability and Vulnerability to Fake News” by David Z. Hambrick, Madeline Marquardt –

‘Fake news’: Incorrect, but hard to correct. The role of cognitive ability on the impact of false information on social impressions (Keersmaecker & Roets, 2017) –

Knowing the Truth is not enough: The Resilience of Discredited Information –

Beliefs Don’t Always Persevere: How political figures are punished when positive information about them is discredited (Cobb et al, 2008) –

Frequency and the Conference of Referential Validity (Hasher & Goldstein, 1977) –

“Influence: The Psychology of Persuasion“ by Robert Cialdini –

“Did Russia Affect The 2016 Election? It’s Now Undeniable” by Molly McKew –


THUNK – 137. Embodied Cognition


Is a brain all you need to have a functional mind? Is the body just peripheral to cognition? Many researchers think so, but advocates of embodied cognition have different ideas.

(Quick note: “Altered Carbon” is racy & gory, but also very good. It doesn’t get particularly hung up on the “Is a copy of my mind REALLY MEEEE?!” question, touches on a lot of interesting problems with technological immortality, & actually has a lot of interesting test cases & scenarios for the concept of embodied cognition.)

-Links for the Curious-

The Metaphorical Structure of the Human Conceptual System (Lakoff & Johnson, 1980) –

“A Brief Guide to Embodied Cognition: Why You Are Not Your Brain” by Samuel McNerny –

“Embodied Cognition,” the Stanford Encyclopedia of Philosophy –

“Embodied cognition is not what you think it is” by Andrew D. Wilson* and Sabrina Golonka –

The Poverty of Embodied Cognition (Goldinger et al, 2016) –

“You Think with the World, Not Just Your Brain” by Sam Kriss, a summary of a related phenomenon called “extended cognition” –

A Cognitive Study of Happiness Metaphors in Persian and English (Safarnejad et al, 2014) –

Why You May Want to Stand Like a Superhero”, by Robin S. Rosenberg –

THUNK – 136. Engineering Wisdom: Tolerance & Failing Gracefully

Engineering is a highly technical field, but it makes use of exceedingly useful & straightforward ideas.

-Links for the Curious-

“How To Accept Over-Engineering For What It Really Is” (Fagner Brack, 2016) –

Summary of the ASME (American Society of Mechanical Engineers) philosophy on engineering tolerances –

A good explanation of Graceful Degradation (aka failing gracefully) vs. Fault Tolerence (aka robustness) –

Boeing 777 Wing Flex Test –

A bigger tank (the Landkreuzer P.1000 Ratte) –

THUNK – 135. Aumann’s Agreement Theorem & Arguing to Learn

Aumann’s Agreement Theorem suggests that rational folk shouldn’t be able to continuously disagree…so why do we?

-Links for the Curious-

Agreeing to Disagree (Aumann, 1976) –
(You might be intimidated to read the original paper, but it’s only 5 pages long!)

Are Disagreements Honest? (Cowen & Hanson, 2004) –

The Influence of Social Interaction on Intuitions of Objectivity and Subjectivity (Fisher et al, 2016) –

Rationally Speaking, Episode 143 – Scott Aaronson on Aumann’s Agreement Theorem (AAT) & its Implications –  

“Political Polarization in the American Public,” by the Pew Research Center –

“Are Toxic Political Conversations Changing How We Feel about Objective Truth?” (Fisher et al, 2018) –

Lesswrong’s entry on the AAT –

THUNK – 134. The Problem of Universals

-Links for the Curious-


The Stanford Encyclopedia of Philosophy is invaluable for getting a broad understanding of this topic quickly. Some key pages I found helpful:

Metaphysics Overview –
Platonism in Metaphysics –

Abstract Objects –

Nominalism in Metaphysics –

Tropes –


The n-tuple representation of tropes was from “Tropes: Properties, Objects, & Mental Causation” by Douglas Ehring –


Is This a Sandwich? Teaching the Platonic Dialogues through sandwiches (M Ritchey, 2014) –

A Bro And A Philosopher Debate The True Meaning Of A Sandwich (Scherer, 2015) –


THUNK – 133. Game Theory & Power Structures

Power structures: Hobbes thought we’d murder each other w/o ’em, but there’s a great game-theoretic reason, too!

NOTE: Sorry this episode is so late – there have been electrical troubles at THUNK studios. 🙁

-Links for the Curious-

“Meditations on Moloch,” one of the primary inspirations for this whole episode –

I also borrowed heavily from Kenneth N. Waltz’s “Theory of International Politics.”

Beyond the Prisoners’ Dilemma: Coordination, Game Theory, and Law –

The 10 Catanmendments (NOTE: The first Catanmendment rightly assumes the eventual collapse of cooperative strategies, & illustrates the failure mode of specialized alliances –
when you see an “ally” building over to the resource you’re specializing in, they’re diversifying so they don’t need you quite so much, in preparation to dump you.) –

A fun sci-fi book about an intergalactic empire built specifically in this fashion –

A set of responses from anarcho-capitalist philosopher Roderick T. Long, including point 2, defending an-cap’s capability to achieve coordination without the state –

THUNK – 132. On Understanding Others

“I will *never* understand why people X.” Oh really? Let’s turn that dial up to 11 and see what happens.

-Links for the Curious-

In case you’re wondering why I didn’t include a reference to the Netflix show “Mindhunter” (a fictional drama based on the origins of FBI profiling & forensic psychology of serial killers), there’s a growing body of criticism regarding the methods & rigor of those techniques, & I didn’t want to get bogged down hashing out details. Pretty much all of the criminal psychology used in law enforcement today assumes the central premise of this video (that even horrifyingly divergent minds can be simulated & understood to some degree), but it’s possible that the FBI’s whole classification system is pseudoscience –

Psychopathology of terrorists (Piccinni et al, 2017) –

Patterns of Thinking in Militant Extremism (Saucier et al, 2009) –

Inside the Terrorist Mind (Schaefer, 2007) –

Psychology of Terrorism (Borum, 2004) –

Psychological Vulnerabilities and Propensities for Involvement in Violent Extremism (Borum, 2014) –

Genesis of Suicide Terrorism (Atran, 2003) –


THUNK – 131. Puzzles of Epistemology

Gettier problems, extreme skepticism, the closure principle – “knowledge” is one hell of a rabbit hole. Thus: epistemology!

-Links for the Curious-

The Stanford Encyclopedia of Philosophy’s (SEP) fantastic article on epistemology –

The SEP on epistemic closure (& academic skepticism) –

Philosophytube’s “Knowledge Explained” –

Crash Course’s “The Meaning of Knowledge” –

“On Certainty” (Wittgenstein, 1969) –

“Debating Scientific Epistemology” (Dougherty, 2016) –

The Epistemology of Democracy (Anderson, 2006) –


On Local Minima

I was thinking about doing this for THUNK, but I always try to keep THUNK upbeat, & I don’t know if I can do that here. So spoiler warning: serious downers.

One of the interesting concepts associated with evolution is the evolutionary local maximum. Imagine two hills right next to each other: a tall one & a shorter one. Imagine a torrential downpour begins to flood the area, & animals of various types are forced to flee to these hills to keep from drowning. Some are lucky enough to have started off next to the big hill, but some head for “high ground” on the short one.

Clever creatures might figure out that their chances are better on the other hill, & if the water hasn’t gotten too deep, can brave the rising waterline trying to swim over. But after a certain point, the water has become so deep (& the distance between the hills so great) that trying to switch hills is tantamount to suicide – they’d simply drown before they got there. So they climb as quickly as they can to the top of the short hill & hope that the rain stops soon.

This is why we often get suboptimal evolutionary “designs” – chlorophyll can’t capture energy from green light, DNA is prone to kinking up & causing cancer, that sort of stuff. There are obviously better solutions, but the rising tide of survival waits for no creature, & if you spend too much time tinkering to find the best hill to climb, you’ve already drowned.

Of course, you can also drown on the shorter hill; we call that “extinction.”

This phenomenon isn’t restricted to evolution; as game theorists know intimately, any scenario with some sort of pressure to find competitive advantage is subject to “terminal” local maxima. Businesses are an obvious example here, but so are cultures, ideas, nations, & politics. Occasionally, we can navigate from one hill to another, frequently at great cost (e.g. the American revolutionary war transitioning from a precarious colonialist monarchy to a more stable local democracy), but even with a known superior solution, transitioning can sometimes be logistically impossible.

Some examples:

1. The American medical education system is very clearly & unapologetically exploitative, requiring incredible sacrifices of time & quality of life from people who want to be doctors (so hospitals can bleed free labor from them). This causes a shortage of doctors, which forces hospitals to exploit med students to stay profitable & open.

2. The method we use for federal elections (a plurality vote) is demonstrably inferior to other methods (e.g. ranked choice voting). But everyone who is elected by that method has at least some incentive to preserve it, & advocating for something new is politically dangerous.

3. App developers largely succeed/fail based on how much attention they can demand from users. Ideally, they’d build apps to maximize user wellbeing (which would probably include shutting off our phones), & given a large enough user base, that strategy could be very successful, but any developer who “defects” to a less attention-grabbing app is at a disadvantage.

4. As businesses grow, the infrastructure laid in their startup days becomes embedded & calcified under necessity – software, processes, etc. become increasingly essential for daily operation as the company gets bigger, & replacing/updating them with something better gets more costly over time.

We can see the higher ground from here, but I don’t know if there’s actually any practical way to get there, &  I don’t know when or if the water will swallow us up here on our little hills. That scares the hell out of me.

THUNK – 129. Immortality

Coping with the knowledge of our own mortality is often tricky – what if we just tried a different angle? #YOLO

If you’re struggling with suicidal ideation or depression call the national suicide prevention hotline right now: 1-800-273-8255
Also see:

-Links for the Curious-

Terror Management Theory Overview (Wikipedia) –

Is Death Always a Misfortune? (SEP, 2014) –

Descartes projected that his newfound rationalist version of medicine would result in people living a thousand years. Let’s just say he was optimistic. (Shapin, 2000) –

Scientists up stakes in bet on whether humans will live to 150 (Fleming, 2016) –

Some history of hype regarding the human genome project and genomics (Eisen, 2014) –

Evidence for a limit to human lifespan (Dong et al, 2016) –

Human age limit claim sparks debate (Geddes, 2016) –

Many possible maximum lifespan trajectories (Hughes & Hekimi, 2017) –

Evolution of the human lifespan and diseases of aging: Roles of infection, inflammation, and nutrition (Finch, 2009) –

Lust for life: breaking the 120-year barrier in human ageing (Roy, 2013) –