Escape from Model Land

This book (Escape from Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It by Erica Thompson) was a fun book to go through. So much of what we read, how we interpret the world, our contexts are models. This helps put models in context.

Overarching Practical Idea

The book’s most practical habit is to ask 4 questions whenever you encounter a model:

1. What is it for? Use the model
2. What assumptions does it make? Inspect its assumptions
3. What does it leave out? Listen for information it cannot represent
4. What should we do if it is wrong? Make decisions that remain defensible if the model is wrong

Escape from Model Land

1. "Locating Model Land"

• Identify the model’s intended purpose before using its output.
• Ask what real-world situation the model represents—and what it leaves out.
• Treat model results as conditional: "If these assumptions hold, then this follows."
• Check whether the model is being used in the same context for which it was designed.
• Look for the boundary between the model’s simplified world and reality.

Example: A city uses a traffic model to predict congestion. The model assumes commuters choose the fastest route and that roads remain open. After a major concert, thousands of people leave at once, police redirect traffic, and some roads close. The model’s output may be mathematically correct within its assumptions but unhelpful in the real situation.

Lesson: First determine the conditions under which a model applies.

2. "Thinking Inside the Box"

• List the model’s assumptions explicitly rather than treating them as neutral facts.
• Ask which variables, groups, behaviors, or outcomes have been excluded.
• Test whether the model’s categories actually fit the problem you are investigating.
• Be cautious when a model produces a very definite answer from a narrow set of inputs.
• Try changing important assumptions to see whether the conclusion changes.

Example: A hospital model predicts emergency-room demand using age, diagnosis, and previous admissions. It excludes transportation problems, language barriers, housing insecurity, and whether patients can afford medication. The model may predict demand reasonably for some patients while systematically underestimating demand among others.

Lesson: Examine who and what the model leaves outside its boundaries.

3. "Models as Metaphors"

• Use models as tools for thinking, not as literal copies of reality.
• Pay attention to what a model helps you see and what it makes harder to notice.
• Do not assume that a model that works metaphorically in one setting applies directly elsewhere.
• Compare multiple models when the issue is complex.
• Explain the model’s meaning in ordinary language before relying on its numbers.

Example: A manager models a company as a machine: inputs go in, processes occur, and outputs come out. That metaphor may help identify bottlenecks, but it misses morale, creativity, conflict, and informal relationships. Applying the machine metaphor too literally could lead the manager to treat employees as interchangeable parts.

Lesson: A model can illuminate one aspect of reality while obscuring another.

4. "The Cat that Looks Most Like a Dog"

• Distinguish between a model that resembles reality in one respect and one that is reliable overall.
• Do not confuse a good fit to past data with a valid explanation of the underlying system.
• Check whether the model has been tested on situations outside the data used to build it.
• Treat statistical fit as evidence, not proof.
• Ask whether the model captures the causal mechanism or merely produces similar-looking results.

Example: A machine-learning system is trained to identify fraudulent credit-card transactions. It performs extremely well on last year’s data, but fraudsters change their methods. The system still produces impressive accuracy statistics while failing to detect new types of fraud.

Lesson: A model’s good performance on familiar data does not guarantee that it understands the underlying process.

5. "Fiction, Prediction and Conviction"

• Separate a plausible story about the future from a genuine prediction.
• Treat scenarios as conditional narratives, not promises.
• Ask which assumptions make a forecast possible.
• Avoid becoming emotionally or institutionally committed to one preferred scenario.
• Use several contrasting scenarios to reveal vulnerabilities and possible surprises.
• Make decisions that remain sensible even if the favored prediction is wrong.

Example: A company develops three sales scenarios: strong growth, moderate growth, and decline. Executives become attached to the strong-growth scenario and hire aggressively. When demand falls, they discover that they had treated a useful scenario as a forecast.

Lesson: Scenarios should prepare us for possibilities, not persuade us that one future is certain.

6. "The Accountability Gap"

• Establish who is responsible for a decision when a model informs it.
• Do not allow "the model said so" to replace human judgment or accountability.
• Record the assumptions, limitations, and uncertainty attached to important model outputs.
• Include affected people in decisions rather than treating them as data points.
• Examine who benefits and who bears the costs when a model is used.
• Build in opportunities to revise decisions as new evidence appears.

Example: An insurance company uses an algorithm to deny some claims. When a customer challenges a denial, employees say the decision was "made by the system," while the software company says it only supplied the tool. Nobody accepts responsibility for the outcome.

Lesson: Human institutions remain accountable even when decisions are assisted by models.

7. "Masters of the Universe"

• Be skeptical of models presented as capable of controlling complicated systems.
• Recognize that models can influence the very behavior they are supposed to predict.
• Watch for feedback loops: predictions may change actions, and actions may change outcomes.
• Avoid concentrating too much authority in technical experts or model designers.
• Stress-test systems against unusual and extreme conditions, not only average ones.
• Treat uncertainty and model failure as governance issues, not merely technical inconveniences.

Example: A central bank relies on a model suggesting that financial institutions are individually safe. Each bank then increases similar investments because the model encourages confidence. When asset prices fall, all the institutions face losses simultaneously, creating a systemic crisis.

Lesson: A model can change behavior and thereby create the risks it failed to predict.

8. "The Atmosphere is Complicated"

• Distinguish between confidence in a broad physical conclusion and confidence in a precise local prediction.
• Use climate models to explore possible futures, not to claim exact knowledge of one future.
• Avoid false precision in estimates of distant or highly uncertain impacts.
• Compare model projections with observations and with other modelling approaches.
• Separate scientific findings from value judgments about acceptable risk, cost, and fairness.
• Prefer robust policies that work across a range of plausible climate outcomes.

Example: A climate model estimates that a region’s average temperature will rise by a certain amount by the end of the century. A policymaker interprets that number as a precise prediction for every town and every year. The result is a false sense of certainty about local weather, even though the broader warming trend may be well supported.

Lesson: Confidence in a broad conclusion does not imply precision at every level of detail.

9. "Totally Under Control"

• Remember that models of human behavior are especially sensitive to changing behavior and incentives.
• Treat epidemic, economic, and social forecasts as conditional rather than inevitable.
• Update models when people change their behavior in response to the forecast or policy.
• Communicate uncertainty clearly without turning it into either panic or complacency.
• Use models to compare interventions, not simply to announce a predicted outcome.
• Preserve room for expert judgment and local information when conditions change quickly.

Example: Public-health officials use an epidemic model to estimate hospital demand. The forecast assumes people maintain their usual behavior. Once the forecast is publicized, people cancel gatherings, adopt protective measures, or change travel plans. Actual infections then differ from the original projection, not necessarily because the model was useless, but because the model influenced behavior.

Lesson: Models of human systems can change the systems they describe.

10. "Escaping from Model Land"

• Step outside the model and ask whether it is framing the right question.
• Combine modelling with observation, lived experience, historical knowledge, and practical expertise.
• Use model disagreement as information about uncertainty and assumptions.
• Prefer decisions that are resilient when forecasts fail.
• Make value judgments openly instead of hiding them inside technical parameters.
• Treat models as aids to judgment, not substitutes for judgment.

Example: A coastal town uses a flood model to decide where to build defenses. Engineers consult the model, but residents point out that drainage fails in places the model treats as safe, and local workers identify roads that become impassable first. The final plan combines modelling with local knowledge, historical flood records, and a policy of avoiding critical facilities in high-risk areas.

Lesson: Good decisions often require stepping outside the model and combining it with other forms of knowledge.

---

Tags:

beekeeping book notes booklists chess economics family how-to life life helps money music philosophy politics productivity quotes race-plans race-results running runs stoicism strategy sundays training