vault backup: 2025-12-16 16:54:09

hubbard_2020_failure.md

@@ -150,6 +150,257 @@ For commentary see the companion
 
 #### A Note About Black Swans
 
+The *exsupero ursus* fallacy is reinforced by authors of very popular books
+who seem to depend heavily on some version of the fallacy.
+One such author is former Wall Street trader and mathematician Nassim Taleb.
+He wrote *The Black Swan* and other books critical of common practice in risk management,
+especially in (but not limited to) the financial world,
+as well as the nonquantitative hubris of Wall Street.
+
+A heretic of financial convention, he argues that Nobel Prize---
+winning modern portfolio theory and options theory (briefly mentioned in chapter 5)
+are fundamentally flawed and are in fact no better than astrology.
+In fact, Taleb considers this prize is itself an intellectual fraud.
+After all, as he rightly points out, it was not established in the will of Alfred Nobel,
+but by the Royal Bank of Sweden seventy-five years after Nobel's death.
+He even claims that once, in a public forum, he riled up one such prizewinner
+to the point of red-faced, fist-pounding anger.
+
+Taleb bases a lot of his thesis on the fact that the impact of chance
+is unappreciated by mostly everyone.
+He sees the most significant events in history as being completely unforeseeable.
+He calls these events *black swans* in reference to an old European expression
+that went something like "That's about as likely as finding a black swan."
+The expression was based on the fact that no European
+had ever seen a swan that was black---until Europeans traveled to Australia.
+Until the first black swans were sighted, black swans were a metaphor for impossibility.
+Taleb puts September 11, 2001, stock market crashes, major scientific discoveries,
+and the rise of Google in his set of black swans.
+Each event, he argues, was not only unforeseen
+but *utterly unforeseeable* based on our previous experience.
+People will routinely confuse luck with competence
+and they will presume that the lack of seeing an unusual event to date
+is somehow proof that the event cannot occur.
+
+Managers, traders, and the media seem to be especially susceptible to these errors.
+Out of a large number of managers,
+some managers will have made several good choices in a row by chance alone.
+This is what I called the Red Baron effect in a previous chapter.
+Such managers will see their past success
+as indicators of competence and, unfortunately,
+will act with high confidence on equally erroneous thinking in the future.
+Taleb recognizes the problems of overconfidence researched by Kahneman and others.
+Indeed, Taleb says Kahneman is the only Economics Nobel Prize winner he respects.
+
+I think part of Taleb's skepticism is refreshing and on point.
+I agree with many of Taleb's observations on the misplaced faith in some models
+and will discuss this further in the next chapter.
+I might even include Taleb as one source of inspiration
+for identifying new categories of fallacies
+(and giving it a Latin name in order to sound official).
+Taleb coined a fallacy he refers to as the *ludic fallacy*,
+derived from the Latin word for "games of chance."
+Taleb defines the ludic fallacy as the assumption that the real world
+necessarily follows the same rules as well-defined games of chance.
+
+Now, here is where Taleb errs.
+He doesn't just argue that risk management is flawed.
+He argues that risk management itself is impossible
+and that all we can do is make ourselves *antifragile*.
+I think he is just using a very different definition of risk management---
+which even he uses inconsistently.
+No matter what he calls it, he is promoting a particular set of (vaguely defined) methods
+that have the objective of reducing risk.
+This reduction in risk will require resources.
+Using the definition I propose in [chapter 6],
+determining how to use resources to reduce risk is part of risk management.
+He actually contradicts himself on this point
+when he promotes redundancy as a method of becoming antifragile
+and refers to it as the "central risk management property of natural systems."
+So, yes, we are both talking about risk management.
+He focuses on particular approaches to it, but it is risk management just the same.
+
+Confusion and inconsistency about whether managing fragility is, in practice,
+part of managing risks is not the only problem in his thesis.
+Taleb commits every form of the *exsupero ursus* fallacy
+throughout most of what he writes.
+
+Specifically,
+(1) he presumes the lack of perfection of one model
+automatically necessitates use of the other regardless of relative performance,
+(2) he commits the anecdotal fallacy
+when looking for evidence of relative performance, and
+(3) he presumes that a given model was even being used
+when he identifies them as the culprit in major risk events.
+
+In an interview for *Fortune* Taleb claimed,
+"No model is better than a faulty model."
+Again, having no model is never an option.
+One way or another, a model is being used.
+Taleb's model is simply his common sense,
+which is, as Albert Einstein defines it,
+"merely the deposit of prejudice laid down in the human mind before the age of eighteen."
+As with every other model, common sense has its own special errors.
+
+We've seen the research that shows overwhelming evidence
+of the flaws of unaided intuition compared to even simple statistical models,
+and Taleb offers no empirical data to the contrary.
+Taleb does briefly mention the work of Meehl but dismisses it.
+Without making any mention of the huge numbers
+of conclusive results by Meehl and his colleagues,
+Taleb claims the entire body of research is invalid
+by claiming "that these researchers did not have a clear idea
+of where the burden of empirical evidence lies"
+and goes on to suggest that they lacked "rigorous empiricism."
+He offers no details about how more than one hundred peer-reviewed,
+published studies by several researchers veers from the required rigorous empiricism.
+
+Kahneman, who actually is a psychologist like Meehl,
+would apparently disagree with Taleb on Meehl's methods.
+Taleb considers Kahneman a significant influence on his work,
+but who does Kahneman consider to be a significant influence on his work?
+<u>Meehl</u>.
+I wouldn't presume to speak for Kahneman
+but I wonder if he might point out to Taleb
+how the burden of proof was accepted and met overwhelmingly by Meehl,
+whereas Taleb's evidence merely amounts to, at best,
+selected anecdotes of shortcomings or entirely imagined straw man arguments.
+Taleb even sometimes cites the work of Phil Tetlock
+to support some other point he makes
+but never references Tetlock's enormous twenty-year study
+where he concluded that it was "impossible"
+to find a domain where humans clearly outperformed algorithms.
+
+Instead of relying on large controlled studies,
+Taleb commits the error of arguing that single events
+effectively disprove a probabilistic model.
+He uses the apparent unforeseeability of specific events
+as evidence of a flaw in risk analysis.
+The implication is that if quantitative analysis worked,
+then we could make exact predictions of specific and extraordinary events
+such as 9/11 or the rise of Google.
+When arguing against the use of various statistical models in economics
+he states that "the simple argument that Black Swans and tail events
+run the socioeconomic world---and these events cannot be predicted---
+is sufficient to invalidate their statistics."[^09-12]
+Yes, the rare events---black swans---
+are individually impossible to predict precisely.
+But unless he can show that his alternative model (apparently his intuition)
+would also have predicted such events exactly,
+then he commits *exsupero ursus* when he says imperfection alone
+is sufficient to prefer intuition over statistics.
+
+In addition to Kahneman,
+it is worth pointing out others whose work Taleb cites to make a point but who,
+if you actually looked at what they are doing, would contradict Taleb.
+For example, Taleb says he admires the mathematician Edward Thorp,
+who developed a mathematically sound basis for card counting in blackjack in the 1960s.
+Now, if the objective of card counting was to predict every hand,
+even the most extraordinarily rare combinations as Taleb would seem to require,
+then Ed Thorp's method certainly fails.
+But Ed Thorp's method works---that's why the casinos quit letting him play---
+because his system resulted in better bets on average after a large number of hands.
+Taleb is also a fan of the mathematician Benoit Mandelbrot,
+who used the mathematics of *fractals* to model financial markets.
+Similar to Thorp and Taleb,
+Mandelbrot was equally unable to predict specific extraordinary events exactly,
+but his models are preferred by some
+because they seem to generate more realistic patterns
+that look like they *could* be from real data.
+
+If anecdotal evidence were sufficient to compare model performance,
+one could simply point out that Taleb's investment firm, Empirica Capital LLC,
+closed in 2004 after several years of mediocre returns.[^09-13]
+He had one very good year in 2000 (a 60 percent return)
+because while everyone else was betting on dot-com, he bet on *dot-bomb*.
+But the returns the following years were far enough below the market average
+that the good times couldn't outweigh the bad for his fund.
+
+Similar to the news pundits rejecting Nate Silver's findings
+or the sportscasters rejecting the methods used by the Oakland A's,
+Taleb merely shows that it is possible to find an error in a model
+if one looks hard enough.
+Again, the question is not whether to model (intuition is a model, too)
+or whether one model is imperfect (both models are imperfect)
+but which measurably outperforms the other
+and does so in many trials not just single anecdotes.
+
+Finally, Taleb makes the error of presuming what methods
+were actually being used when he blames them for an event.
+He argues, for example,
+that the downfall of long-term capital management (LTCM) disproves options theory.
+Recall that options theory won the Nobel Prize for Robert Merton and Myron Scholes,
+both of whom were on the board of directors for LTCM.
+The theory was presumably the basis of the trading strategy of the firm.
+But an analysis of the failure of LTCM shows that a big reason for its downfall
+was the excessive use of leverage in trades---an issue that isn't even part of options theory.
+That appeared to be based on intuition.
+
+Taleb also states that the crash of 1987 disproved modern portfolio theory (MPT),
+which would seem to presume
+that at least some significant proportion of fund managers used the method.
+I find fund managers to be tight-lipped about their specific methods,
+but one fund manager did tell me how
+"learning the theory is important as a foundation
+but 'real-world' decisions have to be based on practical experience, too."
+In fact, I found no fund managers who didn't rely partly, if not mostly, on intuition.
+Finally, if we are looking for explanations of the mortgage crisis,
+neither MPT nor options theory had anything to do with the practice
+of giving out mortgages to large numbers of people lacking the ability to pay them.
+That was more of a function of a system
+that incentivized banks to give risky loans without actually accepting the risk.
+
+Finally, Taleb seems to make a variety of other points
+that, similar to the previous points, seem so inconsistent
+he ends up undermining the point he makes.
+For example, explaining the outcomes
+in terms of the narrative fallacy committed by others
+is sometimes itself a narrative fallacy.
+Arguing that "experts" don't know so much is not supported by quoting other experts.
+He argues that rare events defy quantitative models,
+but then gives specific examples of computing rare events with quantitative models
+(he shows the odds of getting the same result in a coin flip many times in a row
+and argues the benefits of Mandelbrot's mathematical models
+in the analysis of market fluctuations).
+
+Taleb criticizes the use of historical data in forecasts
+but apparently sees no irony in his argument.
+He looks at several examples in which history was a poor predictor.
+In other words, he is assessing the validity of using historical examples
+by using *historical examples*.
+What Taleb and others prove with such examples
+is merely that what I will call a *naive* historical analysis can be very misleading.
+Taleb demonstrates his point by using the example of a turkey.
+The turkey had a great life right up until Thanksgiving.
+So, for that turkey, history was a poor indicator.
+So how is Taleb able to see this problem?
+He simply looks at the larger history of turkeys.
+
+All he is doing is using what we may call a *history of histories*,
+or *meta-historical analysis*, to show how wrong naive historical analysis can be.
+The error in historical analysis in a stock price, for example,
+is to look only at the history of *that* stock and only for recent history.
+If we look at all historical analysis for a very long period of time,
+we find how often naive historical analysis can be wrong.
+
+Taleb's own "experience," as extensive as it might be (at least in finance),
+is also just a historical analysis---just a very informal type
+with lots of errors in both recall and analysis, as shown in [chapter 7].
+No thinking person can ever honestly claim
+to have formed any idea totally independent of previous observations.
+It just doesn't happen.
+
+Even Taleb's ludic fallacy seems to be a fallacy itself.
+Sam Savage calls it the "ludic fallacy-fallacy."
+As Savage describes it, we cannot rationally address real-world problems of uncertainty
+"*without* first understanding the simple arithmetic of dice, cards, and spinners."
+Of course, Taleb is right when he says we shouldn't *assume*
+that we have defined any problem perfectly.
+That certainly would be an error, and if that were Taleb's point, that would be valid.
+But, again, whether a particular model is perfect is not the right question.
+The most relevant question is whether a probabilistic model---even a simple one---
+outperforms the alternative model, such as intuition.
+
 #### Major Mathematical Misconceptions
 
 #### We're Special: The Belief That Risk Analysis Might Work, But Not Here
@@ -182,6 +433,14 @@ For commentary see the companion
 
 #### Using Data For Initial Benchmarks
 
+##### It's Been Measured Before
+
+##### You Have More Data Than You Think
+
+##### You Need Less Data Than You Think
+
+##### A Reference Class Error: Revisiting the Turkey
+
 #### Checking The Substitution
 
 #### Simple Risk Management