The Psychology of Intelligence Analysis

Listen to the Special Situation Investing Podcast on Fountain, on Apple Podcasts, on the website, or wherever you listen.

Welcome to Episode 55 of Special Situation Investing.

Today we bring you an off-the-beaten-path book review that has the potential to boost you investment analysis and critical thinking skills. I came across the book in a footnote of William Green’s Richer Wiser Happier in which Michael Mauboussin was credited with having found the book insightful.

The Psychology of Intelligence Analysis was written by Richards J. Heuer a multi-decade CIA analyst who hoped to better organize the process of intelligence analysis and improve the utility of CIA analysts reports. There is a clear parallel between the intelligence analysts task of compiling opaque and incomplete information into actionable reports and that of the securities analyst who also seeks to combine fractured information into statistically probable or actionable bets.

I was impressed with the whole of Mr. Heuer’s book as it covers several areas that are relevant to life and investing including memory, perception, cognitive biases and many other tools to improve the clarity of your thinking. I won’t cover all of his work in this short write up, but again, I do highly recommend that you read the full book yourself and that you also consider reading Poor Charlie’s Almanac and Thinking Fast and Slow if you’re looking for additional information on the same topic.

For the purpose of today’s write up, I’d like to focus exclusively on chapter eight of the book. I found it amusing that chapter eight also happens to be one of the three chapters that Buffett always recommends in The Intelligent Investor, perhaps there is something to the eighth chapter of books? In any case, chapter eight of the Psychology of Intelligence Analysis is the heart of the book’s thesis and outlines how to conduct an Analysis of Competing Hypothesis, or (ACH).

But before we dive into the specifics of the ACH method, I’d like to discuss one more parallel between The Intelligent Investor and The Psychology of Intelligence Analysis. The parallel that I drew after reading both books was that they each provide a framework for analyzing specific types of problems. Graham’s work gives us the framework required to value a business and how to view the market itself, while Heuer’s work allows us to deal with broader questions. His book was aimed at improving the utility of intelligence analysts reports with acknowledgment of the fact that the world is opaque and nothing can be fully proven or disproved. This task, is of course similar in many ways to our task as investors and I believe it most closely aligns with the investor’s task of evaluating the “moat” around any business.

The moat, a term popularized by Buffett and Munger, is a term meant to encapsulate all the various aspects of a businesses competitive advantage. The example that comes to my mind is that of Dexter Shoe Company which Buffett purchased in whole because it was a fantastic business by all the standard valuation metrics known to Berkshire’s management. As an investment, however, it was one of Buffett’s rare flops as cheap, overseas labor eventually eroded the company’s profitability and slowly bled away its competitive advantage. Nothing in The Intelligent Investor could have predicted the effects of cheap, foreign labor on an otherwise solid business, but using the ACH method that Heuer proposes in The Psychology of Intelligence Analysis we can consider such factors in an organized and productive way and quite possibly avoid committing our own forced investment errors.

With that broad context out of the way, let’s get back to the specifics of the ACH method. The ACH method forces the practitioner to formulate multiple hypothesis and then try and disprove, or discredit, each theses one by one. In the end you don’t have a winning hypothesis but rather a least disproved hypothesis. This focus on invalidating a theory rather than proving a theory is one of the aspects I found to be quite powerful. Typically, we form a single hypothesis that most appeals to ourselves and then go about piling on more and more confirming evidence for that hypothesis, but by flipping this model to one that works to disprove our own best loved idea, we’re more likely to arrive at a well-reasoned conclusion.

As with most things, the ACH method is best illustrated by working through an example of the process. For today’s example, let’s revisit one of the show’s real-time investment write-ups, that of Mesabi Trust. As regular listeners of the show will remember, Mesabi is a royalty trust with no employees that derives its income passively via royalties it collects on all iron ore mined from its properties in Minnesota’s iron range. The operating company which extracts the actual ore, and which owes a royalty to Mesabi Trust unit holders, is the steel company Cleveland Cliffs.

Cleveland Cliffs CEO, Mr. Goncalves, recently took issue with what he believes to be exorbitant royalty fees owed to Mesabi Trust and announced that he would idle the Mesabi Trust mining operations on Mesabi Trust properties to avoid paying royalty fees. For more details on the company’s history and the dispute you can review one of our previous podcasts (Episode xx) on the topic, but for the time being you now understand the main issue under dispute. As we work through the ACH method using the Mesabi Trust example, please reference the ACH chart included at the end of the write up.

Now if you’re a unit holder in Mesabi Trust, which I am, and Cleveland Cliffs idles the mining operation, which they did, then you’re not going to get paid. As a unit holder, you’re keenly interested in how this dispute will be resolved, when it will be resolved, and most importantly when your dividend will resume.

Using the ACH method, we can get this problem out of our head and onto a piece of paper for better analysis and the process begins with brainstorming multiple hypotheses. It may be difficult to follow along if you’re listening to the podcast, so I highly recommend you review the Substack article and the example ACH when able. Now, continuing with our brainstorming of hypotheses, we can come up with the following possible options that Cleveland Cliff’s might take.

1. Cleveland Cliffs resumes mining operations on Mesabi Trust land in May 2023.

2. Cleveland Cliffs breaches contract with Mesabi Trust.

3. The contract between Cleveland Cliffs and Mesabi is renegotiated at a lower royalty rate.

The key thing to remember at this stage is that any possible hypotheses should be included. We’re not attempting to select a best hypotheses at this stage. In fact, just the opposite, we want to capture all possible outcomes without bias to avoid excluding a valid hypotheses early on due to primacy, strength of an idea, group think, or any number of other psychological biases that might lead us down the wrong fork in the road.

Now, once we’ve captured the relevant hypotheses, we’re going to list them one at a time from left to right, with each one at the top of its own vertical column. Listing our hypotheses from left to right across the top of a page will allow us to consider the impact of each relevant fact independently and sets us up for the next step in the ACH process which is listing relevant facts on the left side of the page beginning at the top and moving down one line for each new fact. Again, difficult to envision if you’re only listening to the podcast, but the idea is that you have hypotheses listed from left to right at the top of a page and facts listed from top to bottom on the left side of the page and you’re left with one individual cell for each hypotheses/fact intersection in which to consider that facts impact on a specific hypotheses.

As we move into analyzing the impact of specific fact on a given hypotheses we’re going to place a plus sign in the box where that fact intersects that hypotheses if the fact supports the hypotheses and a negative sign if it negates it. Again, an example will best illustrate what we’re doing so we will now revisit the Mesabi Trust discussion.

Across the top of my ACH spreadsheet I listed my hypotheses from one through three and down the side of the spreadsheet I listed relevant facts with the first fact at the top being that, in 2019 Cleveland Cliffs invested over $100 mm in a taconote processing facility adjacent to and serving the Mesabi Trust mining operation. In considering this facts impact on each hypotheses, I put a plus under Hypotheses 1 (Cliffs will resume mining in May 2023 after the previously announced mine idle is complete) and a minus under Hypotheses 2 (Cliffs breaches contract) and a minus under Hypotheses 3 (The contract is re-negotiated). I did this because I believe that Cliffs’ sunk capital cost in the mine will prompt them to resume operations versus breach contract or drag the process out in contract negotiation.

Clearly, we’re making subjective judgments as we move, fact by fact, through the spreadsheet, but ultimately we’re developing a picture of which hypotheses are dis-proven, which are very unlikely, and which are the least disproven (which is another way of saying most likely). That we’re trying to disprove rather than prove each hypotheses is no small matter. Focusing on disproving a theory is, analytically, much more rigorous than attempting to prove a theory for reasons that are covered extensively in the book.

Once you’ve fully worked through an ACH clear patterns emerge and you see some columns with mostly minus signs listed and others with mostly positive signs interspersed with a few negatives. This final stage is where I started to see the power of the ACH framework in several ways.

First off, it prevented me from latching on to an idea at the outset and then gathering more and more confirming evidence for my first or favorite idea. Confirmation bias is a well documented error in human cognition and one that can cost investors significantly when we fall prey to it. The ACH’s focus on disproving each theory, however, forced me to be a much harsher grader of my own best loved ideas and prevented me from feeling more sure of a position than the facts justified.

ACH also allows you to identify the diagnosticity of information. The example provided in the book is that of a physician who aims to specifically diagnose a patients illness. Let’s assume that the doctor believes the patient has either pneumonia, typhoid fever, or malaria. Because a fever is associated with each illness the presence of a fever does not allow the doctor to further diagnose the disease and therefore the presence of a fever would in this case have low diagnosticity. Non-diagnostic information in an ACH chart is evident whenever all pluses or all minuses are presented for one fact across multiple hypotheses. Whenever we see this pattern in our ACH we know that the information has no diagnostic power and this is something that casual thinking about the problem would not have revealed to us.

Another benefit to the ACH method is that you can weigh the significance of each fact independently and even assign an impact percentage to each in addition to the plus or minus symbol. An example from the Mesabi Trust scenario might be a catastrophic collapse in global iron demand. This would be a negative for resumption of any mining and the impact would be very significant. If I were to add percentages to the existing plus or minus symbols, I would add a significant percentage to that fact to remind me that if this scenario played out it would be a major factor effecting my thesis. Again, this aspect of ACH reminds us that no matter how likely we believe our thesis is to play out there are always a few black swans waiting in the darkness that can negatively impact our favorite theory.

A further benefit of ACH is that it allows us to consider far more information and to consider it much more clearly than would otherwise be possible. More often than not, we run through problems in our head, playing out scenarios and weighing one fact against another. But as anyone who’s ever tried to remember a 1-800 number for more than a few seconds while working another task can attest to, our brains working memory is quite limited. As investors charged with putting others and our own capital at risk, working through multiple potential future scenarios in our head using opaque and incomplete information is setting ourselves up for failure. The ACH method allows you to systematically work through multiple theories about any given question and individually play facts and data off of each other in an organized way.

Breaking facts and thesis out of your head and analyzing them separately on paper also clarifies the areas in which you haven’t done your homework. Brainstorming hypotheses might reveal potentialities that you hadn’t already considered or would not have thought of without ACH-level analysis and considering each fact independently shows us the gaps in our knowledge. Unless we removed the problem from our head and organized it on paper we might never have considered each alternative or gathered all the required information and we’d be left with a poorly reasoned investment thesis.

This write-up is in no way a complete list of the benefits of ACH but for the sake of time I’ll wrap up with one additional benefit of the process. The final benefit I’ll cover is how the framework actively informs your thesis and allows for constant revision. To revisit the Mesabi Trust example, once I know that global collapse in iron ore demand kills my investment thesis I’m going to list that fact separately along with other things to watch out for. In other words, I’ll produce a list of information that I’m on the lookout for because it significantly impacts my investment thesis. In this way, I’m able to find signal in the noise of information that all investors are daily barraged with. Furthermore, the ACH is a living document, you can come back to it later and add a new theses or a new set of facts that just came to light and because your thoughts were organized so well by the ACH initially, the addition of new information does not disturb the work you’ve already done.

With that I hope you enjoyed this episode of the show. I highly encourage you to read The Psychology of Intelligence Analysis on your own as there are far more useful insights than I can cover here. I also hope that you’re now armed with a new tool in your toolkit that can help your organize the myriad facts and hypotheses that go into every investment decision. Between The Intelligent Investor’s valuation framework and The Psychology of Intelligence Analysis framework for evaluating a business moat you should be well armed to make your own assessment of any investment proposal.

* Note: This chart is only meant to be an ACH example and not a fully developed analysis of the Cleveland Cliff’s royalty dispute with Mesabi Trust.

So in summary, I just want to say again, I really enjoyed the book. I think that the method is a very powerful one for organizing your thoughts and your theories about the future. When it comes to analyzing how enduring is a moat, how enduring a company’s competitive advantage is, I found this very useful. And whether your looking at iron and an iron royalty business or an oil and gas business, how enduring are those businesses? What factors could impact them, ESG, finding a new energy source, etc. All these various things could come in and change the industry and change the profitability. With a method like ACH you have a tool where you can start to put the facts and theories about the future out, and organize them, and see the information in a way that informs decisions.

With that, I really hoe you enjoyed the episode. We for sure enjoyed bringing this information to you.

We really appreciate all the recent support on Fountain. It seems like that’s the easiest avenue for folks to support the show. And I’d just say that any amount of support, even a very small amount, is just hugely appreciated because it shows us that we are bringing you content that you appreciate, that you’re learning, that you’re growing as an investor along with us. The positive feedback and the encouragement is probably the biggest benefit and then what is throw us in the way of Sats is also hugely appreciated. Thank again.

We will be back soon with another investment write-up.