ATRIUMsearch → argument graph
Audio · 2024-04-14 · 1h 10m · 4 moments

TIP623: The Art Of Decision Making w/ Annie Duke

Kyle Grieve chats with Annie Duke about her own story of quitting and how it helped sparked the idea for one of her books, the importance of base rates in helping us make better decisions, how to improve our investing processes when we have long feedback loops, the importance of using kill criteria to quit an investment or hypothesis, how to use a quitting coach to help you quit things we hold onto for too long, the importance of dissociating ourselves from our most cherished ideas, and a whole ✦ AI generated

timeline · colored by role

01
Mechanism

You can't manage risk unless you know what your edge is, yet most investors focus on risk management formulas while making too many assumptions about their expected value.

Annie argues that risk management is downstream of knowing your edge (expected value). Jeff Yass's key concern is that people assume they're winning when they're actually losing, and they over-invest in complex risk formulas while neglecting the EV question.

transcript

Annie Duke: But again, the thing about all of this is that with risk, it's like you can't manage risk unless you know what your edge is. You have to know what your expected value is in order to be able to manage risk. And I think the big mistake that people make in investing is that you have risk managers and there are formulas that you can apply to risk. And I think they make an assumption about their expected value and they get really, really focused on the risk management side of things because it's a problem that I think is easier to solve. In other words, if you have an assumption about what your EV is and what the vol is, right, now you can actually just apply a formula and you can start to get into some sort of dive headlong into the risk management side of things. And that's Jeff Yass's point, is that he feels like people are making too many assumptions about the EV part. Just like what is your edge in the 1st place, right? And I think everybody comes in assuming they have an edge. And then they go to all of these risk management formulas, right? And he's like, I don't even care about that. Like, I'm just so afraid that I think I'm winning when I'm actually losing.

gives example · 1

02
Mechanism

To find your edge, you must start with the base rate (what usually happens in similar situations) before layering in your own personal experience — not the other way around.

Annie explains that people typically start from their own personal experience and skill when estimating success, but they should instead start with the base rate — the statistical reality for similar situations — and only then adjust up or down based on their specific circumstances.

transcript

Annie Duke: So prior experience matters, but not as much as the base rate. This is something Michael Mobison just like hits home so well. The thing that he is always saying, and I've seen a lot of talks that he's given, is the way we normally go about making decisions is we think about the problem that we're approaching and then our own experiences. And then we use that to decide what we think our EV is going to be. So if I'm trading and I'm thinking about a particular thesis that I might trade or whatever, I'm going to be thinking about how smart I am and how unique the thesis is and so and so forth. And my past experiences with winning and all of that. But what I should really do to start with is his point is before I get to my own experience, I have to start with base rate. Let me give you an example. Let's imagine that I'm thinking about opening up a restaurant. And I've worked in restaurants before, and the chefs that I've worked for have said that I'm amazing and I'm a great cook. and the customers are always complimenting my food. And now I'm opening up my own joint and I'm thinking about how much people love my food and how busy the other restaurants I've been in have been. And I estimate that the probability that my restaurant is going to thrive is 80%. Because notice I'm thinking about all these things that are personal to my experience and how much people love the food that I make and how good a cook I am and how well the restaurants that I've been in have done and so on and so forth that I've cooked in have done. But that's not the place that I want to start. That stuff matters. But what I actually want to start is with the base rate. So what I want to think about is what usually happens in a situation that's similar to the one that I'm considering. So this is going to give you, in investor speak, beta. You need to know what beta is. In restaurants, if I think about, if I actually look up what percentage of first-time restaurants are still open at the end of the first year, survive a year, it's 40%. So now I'm going to start with 40%. And now I can say all those same things. I'm a great cook. People love my food. Customers are always complimenting me at the restaurants that I've cooked for, the chefs that I've worked with have said that I'm amazing and I should open my own joint. And so I think that my chances are better than 40%, but not 80%, right? Like I'm not doubling my chances here. So this allows me to get grounded in a reality that I can now toggle up or down from.

explains mechanism · 1

03
Claim

Investors who claim feedback loops are too long (5-10 years) are wrong — you can close them much faster by tracking intermediate objective signals and whether your thesis is playing out.

Annie pushes back hard on the venture capital argument that feedback loops are too long to be useful. She points out that investors don't go to sleep for 10 years — they can track objective milestones like ARR growth, talent retention, and fundraising rounds, and can check whether their original thesis is being validated, all within months.

transcript

Annie Duke: I said, What do you mean the feedback loops are long? And they said, What do you mean? We don't know if it excerpts for... And I said, I'm sorry, do you invest in the company and then you go to sleep like Rip Van Winkle and 10 years later you wake up and you find out what happened? When you invest in a company, a couple of things are true. Two different categories of things are true. Both of which allow you to close the feedback loop more quickly. Thing number one is that you actually know objective things about the company. You know whether ARR is growing. You know whether they're hiring top talent and retaining the top talent. You know whether they fund a series A. You know whether it's an up round, a flat round, a down round. You know what the quality of the syndicate is, same thing for B, same thing for C. If you're, depending on the speed of the market, if you invest at seed, for example, you're going to know something very significant about that company between six and 16 months later. That sounds like a lot faster than 10 years, the first thing that you know. The second thing, and this is true across all investing, is that you're investing in the company because you're making a particular bet, and the bet is your thesis. If it's in the market, you're saying, I think that I know something that the market doesn't know. Why do I know that that's what your thesis is? Because otherwise you would be indexing the market. You're not indexing the market. So you're saying, I believe that the market has this mispriced temporarily. I believe the market is efficient, but not every single moment. It's overall efficient. So I believe that at this moment, when it comes to this stock or whatever, this stock or this option, the market does not have this price efficiently. So you have a thesis about why that is true. It's true when you're investing in a company. I believe that this market is going to be a great market to be in. This product is going to have a competitive advantage. They're going to execute in this particular way, so on, so forth. And those kinds of things, you can find out very quickly, even when you're investing in a seed stage company. You can see like, are they executing in the way that I thought they were? Is their product gaining traction? Like, so on, so forth, right? So all of these things like, look, is it like poker or jujitsu where you're going to find out two seconds later? No, but you're going to find out way more than 10 years.

04
Mechanism

To evaluate your process when you have few positions (e.g., 7 in a long-short fund or 20 in a venture fund), you must generate many more data points by making intermediate forecasts on milestones, not just waiting for final outcomes.

Annie explains that with only 7 or 20 investments, final outcomes are too luck-driven to evaluate your process. The solution is to make explicit probability forecasts on intermediate milestones (like whether a seed company will raise a Series B at an up round) for every opportunity that comes through. With 100 companies per year and an 18-month resolution, you rapidly accumulate enough predictions to assess your judgment, stripping luck out of the evaluation.

transcript

Annie Duke: But how do we deal with that in a situation where your portfolio might have seven positions at a time, for example, or in venture where maybe you have 20 companies or 25 companies in a fund, right? And you're generating more outcomes than the final outcome. So it goes back to what I'm saying. So let's imagine this. Let's imagine that you're an investor investing in Series A. That's your specialty. And for every single company, every company that comes into partner meeting, You have to make a forecast of the probability that company will fund at series B. Now, you can put restrictions around it, right? Like you can say the probability that it will fund at series B with an up round. So you can do whatever you want. So we can think about what are those things that are really necessary for this to survive? Because the thing is, I can guarantee you, won't ever have a fund returner if you invest at series A that doesn't invest at series B, likely not a down round, which is a bad sign. Let's take that one data point, right? But then you can actually think about what are all the data points that I now want to be making predictions about where I'm going to start to know those things more quickly. Now, what's nice about that is it does double duty. What is it massively increases the outcomes that you have so that you can get more into the thousand coin flip situation? Because let's imagine that you have 100 companies come into partner meeting in a year And you've made that, we're just talking about the one forecast right now, you've made that one forecast for all 100 companies in the space of 18 months, you're going to know for every single one of those companies, because it's public whether they do a B, how good of a predictor am I of this particular thing that really, really matters. So now you start to be able to close these feedback loops faster because you're generating so much more data that we start to remove the luck and it becomes about sort of your predictive power.

extends · 2

Highlight slides
Related episodes