False Positives And False Negatives

We often make mistakes, often due to poor decision making and often with negative consequences.

I have the habit of leaving my umbrella at home, thinking I will be able to get from the bus stop to work before it rains. I get caught in the rain often. My decision is based on the expectation it won't rain in the brief time I am outside.

In the scientific process, this is described by type 1 and type 2 errors. A type 1 is a false positive; a type 2 is a false negative. In other words, a false positive would be expecting it to rain but it doesn't. A false negative is expecting it to not rain but it does.

What are the implications for me? Well a false positive means I end up carrying an umbrella I don't need, not such a big deal. But the false negative, which I always suffer from, means I don't bring my umbrella and I get soaked.

While this is a light-hearted example, type 1 and type 2 errors are essential considerations in a range of applications, including law, medicine, business and investing.

Gavel.png

High Standards of Courtroom Evidence

A new entertainment trend is the "true crime" story. The stories revisit high profile murder cases, presenting the timeline of events and usually implying guilt or innocence of the accused. Even the infamous Bain family murder case is being made into a mini-series.

Two popular examples are the dramatised recounting of the O.J. Simpson trial in The People V. O.J. Simpson and the story of Stephen Avery in Making a Murderer. Although these shows follow the same formula, the distinction is the first implies a guilty man went free and the second implies an innocent man was found guilty. A false negative and a false positive.

I certainly don't know enough to comment on these cases. However, they do illustrate a key underpinning of the legal process and our society's ethics as a whole.

The idea of innocent people being found guilty is considered a higher cost than the possibility of a guilty person being found not-guilty. Hence the high burden of proof in the legal system, requiring evidence to support the guilty verdict "beyond a reasonable doubt".

While many believe the justice system fails in these high profile cases, the high standard exists to stop the "false positives", where innocent people are declared guilty.

Medicine.jpg

Reliability of Medical Testing

Imagine you are a researcher producing cheap tests for a certain cancer. You have produced two methods; method one produces false positives 5% of the time and method two produces false negatives 5% of the time. What are the implications?

If a patient uses the first method and is incorrectly given a positive result, it would be nerve-wracking for them, but they would continue with the more expensive yet accurate testing. Sure, more costs are incurred, but the patient receives the correct diagnosis eventually.

More dangerous is the second method, where a false negative is produced. The patient in this example would be incorrectly diagnosed as healthy and sent home, which is much worse.

Similar to this example is how NZ police use breathalysers to keep drunk drivers off the road. The handheld versions aren't very accurate, often giving false positives for BAC being over the legal limit. These tests are followed by more rigorous methods if they are to be submitted as evidence.

 
Binoculars Guy.jpg
 

Sensible Investing Decisions

How does the idea of type 1 and type 2 errors apply to investing? It is all about how they add to the decision making process.

One idea commonly held among NZ investors is that stock picking is more effective here than overseas. The reasoning is we have a smaller market where pricing is less efficient and savvy investors can take advantage of this.

This has been largely disproven by Professor Bart Frijns of AUT, in KiwiSaver funds at least. But lets say you are considering an active strategy due to this idea.

First the false positive; we've taken the word of the stock picker and (as the evidence suggests) they are wrong. What are the consequences?

Professor Ken French, one of the academics who pioneered evidence-based investing, published a paper showing US investors incur an additional 0.67% of fees each year using active strategies over passive. The consequence is a reduction in expected returns of 0.67% p.a.

Now the false negative; we decide to forego stock picking in favour of indexing. We're assuming what is true in every other developed market is also true in NZ. But in this hypothetical, we are wrong. What are the consequences here?

The stock picker has already started with a handicap of 0.67% each year in fees. They would need to outperform by at least this amount to have added net value. If they were able to consistently add 1% of value, a very impressive feat, they would still only be 0.33% above the passive strategy.

Simply put, being wrong about indexing is likely to have smaller consequences than being wrong about stock picking. Given all the evidence is in favour of passive strategies, why place faith in the arguments for stock picking?

Evidence-based investing reduces the chances of type 1 and type 2 errors by relying on the highest standard of evidence, peer-reviewed research. We only include strategies in portfolios which are verified through the scientific process, all with the goal of improving investment outcomes for clients.