Bayes theorem and how we talk about medical tests

We want medical tests to give us a yes or no answer: you have the disease, you're cured. We treat them this way, often. My labs came back saying I'm healthy. I have immunity. I'm sick. Absolutely concrete results.

The reality is more complicated, and tests do not give you a yes or no. They give you a likelihood. And most of the time, what the results mean for me, the test taker, is not immediately obvious or intuitive. They can mean something quite the opposite of what they seem.

I ran into this recently on a page about celiac disease.

* * *

The Celiac Disease Foundation has a page about testing for celiac disease. On this page, they give a lot of useful information about what different tests are available, and they point to some other good resources as well. In the section about one of the tests, it says (emphasis original):

The tTG-IgA test will be positive in about 93% of patients with celiac disease who are on a gluten-containing diet. This refers to the test's sensitivity, which measures how correctly it identifies those with the disease. The same test will come back negative in about 96% of healthy people without celiac disease. This is the test's specificity.

This is great information, and it tells you what you need to start figuring out what your chance of celiac disease is. The next paragraph says this, however:

There is also a slight risk of a false positive test result, especially for people with associated autoimmune disorders like type 1 diabetes, autoimmune liver disease, Hashimoto's thyroiditis, psoriatic or rheumatoid arthritis, and heart failure, who do not have celiac disease.

And this is where things are a little misleading. It says that there is a "slight risk" of a false positive test result. What do you think of as a slight risk? For me, it's maybe somewhere around 5%, maybe 10%. The truth is, the risk of a false positive is much higher (under many circumstances).

* * *

When I take a test, I want to know a couple of things. If I get a positive test result, how likely is it that I have the disease? If I get a negative test result, how likely is it that I do not have the disease? The rates of positive and negative results listed above, the sensitivity and specificity, do not tell us these directly. However, they let us to calculate this with a little more information.

Bayes' theorem says that P(A|B) = P(B|A) * P(A) / P(B). You can read P(A|B) as "the probability of A conditioned on B", or the chance that A happens if we know that B happens. What this formula lets us do is figure out one conditional probability we don't yet know in terms of other ones that we do know.

In our case, we would say that A is having celiac disease, and B is getting a positive test result. This leaves P(A|B) as the chance that if you get a positive test result, that you do have celiac disease, which is exactly what we want to know. To compute this, we need a few more pieces of information.

• P(A), the probability in general that the person taking the test has celiac disease. This is also called the prior probability, as it's what we would say the probability is if we did not know anything from this computation and test.
• P(B), the probability that for any given test taken, it comes back positive.
• P(B|A), the probability that if one has celiac disease, the test will come back positive.

We already know that P(B|A) is 0.93, as we were told this above. And we can find P(A) prety easily. Let's say P(A) is 0.01, since about 1 in 100 people in the US have celiac disease. Estimates vary from 1 in 200 to 1 in 50, but this will do fine.

That leaves us with P(B). We have to compute it from both possibilities. If someone who has celiac disease takes the test, they have a 93% chance of it coming back positive, but they're only 1% of the population. On the other hand, someone without celiac disease has a 4% chance of it coming back positive (96% of the time it gives a true negative), and they're 99% of the population. We use these together to find that P(B) = 0.93*0.01 + 0.04*0.99 = 0.0489.

Now we plug it all in! P(A|B) = P(B|A) * P(A) / P(B) = 0.93 * 0.01 / 0.0489 = 0.19. Neat, 19%!

* * *

Wait.

So this says that, if you get a positive test result, you have a 19% chance of having celiac disease? Yes, exactly! It's less than 1 in 5!

So if you get a positive test result, you have an 80% chance of it being a false positive. This is quite a bit higher than the aforementioned "slight risk." In fact, it means that the test doesn't so much diagnose you with celiac disease as say "huh, something's going on here" and strongly suggest further testing.

Now let's look at the test the other way around, too. How likely is it you don't have celiac disease if you get a negative test result?

Here we'd say that A is "we don't have it" and B is "we have a negative test". Doing some other calculations, pulled out of the oven fully baked in cooking show style, we can see that P(A | B) = 0.96 * 0.99 / 0.951 = 0.999. So if you get a negative test result, you have a 99.9% chance of not having the disease. This can effectively rule it out!

* * *

But... We know that 7% of people who take this test and do have celiac disease will get a negative result. How does this makes sense?

The truth is, things are a little bit deeper. People don't actually present with exactly a 1% chance of having celiac's disease. That would be true if you plucked a random person from the population and subjected them to a blood test. But it's not true if you go to your doctor with GI symptoms which are consistent with celiac disease! If you're being tested for celiac disease, you probably are symptomatic.

So that prior probability, P(A)? It's better as something else, but how we set it is a good question.

Let's say you present with symptoms highly consistent with celiac disease, and that this gives you a 10% chance of having celiac disease and a 90% chance of it being something else, given these symptoms. This changes the probability a lot. If you get a positive test in this case, then P(A|B) = 0.93 * 0.1 / (0.93 * 0.1 + 0.04 * 0.9) = 0.72. So now a positive test is a 72% chance of having celiac disease, instead of just 20%. And a negative test here gives you a 10% chance of a false negative, better than the 0.1% chance before.

The real question is how we go from symptoms to that prior probability accurately.

* * *

I spent a lot of 2024 being poked with needles and tested for various diseases while we tried to figure out what was wrong with me. Ultimately it was Lyme disease, and the diagnosis took a while because of a false negative. That false negative happened because the test was calibrated for broad population sampling, not for testing individuals presenting with symptoms already.

The whole story is a lot longer, and it's for another post. But maybe, just maybe, it would've been a shorter story if we'd learned reason about probabilities and medical tests better. Things are not intuitive, but Bayes is your friend, and Bayes' theorem can show us the information we really need to know.

Or, we can keep going with things how they are. I mean, I did enjoy getting to know Barbara, my phlebotomist, from all my appointments.