99% Accurate, and Mostly Wrong: Why Rare Things Break Good Detectors

A screening test comes back positive. The leaflet says the test is 99% accurate. How worried should you be?

For a lot of real tests pointed at a whole population, the honest answer is: much less than you think. The 99% is true. It is also nearly irrelevant to the question you just asked — and the gap between those two things is one of the most expensive misunderstandings there is, in medicine, in fraud detection, and in every machine-learning system built to find something rare.

The number you were handed answers “of all the calls this test made, how many were right?”

The question you asked is “of the people it flagged, how many are like me?”

Those are different questions. When the thing being looked for is rare, their answers are nowhere near each other.

Fix the detector

You run the screening programme. Below are the only two things you can change about it.

Most people spend the first dial first. It is the one that feels like engineering.

The counts

POP = 10_000
SENSITIVITY = 90          # the test finds 90% of the people who really have it

GROUPS = [
    ("everybody",                  1_000),   # parts per million, so 1 in 1,000
    ("adults over fifty",          5_000),   # 1 in 200
    ("people with the symptom",   20_000),   # 1 in 50
    ("people a doctor suspects", 100_000),   # 1 in 10
]
FALSE_ALARM = [50_000, 20_000, 10_000, 5_000]     # 5%, 2%, 1%, 0.5%


def screen(prevalence_ppm, false_alarm_ppm):
    """Everything here is integer counts of people, and rates are carried in parts per million so
    that every division is exact. Nothing below touches floating point, which is why these numbers
    are identical on any machine and why the widget above reproduces them exactly."""
    sick = POP * prevalence_ppm // 1_000_000
    well = POP - sick
    caught = sick * SENSITIVITY // 100
    false_alarms = well * false_alarm_ppm // 1_000_000
    return sick, well, caught, sick - caught, false_alarms


def hit_rate(caught, flagged):
    return caught * 100 // flagged            # floored, so the number is never flattering


def one_dp(permille):
    return f"{permille // 10}.{permille % 10}%"     # 999 -> '99.9%', still no floats


def as_pct(ppm):
    return one_dp(ppm // 1_000)


LEFT = 22
sick, well, caught, missed, false_alarms = screen(1_000, 10_000)
flagged = caught + false_alarms
rate = hit_rate(caught, flagged)

print(f"{POP:,} people. The test finds {SENSITIVITY}% of real cases and false-alarms on 1%.")
print()
print("Screening everybody, where 1 person in 1,000 has it:")
print(f"{sick:>7,} have it".ljust(LEFT) + f"-> {caught:>4,} flagged, {missed:>2,} missed")
print(f"{well:>7,} do not".ljust(LEFT) + f"-> {false_alarms:>4,} flagged anyway")
print(" " * LEFT + "   " + "-" * 4)
print(" " * LEFT + f"   {flagged:>4,} positive results, {caught} of them real")
print()
print(f"So a positive result is real {rate} times in 100.")
print(f"{100 - rate}% of the people this test alarms about are fine.")
print()

right = caught + (well - false_alarms)
print(f"Overall accuracy of that test: {right:,} of {POP:,} calls correct, {one_dp(right * 1_000 // POP)}.")
print(f"Accuracy of answering 'no' to everyone: {well:,} of {POP:,}, {one_dp(well * 1_000 // POP)}.")
print("The useless detector scores higher than the real one.")
print()
print()
print(f"{'':<26}{'in the':>10}   how often the test false-alarms")
print(f"{'who you point it at':<26}{'group':>10}" + "".join(f"{as_pct(fa):>9}" for fa in FALSE_ALARM))
print("-" * 72)
for name, prev in GROUPS:
    row = f"{name:<26}{'1 in ' + format(1_000_000 // prev, ','):>10}"
    for fa in FALSE_ALARM:
        _, _, c, _, f = screen(prev, fa)
        row += f"{str(hit_rate(c, c + f)) + '%':>9}"
    print(row)
print()
print("Each cell: of everyone the test flagged, how many actually have it.")
print()

_, _, c, _, f = screen(GROUPS[0][1], FALSE_ALARM[0])
worst = hit_rate(c, c + f)
_, _, c, _, f = screen(GROUPS[0][1], FALSE_ALARM[-1])
best_test = hit_rate(c, c + f)
_, _, c, _, f = screen(GROUPS[-1][1], FALSE_ALARM[0])
best_group = hit_rate(c, c + f)
print(f"Along the top row, making the test ten times better: {worst}% -> {best_test}%.")
print(f"Down the first column, same test, different people: {worst}% -> {best_group}%.")

10,000 people. The test finds 90% of real cases and false-alarms on 1%.

Screening everybody, where 1 person in 1,000 has it:
     10 have it       ->    9 flagged,  1 missed
  9,990 do not        ->   99 flagged anyway
                         ----
                          108 positive results, 9 of them real

So a positive result is real 8 times in 100.
92% of the people this test alarms about are fine.

Overall accuracy of that test: 9,900 of 10,000 calls correct, 99.0%.
Accuracy of answering 'no' to everyone: 9,990 of 10,000, 99.9%.
The useless detector scores higher than the real one.


                              in the   how often the test false-alarms
who you point it at            group     5.0%     2.0%     1.0%     0.5%
------------------------------------------------------------------------
everybody                 1 in 1,000       1%       4%       8%      15%
adults over fifty           1 in 200       8%      18%      31%      47%
people with the symptom      1 in 50      26%      47%      64%      78%
people a doctor suspects     1 in 10      66%      83%      90%      95%

Each cell: of everyone the test flagged, how many actually have it.

Along the top row, making the test ten times better: 1% -> 15%.
Down the first column, same test, different people: 1% -> 66%.

Why the obvious dial cannot win

Read the table twice, in two directions.

Along the top row, the test gets ten times better — from a sloppy 5% false-alarm rate down to 0.5%, about as good as real screening tests get — and the answer moves from 1% to 15%. You have made the instrument nearly perfect and a positive result is still wrong five times out of six.

Down the first column, the test never changes. It stays at its worst setting the whole way. Only the people it is pointed at change, and the answer moves from 1% to 66%.

That asymmetry is the subject. A test does not have a trustworthiness. It has one per population, and the population is usually the cheaper thing to change.

The arithmetic behind it is not subtle once it is laid out. Screening everybody, there are 10 real cases and 9,990 healthy people. A 1% false-alarm rate applied to 9,990 people produces 99 false alarms — ten times more than there are real cases in the entire room. The test is not doing anything wrong. There are simply so many more chances to be wrong than to be right that even a small error rate, multiplied by the larger group, buries the answer.

This is why screening guidelines are gated by age and risk factor rather than offered to everyone. It is not rationing. Pointing a test at a population where the answer is nearly always no makes its positives nearly always wrong.

The detector that wins by doing nothing

The program prints one more comparison, and it is the one worth keeping.

That test — 90% of real cases caught, 1% false alarms — gets 99.0% of its calls right. A detector that ignores its input entirely and answers no to every single person gets 99.9%.

The do-nothing detector scores higher. It finds nothing, helps nobody, and beats a real instrument on the metric, because 999 of every 1,000 answers really are no and it gets all of them right.

Any time an accuracy figure is attached to a rare event, that is the number it is competing with. Which is why the useful measures are stated as a pair: precision — of the things you flagged, how many were real, the column this page has been computing all along — and recall — of the real things, how many did you catch. Neither can be gamed by a detector that abstains, and neither is reported by an accuracy score.

The same shape, everywhere something is rare

• Fraud and abuse. A model tuned on all traffic, where fraud is 1 in 1,000, hands its reviewers a queue that is mostly innocent customers. The reviewers learn the queue is mostly noise and start clearing it fast — which is the failure, and it was caused by where the model was pointed.
• Security alerting. Alert fatigue is this table. A detector firing on a genuinely rare event produces mostly false alarms however good it is, and a team that has dismissed a hundred in a row will dismiss the hundred and first.
• Filtering a large applicant pool. A filter with a small error rate, applied to thousands, still rejects real candidates in numbers nobody counts — because rejections are not audited.
• Rare failures in a model’s own evaluation. A benchmark where the interesting behaviour appears in 1 test out of 500 will be dominated by the 499, and a headline accuracy will say almost nothing about the case anybody cared about.

The fix is the one the second dial makes: narrow the population before you apply the test. Triage first, then screen. A cheap filter that raises the rate from 1 in 1,000 to 1 in 50 does more for the final answer than any plausible improvement to the detector itself.

The bill from the last page

The overfitting page ended on a caveat it did not pay off: with rare positives, accuracy is actively misleading — a detector that always says no scores 99% on a 1-in-100 problem. This page is that debt settled, with the counts printed.

The two mistakes are the same mistake in different clothes. Both are a single number going up while the thing you actually wanted quietly goes the other way, and in both cases the defence is not a better model but a better question.

What this page simplifies

• One test, fixed sensitivity. Real detectors trade sensitivity against false alarms rather than improving both, and moving that trade-off is a third dial this page does not offer. It would not rescue the low-prevalence case anyway: catching more of 10 real cases cannot outrun 99 false alarms.
• The false-alarm dial stops at 0.5%. That is roughly where real screening tests stop. A test with a false-alarm rate below about 0.09% would clear 50% even when screening everybody — the base rate makes the target hard, not mathematically unreachable.
• Prevalence is treated as a choice. In practice it comes from prior studies, carries its own uncertainty, and deciding who to screen has costs and equity consequences well outside arithmetic.
• Whole numbers of people. Every division is floored, so a few cells sit a percentage point below their exact value. That is deliberate: floored counts are reproducible everywhere and never round in the test’s favour.
• No cost asymmetry. A missed case and a false alarm are counted alike here. They almost never cost the same, and which one you would rather have is the decision that sets the dial in real life.