Scenario:
An agricultural researcher wants to compare the effectiveness of three different fertilizers (Fertilizer A, B, and C) on crop yield. The researcher applies each fertilizer to 10 different fields and measures the crop yield after the growing season.
Task:
Determine whether the mean crop yield significantly differs among the fertilizers.
T-test
A t-test is a statistical test to determine if there is any significant difference between the averages or means of two groups.
The three primary types of t-tests are:
Independent (Unpaired) t-test: Evaluates the difference between the means of two separate, unrelated groups.
Paired t-test: Assesses the mean difference within the same group measured at different times, such as before and after a treatment.
One-sample t-test: Compares the mean of a single group to a known reference value or population mean.
When t-test start to stand for (t)edious-tests
If you really want to stick to t-test — keep in mind this means that you can only compare 2 substances at once. You need to compare
A & B
A & C
A & D
A & E
B & C
B & D
B & E
C & D
C & E
D & E
That’s 10 separate t-tests — and I’m tired just typing this out!
Rather than comparing these substances 2 at a time (pairwise fashion) — is there any better way where we can compare them all at once?
However, this isn’t just a matter of brute force — and to understand why, we go to the next reason why conducting multiple t-tests is no good.
Inflated Type 1 Error
You remember that when we do a t-test — we always compare our p-value to an α value? α is — it represents type 1 error — the chance of false positive that we are willing to accept.
Assuming the null hypothesis is true, using a 5% significance level makes us wrong 5% of the time. Out of 100 significant t-tests done under the assumption that H₀ is true, 5 OF THEM ARE WRONG.
We set α at 5% because that’s the level of uncertainty we’re willing to accept.
When you choose to compare groups by conducting multiple t-tests — you are basically increasing the chance of you getting false positive results drastically.
“Each added t-test widens the doorway for chance, allowing false positives to quietly wear the mask of discovery.”
 |
ANOVA: The Solution
Which is why ANOVA was even invented in the first place. It is a straightforward method to compare ALL groups at once — while keeping your type 1 error rate constant at 0.05.
“ANOVA turns many comparisons into one clear decision, keeping statistical error fixed even as the number of groups expands.”
Best part is that doesn’t increase in complexity as the number of groups increases — so no matter how many groups you want to compare, you just need to do the ANOVA once.
You conduct your ANOVA and fail to reject H0 (no evidence of difference between the means). You smugly smile, happy to prove your boss wrong.