Normality testing asks a simple question: does your data follow a bell-shaped curve? You can answer it two ways. Visual methods like histograms and Q-Q plots let you see the shape with your own eyes. Statistical methods like the Shapiro-Wilk test give you a number and a p-value. Most analysts use both, because each catches problems the other misses.
Why Does Normality Matter In The First Place?
Many statistical tests assume your data comes from a normal distribution. The t-test, ANOVA, and linear regression all lean on that assumption to varying degrees. When the assumption holds, the math works cleanly. When it fails badly, your p-values and confidence intervals can be wrong.
The normal distribution is a specific mathematical shape. It is symmetric, bell-shaped, and defined entirely by two numbers: the mean and the standard deviation. About 68% of values fall within one standard deviation of the mean. About 95% fall within two. Those percentages are fixed properties of the distribution, not rules of thumb.
Here is where people often go wrong. Normality of the data is not always required. What many tests actually assume is normality of the residuals or of the sampling distribution of the mean. Thanks to the central limit theorem, the sampling distribution tends toward normal as sample size grows, even when the underlying data is skewed. So a t-test on a few hundred observations can behave well even with moderately non-normal data. A t-test on twelve observations cannot make that claim.
That distinction matters because it changes what you are testing. You are not always checking whether the raw data is normal. Sometimes you are checking whether the residuals from a model are normal. Those are different questions with different answers.
How Do You Test Normality Visually?
Visual methods are fast, intuitive, and surprisingly sensitive once you know what to look for. They also require judgment, which is both their strength and their weakness.
Histogram
A histogram bins your data and shows how often values fall in each bin. A normal distribution produces a roughly symmetric bell. Problems show up as skew (a long tail on one side), multiple peaks (suggesting mixed populations), or heavy tails (more extreme values than expected).
The catch is bin width. Change the number of bins and the same data can look normal or lumpy. Always try a few bin widths before drawing conclusions. A histogram with too few bins hides structure. Too many bins creates noise that looks like structure.
Q-Q Plot
A quantile-quantile plot is the workhorse of visual normality checking. It plots your sorted data against the values you would expect from a perfect normal distribution. If the data is normal, the points fall along a straight diagonal line.
Deviations have meaning. Points curving up at the right end suggest a right-skewed distribution. An S-shaped curve suggests heavy tails. A few points far off the line at either end suggest outliers. The Q-Q plot is generally more informative than a histogram because it does not depend on arbitrary bin choices.
Other Visual Tools
- Box plot: shows median, quartiles, and outliers. A normal distribution gives a symmetric box with whiskers of similar length.
- Stem-and-leaf plot: useful for small datasets where a histogram would be too coarse.
- Density plot: a smoothed version of the histogram, less sensitive to bin width but still affected by the smoothing parameter.
Visual methods share one weakness. They are subjective. Two people can look at the same Q-Q plot and disagree about whether the deviation matters. That is why visual inspection is usually paired with a formal test.
How Do You Test Normality Statistically?
Statistical tests give you a p-value. The null hypothesis is always the same: the data comes from a normal distribution. A small p-value is evidence against normality. A large p-value means you failed to find evidence against it.
That last point trips up a lot of people. Failing to reject normality is not the same as proving normality. With a small sample, these tests have low power and will often fail to detect real departures from normal. With a large sample, they can detect trivial departures that do not matter in practice.
Shapiro-Wilk Test
The Shapiro-Wilk test is widely regarded as one of the most powerful general-purpose tests for normality, especially for small to moderate samples. It compares the ordered data to the values expected under normality using a weighted correlation approach. Many statistical software packages treat it as the default.
Anderson-Darling Test
The Anderson-Darling test gives more weight to the tails of the distribution than some alternatives. If your concern is extreme values, this test is often a good choice. It is available in most statistical software.
Kolmogorov-Smirnov Test
The Kolmogorov-Smirnov test compares your data’s cumulative distribution to a normal one. It is less powerful than Shapiro-Wilk for detecting normality departures, partly because it is a general test not specifically tuned for normality. It also performs poorly when you estimate the mean and standard deviation from the same data, which is almost always the case.
D’Agostino-Pearson Test
This test combines measures of skewness and kurtosis into a single statistic. It works well for larger samples and gives you information about which aspect of normality is violated, not just whether it is.
Jarque-Bera Test
The Jarque-Bera test is also based on skewness and kurtosis. It is common in econometrics and works best with large samples. In small samples it has low power.
How Do You Choose Between Visual And Statistical Methods?
You do not have to choose. The standard practice is to use both and let them inform each other.
Start with a Q-Q plot and a histogram. They tell you the shape of the problem, if there is one. Then run a formal test to get an objective measure. If the visual and statistical results disagree, dig deeper before deciding.
Sample size changes everything here. With fewer than about 30 observations, formal tests have little power, so visual inspection carries more weight. With hundreds or thousands of observations, formal tests will flag almost any deviation, even ones too small to affect your analysis. In that case, look at the effect size of the deviation, not just the p-value.
Consider what your downstream analysis actually needs. A small departure from normality rarely matters for a t-test with a decent sample. It matters much more for procedures that rely heavily on the tails of the distribution.
What Should You Do When Data Is Not Normal?
Non-normal data is not a dead end. There are several established approaches, and the right one depends on the source of the non-normality and what you plan to do next.
- Transform the data: a log transformation can reduce right skew. A square root transformation can help with count data. These are common and well established.
- Use a non-parametric test: the Mann-Whitney U test replaces the t-test. The Kruskal-Wallis test replaces one-way ANOVA. These do not assume normality.
- Rely on the central limit theorem: with a large enough sample, many tests are robust to moderate non-normality. This is a judgment call, not a blanket rule.
- Use robust methods: some modern statistical methods are designed to perform well even when normality fails.
- Investigate the cause: sometimes non-normality signals a data problem, a mixed population, or a variable that should be modeled differently.
One non-obvious point: outliers are often the reason a normality test fails. Removing them without justification is bad practice. But understanding whether they are data errors, natural extreme values, or something else is essential before deciding what to do.
What Are The Common Mistakes In Normality Testing?
The most common mistake is treating the p-value as a pass/fail grade. A p-value of 0.06 does not mean the data is normal. It means you did not find strong evidence against normality. A p-value of 0.04 does not mean your analysis is ruined.
Another mistake is testing the wrong thing. As noted earlier, many analyses assume normality of residuals, not of the raw data. Testing the raw data when you should be testing residuals gives you a misleading answer.
A third mistake is ignoring sample size. The same deviation from normality will produce a tiny p-value in a sample of 5,000 and a large p-value in a sample of 20. The p-value alone tells you nothing about whether the deviation matters.
Finally, some people skip visual inspection entirely and rely only on a test. That throws away information. A Q-Q plot can reveal patterns — like bimodality or a few extreme outliers — that a single p-value will never show you.
Frequently Asked Questions
What is the best statistical test for normality?
The Shapiro-Wilk test is generally considered the most powerful for small to moderate samples. For larger samples, the Anderson-Darling test is often preferred because it gives more weight to the tails.
Can I rely only on a Q-Q plot to check normality?
You can, but visual judgment is subjective and two people may read the same plot differently. Pairing a Q-Q plot with a formal test gives a more complete picture.
What does a non-significant normality test mean?
It means you did not find statistical evidence against normality. It does not prove the data is normal, especially in small samples where these tests have low power.
Should I test the raw data or the residuals for normality?
It depends on your analysis. Regression and ANOVA assume normality of residuals, not of the raw data. Testing the wrong one can lead you to the wrong conclusion.
