What a before-and-after study without a control group cannot show

A school introduces a new reading program and test scores rise the next semester. The timing is consistent with the program helping, but it does not prove the program caused the increase. Without a comparison group, it is difficult to separate the program’s effect from other changes that happened at the same time.

What a before-and-after design measures

The study measures the same group before and after an intervention and compares the outcomes. The observed change is a real description of the two measurements, assuming the data were collected consistently. The causal interpretation is harder because the study does not show what would have happened to the same group over the same period without the intervention.

Alternative explanations

• Maturation: participants improve or change naturally over time.
• History: an outside event affects the outcome during the study.
• Regression to the mean: unusually high or low initial results move closer to typical levels on a later measurement.
• Instrumentation: the test or measurement method changes between time points.
• Selection: the group differs in a way related to the outcome or intervention.
• Attrition: people who leave the study differ from those who remain.

How a comparison group helps

A control or comparison group measured over the same period can help account for changes that would have occurred anyway. Random assignment, when feasible and ethical, improves comparability at the start. A well-designed observational comparison can also be useful, but researchers must address confounding and explain the limits of causal conclusions.

Use precise conclusions

For a test question, distinguish “scores increased after the program” from “the program increased scores.” The first reports an association over time; the second makes a causal claim that requires stronger evidence. Look for a comparison group, random assignment, consistent measurement, and consideration of other variables before accepting the causal conclusion.

Key takeaway

A before-and-after change without a control group leaves competing explanations uncontrolled. It can suggest a relationship, but by itself it does not establish cause and effect.

Why timing alone does not establish cause

Suppose students take a practice assessment, complete a study program, and then score higher on a second assessment. The improvement is real as a before-and-after difference, but the design alone cannot show that the program caused it. Other events occurred during the same period: students may have practiced independently, learned the material in class, received tutoring, or become familiar with the test format. Any of these could contribute to the higher score.

The missing comparison is what would have happened to similar students over the same period without the program. Because the same students cannot simultaneously receive and not receive the intervention, researchers estimate that counterfactual with a comparison group. Without one, the observed change combines the program’s possible effect with maturation, history, testing effects, regression to the mean, and changes in who completed the second test.

Recognize common alternative explanations

Maturation means participants change naturally with time. Younger children may improve on a developmental measure even without a new intervention. History refers to outside events that occur between measurements, such as a new curriculum or a community emergency. A testing effect occurs when taking the pretest itself changes later performance: students may remember questions or learn what to review. These explanations are especially important when the interval is long or the measure is familiar.

Regression to the mean matters when participants are selected because of unusually low or high initial results. Extreme measurements often move closer to a person’s typical level on a later measurement, partly because random variation is less likely to be extreme twice. If a support program recruits only students with exceptionally low test scores, a later average increase may occur even if the program has no effect. The design needs an appropriate comparison group selected using the same criterion.

What stronger designs add

A randomized controlled trial assigns eligible participants by chance to an intervention or comparison condition. Random assignment tends to balance known and unknown confounders across groups, although a small sample can still have imbalances. If randomization is not possible, a matched comparison group, repeated measurements, or a carefully designed interrupted time series can improve the evidence. Each approach has assumptions; matching cannot remove confounders that were not measured.

When interpreting a report, ask whether the groups were comparable before the intervention, whether they were measured at the same times, how many participants were lost to follow-up, and whether the outcome was defined in advance. A large pre/post change with heavy attrition may describe only participants who remained. A small difference with a well-designed comparison may be more informative about cause than a dramatic uncontrolled change.

• A pre/post difference describes change, not necessarily its cause.
• Identify what else changed between measurements.
• Selection on an unusually extreme score raises regression-to-the-mean concerns.
• A comparison group estimates the change that might have occurred without treatment.
• Check random assignment, attrition, measurement timing, and baseline comparability.

Check who completed both measurements

Attrition occurs when participants drop out before the second measurement. If people with poor outcomes are more likely to leave, the average among those who remain can improve even if the intervention had little effect. Compare the number and characteristics of participants measured at each time, and ask whether missing results were handled transparently.

Measurement can also change between the two observations. A new test may be easier, a scoring rubric may be applied differently, or participants may know what outcome is expected. A valid before-and-after report should describe the measure and keep procedures comparable; otherwise the observed difference may partly reflect the measuring process.

Apply the distinction carefully

A before-and-after comparison shows that an outcome changed after an intervention, but it cannot by itself show that the intervention caused the change. Other factors may have shifted during the same period: participants gained experience, the measurement changed, or an outside event affected performance. A comparison group measured over the same period can help estimate those background changes, especially when assignment makes the groups comparable. Even then, the design and measurement matter. When reading a study, separate the observed result from the causal interpretation and note what alternative explanations the method can rule out. The cautious conclusion is that the outcome improved after the program; the stronger claim that the program produced the improvement requires supporting design evidence.