Historical Context & Motivation
For centuries, scientists drew conclusions from uncontrolled observations—comparing outcomes without systematically ruling out alternative explanations. Agricultural yields, medical treatments, and industrial processes were evaluated anecdotally, leaving results vulnerable to hidden biases that today we call confounding variables. The question of how to design an experiment so that observed differences can be attributed to a treatment, rather than to lurking factors, drove the development of modern experimental design. This intellectual thread runs from early agricultural field trials to the randomized clinical trials that underpin modern medicine and policy evaluation.
The central question that unites these milestones is deceptively simple: How should experimental units be assigned to treatments so that the resulting data support valid causal conclusions? Different designs answer this question in different ways, each balancing simplicity, control of variability, and practical constraints. Understanding when to use a completely randomized design, a randomized block design, or a matched-pairs design is a core competency in AP Statistics and a skill that extends well beyond the exam.
Core Principles of Experimental Design
Before choosing a specific layout for an experiment, you must internalize the three foundational principles that Fisher established. Every well-designed experiment incorporates comparison through control, random assignment, and replication. A fourth principle—blocking—enters when researchers have advance knowledge about a source of variability that can be accounted for structurally. These ideas are not mutually exclusive; rather, they layer upon one another to strengthen the evidence an experiment can provide.
Control / Comparison
Random Assignment
Replication
Blocking
Visual Overview of Three Experimental Designs
The diagram below illustrates the structural differences among the three experimental designs you need to know for AP Statistics: the completely randomized design (CRD), the randomized block design (RBD), and the matched-pairs design. Pay attention to where the random assignment step occurs in each layout and how subjects flow from the pool of experimental units to the comparison of responses.
Notice that the key structural difference lies in what happens before randomization. In a CRD, nothing—subjects go directly to the randomization step. In an RBD, subjects are first sorted into homogeneous blocks. In matched pairs, subjects are explicitly paired so that within-pair differences reflect only the treatment effect plus random noise, not the blocking variable. The choice among these designs depends on whether you can identify a known source of variability and whether the practical constraints of the study permit blocking.
How Each Design Works
Completely Randomized Design (CRD)
In a completely randomized design, every experimental unit has an equal probability of being assigned to any treatment group. This is the simplest design and is appropriate when the researcher has no prior information about sources of variation among units, or when units are fairly homogeneous. A random number generator or a table of random digits assigns each unit to a group, and the resulting treatment groups are compared directly.
The major advantage of the CRD is its simplicity: no blocking structure must be determined in advance, and the analysis (typically a two-sample t-test or one-way ANOVA) is straightforward. The disadvantage is that if a lurking variable creates substantial variation among units, the randomization may not perfectly balance groups—especially with small sample sizes—reducing the experiment's power to detect real effects.
Randomized Block Design (RBD)
A randomized block design first groups experimental units into blocks that are homogeneous with respect to some known source of variation, then randomly assigns treatments within each block. For example, if you are testing two fertilizers on crop yield and know that field position (sunny vs. shady) affects growth, you would create blocks of plots that share similar sunlight and randomize treatments within each block. By accounting for the block-to-block variation, the RBD reduces the experimental error and increases the power to detect the treatment effect.
Matched-Pairs Design
A matched-pairs design is a special case of the randomized block design in which each block contains exactly two experimental units (or one unit measured under two conditions). The two most common forms are: (1) pairing two subjects who are similar on a key characteristic and randomly assigning one to each treatment, and (2) having each subject serve as their own control by receiving both treatments in a randomly determined order (a crossover design). The analysis focuses on the within-pair differences rather than on individual responses, which typically reduces variability dramatically.
Matched pairs are especially powerful when individual variability is large relative to the treatment effect. By looking at how each pair's responses differ, you effectively remove the pair-to-pair variation from the analysis. The cost is logistical: you must be able to identify appropriate pairs or ensure that carry-over effects do not contaminate a crossover design.
Decision Framework: Choosing the Right Design
Selecting an experimental design is a decision that depends on the researcher's knowledge of the experimental units, the number of treatments, and practical constraints. The flowchart below provides a systematic decision framework. Note that the AP exam may present a scenario and ask you to identify the most appropriate design, or it may describe a design and ask you to justify it. Either way, you must articulate the reasoning behind the choice.
| Feature | CRD | RBD | Matched Pairs |
|---|---|---|---|
| Pre-grouping? | None | Blocks of ≥2 units based on a known variable | Pairs of 2 or 1 subject under 2 conditions |
| Randomization | All units randomly assigned to treatments | Within each block | Within each pair |
| Best when | Units are relatively homogeneous; no known confound | A known confound can be identified and used to create blocks | Individual-level variation is large; only 2 treatments |
| Analysis | Two-sample t-test or ANOVA | ANOVA with block as a factor | One-sample t-test on paired differences |
| AP Exam focus | Describe random assignment procedure | Identify blocking variable and justify | Explain how pairs are formed and why |
Worked Example: Designing an Experiment
A nutritionist wants to determine whether a new protein supplement increases muscle mass more than a standard supplement over an eight-week period. She recruits 40 volunteers from a gym—20 are experienced weightlifters and 20 are beginners. She suspects that fitness level affects muscle gain independently of the supplement. Design an experiment to test the supplement's effectiveness.
Strengths and Limitations of Each Design
No single design is universally superior; each involves trade-offs between simplicity, statistical efficiency, and practical feasibility. The table below summarizes the key strengths and limitations you should be prepared to discuss on the AP exam. When a free-response question asks you to justify a design, referencing both its strengths and the limitations of alternatives demonstrates a sophisticated understanding.
| Design | Strengths | Limitations |
|---|---|---|
| Completely Randomized (CRD) | Simplest to implement; no advance knowledge of variability needed; flexible for any number of treatments; straightforward analysis | Uncontrolled variability inflates experimental error; less powerful when units are heterogeneous; with small n, groups may be unbalanced on key characteristics by chance |
| Randomized Block (RBD) | Accounts for known source of variation; reduces error variability; more powerful than CRD when blocks are truly homogeneous; can handle multiple treatments | Requires advance knowledge of a blocking variable; ineffective if blocks are poorly chosen (wrong variable or insufficient homogeneity); more complex analysis; loses degrees of freedom to blocks |
| Matched Pairs | Maximally controls individual-level variation; very powerful when subject-to-subject differences dominate; simple paired-difference analysis | Limited to exactly two treatments; finding matched pairs can be difficult or subjective; crossover designs risk carry-over effects; loss of one subject eliminates the entire pair |
Connections to Advanced Concepts
The three designs covered in AP Statistics—CRD, RBD, and matched pairs—are entry points into a much richer landscape of experimental design methodology studied in college-level courses and professional research. Understanding how these foundational designs connect to more advanced concepts deepens your appreciation of why design matters and helps you answer AP questions that probe the reasoning behind experimental structure.
| AP-Level Concept | Advanced Extension | Connection |
|---|---|---|
| Completely Randomized Design | Factorial Experiments (2 × 2, 2 × 3, etc.) | CRDs can test multiple factors simultaneously, examining both main effects and interaction effects |
| Randomized Block Design | Latin Square and Split-Plot Designs | Blocks can be extended to control for two sources of variation simultaneously or to handle constraints where some factors must be applied at different levels of hierarchy |
| Matched-Pairs Design | Crossover Designs and Repeated Measures | When each subject receives both treatments in sequence, time-related effects and wash-out periods become design considerations |
| Random Assignment (all designs) | Randomization Tests and Permutation Inference | The act of randomization itself provides the mathematical justification for inference without distributional assumptions |
For the AP exam, you do not need to implement factorial or Latin square designs, but understanding that blocking and randomization are scalable principles will help you reason about unfamiliar scenarios. The AP free-response section sometimes presents multi-factor experiments or unusual blocking structures, and students who understand the underlying logic—rather than just memorizing three design names—are best positioned to earn full credit.
Practice Problems
Summary & Review
Selecting an experimental design is the process of choosing the structural framework that determines how experimental units are assigned to treatments. The three designs tested on the AP Statistics exam are the completely randomized design (CRD), which randomly assigns all units to treatments without pre-grouping; the randomized block design (RBD), which first sorts units into homogeneous blocks based on a known source of variability and then randomizes within each block; and the matched-pairs design, a special case of blocking where each block contains exactly two units or one unit is measured under both conditions.
All three designs rest on the foundational principles of control (a comparison group ensures a baseline), random assignment (a chance mechanism balances known and unknown confounders, enabling causal conclusions), and replication (enough units per treatment to detect genuine effects). When choosing a design, ask: Is there a known source of variability? If yes, block on it. Can blocks have exactly two units? If yes, use matched pairs. If no known variability exists or blocking is impractical, the CRD remains a valid and powerful choice. On the AP exam, always justify your design by explaining what variability the design controls and why random assignment is necessary.