Sampling: With Replacement vs. Without Replacement

The core distinction

• With replacement: After each draw, the item goes back. The population is unchanged, so each draw has the same probabilities. Draws are independent.
• Without replacement: The drawn item stays out. The population shrinks, so the probability of the next draw shifts. Draws are dependent.

These two rules use the same "multiplication" idea, but the second factor changes.

Setup

Bag: 5 red, 3 blue — 8 marbles total. Let $R_1, R_2$ denote "draw is red on trial 1/2."

Scenario A — With replacement

Step 1 — First red. $P(R_1) = \dfrac{5}{8}$

Step 2 — Second red, marble returned. Still 5 red out of 8: $P(R_2 \mid R_1) = P(R_2) = \dfrac{5}{8}$

Step 3 — Multiply: $P(R_1 \cap R_2) = \dfrac{5}{8} \cdot \dfrac{5}{8} = \dfrac{25}{64} \approx \boxed{0.3906}$

Scenario B — Without replacement

Step 1 — First red. $P(R_1) = \dfrac{5}{8}$

Step 2 — Second red, first NOT returned. Now 4 red remain out of 7: $P(R_2 \mid R_1) = \dfrac{4}{7}$

Step 3 — Multiply: $P(R_1 \cap R_2) = \dfrac{5}{8} \cdot \dfrac{4}{7} = \dfrac{20}{56} = \dfrac{5}{14} \approx \boxed{0.3571}$

Side-by-side comparison

• With replacement: $\approx 39.1\%$
• Without replacement: $\approx 35.7\%$

Without replacement is lower because once you remove a red marble, fewer remain to draw on the second try — the probability of success drops from $5/8$ to $4/7$.

When does the difference matter?

The two answers diverge most when the sample is large relative to the population. If you drew 5 marbles from a small bag, without-replacement shrinks probabilities a lot. If the population were huge (e.g. drawing 5 cards from a deck of a million), the two would be nearly identical — which is why with-replacement calculations are often used as approximations.

Verification by enumeration (Scenario B)

Ordered draws: $5 \cdot 4 = 20$ both-red sequences out of $8 \cdot 7 = 56$ total. $20/56 = 5/14$. ✓

Unordered: $\binom{5}{2} / \binom{8}{2} = 10/28 = 5/14$. Same answer. ✓

Connection to named distributions

• With replacement + counting successes → binomial.
• Without replacement + counting successes → hypergeometric.

Each draw-by-draw derivation above is a special case of these distributions for $n = 2$.

Common mistakes

• Using the with-replacement formula for a finite population. Classic in card/dice problems where the natural reading is without replacement — re-check the problem statement.
• Forgetting to shrink both numerator and denominator. In the without-replacement step, both the red count and the total count dropped by 1.
• Misreading "independent" as "with replacement." They correlate, but without-replacement draws can still be conditionally independent across disjoint categories — always ground the rule in the actual experiment.

Try it in the visualization

Side-by-side bags: the left bag keeps the marble, the right bag removes it. Step through the draws and watch the two probabilities accumulate in real time. A slider changes the red/blue counts so you can see how the gap between the two scenarios scales with population size.