M5 — Testing, Testability, and Coverage

SENG 365 — Software Engineering

Neil Ernst
nernst@uvic.ca

University of Victoria

2026-10-01

Module Goals

Design Judgement

By the end of this module, you should be able to:

  • identify the unit under test, input, oracle, and assertion;
  • design focused, independent, repeatable, fast, and readable tests;
  • use test difficulty as evidence about production-code design;
  • inject and stub a collaborator to force an edge case; and
  • interpret coverage as a question generator rather than proof of correctness.

JUnit, Gradle, and Test Qualities

Preparation

Before class: Read Chapter 5, Sections 5.1–5.6.

Entry Decision

A method has been manually tried with "VII" and returned 7.

What claim has actually been tested? What important behaviour remains unspecified?

Roman Numerals

int toDecimal(String roman)

I=1 V=5 X=10 L=50 C=100 D=500 M=1000

  • Add values: VII = 7, MDXX = 1520.
  • Subtract a smaller value before a larger one: XL = 40.
  • Decide what the contract says about empty, malformed, or null input.

Anatomy of a Test

@Test
void convertsSubtractivePair() {
    String input = "XL";
    int expected = 40;

    int actual = RomanNumerals.toDecimal(input);

    assertEquals(expected, actual);
}

Identify the unit under test, input, oracle, execution, and assertion.

Build a Small Test Suite

Write a suite that distinguishes plausible incorrect implementations.

Include:

  • an additive case;
  • a subtractive case;
  • a boundary or repeated-symbol case;
  • one exceptional case required by the contract; and
  • test names that state the behaviour.

For every test, state which fault it could reveal.

Test Quality Review

Exchange suites and assess:

  • Focused: is one coherent behaviour under test?
  • Independent: does execution order matter?
  • Repeatable: does environment or randomness change the result?
  • Fast: would the suite remain cheap at scale?
  • Readable: are setup, oracle, and intent obvious?

Delete or revise one redundant or brittle test.

JUnit and Gradle

./gradlew test
./gradlew test --tests RomanNumeralsTest

Tests are executable interface expectations. Gradle makes those expectations repeatable for every developer and CI environment.

Course Extension: TDD

TDD is not the chapter’s organizing framework, but it is a useful discipline:

  1. Red: express the smallest missing behaviour.
  2. Green: implement only enough to satisfy it.
  3. Refactor: improve structure while the tests constrain behaviour.

The Roman-numeral kata makes the design consequences visible: test order influences the implementation you discover.

Exit Note

This test would be brittle because it depends on __________ rather than observable behaviour.

Stubs, Dependency Injection, and Coverage

Preparation

Before class: Read Chapter 5, Sections 5.7–5.9; review Chapter 3, Section 3.8.

Entry Decision

How would you reliably test the rare branch?

final class Dealer {
    Card deal(List<Card> cards) {
        int index = new Random().nextInt(cards.size());
        return cards.get(index);
    }
}

Running the test many times is not control.

The Hidden Collaborator

The method constructs its own source of randomness.

  • The dependency is hidden.
  • A test cannot force an index.
  • Failure may be intermittent.
  • The design couples selection policy to dealing.

What is the smallest useful seam?

Refactor for Control

Introduce and inject a collaborator:

interface CardPicker {
    int pickIndex(int size);
}

final class Dealer {
    Dealer(CardPicker picker) { /* ... */ }
    Card deal(List<Card> cards) { /* ... */ }
}

Then create a hand-written stub that returns a chosen index and records how it was called.

Coverage as Evidence

For this implementation:

Card deal(List<Card> cards) {
    if (cards.isEmpty()) {
        throw new IllegalStateException();
    }
    int index = picker.pickIndex(cards.size());
    return cards.get(index);
}
  • Statement coverage asks whether every statement ran.
  • Branch coverage asks whether each decision outcome ran.
  • Condition coverage matters when a decision combines conditions.
  • Path coverage quickly becomes impractical, especially with loops.

Which uncovered branch represents a missing requirement? Which might be unreachable by contract?

Coverage-Guided Improvement

Inspect the tests and control flow together.

Produce:

  1. one missing test justified by a branch or contract;
  2. one production-code improvement revealed by test pressure; and
  3. one coverage number you would refuse to interpret without context.

Coverage can reveal absence. It cannot prove correctness.

Testing Boundaries

  • Avoid weakening production visibility only to test private methods.
  • Test observable behaviour through the public interface when possible.
  • Reflection can cross the boundary, but trades compile-time safety for run-time machinery.
  • Difficult tests often indicate too many responsibilities or hidden dependencies.

Exit Note

The tests revealed pressure to change __________ because __________.