M5 — Testing, TDD, and Coverage

SENG 365 — Software Engineering

Neil Ernst
nernst@uvic.ca

University of Victoria

2026-08-06

Learning Objectives

Learning Objectives

Reading: Robillard, 3rd ed. — Ch. 5, Unit Testing (pp. 97–123).

  • Why and how to test in Java
  • Reflection and metaprogramming in Java and Python
  • Getting around encapsulation in tests
  • Testing with stubs — why and how
  • Coverage metrics and when to use them

Why Test

Testing

  • Why do we care?
  • Types: unit, integration, regression, performance, fuzz, exploratory, UI, smoke, acceptance.
  • Some testing is manual; ideally as much as possible is automated.
  • Video: JUnit and Gradle — units; Fast, Independent, Repeatable, Focused, Readable; coverage in IntelliJ.

Test-Driven Development

  • Rule 1: no new code until a test fails
  • Rule 2: eliminate duplication
  • Forces you to write tests; limits implementation to whatever makes the test pass
  • Design is necessarily testable — likely loosely coupled
  • TDD is a conceptual shift

The TDD Cycle

  • Pick the smallest behaviour from the domain space
  • Red: write the test that describes it
  • Green: write the minimum code to make it pass
  • Refactor any duplication
  • Continue to the next behaviour

Exercise: Roman Numerals

Write a set of TDD-driven tests that convert Roman numerals to decimal.

I=1 V=5 X=10 L=50 C=100 D=500 M=1000 — e.g. VII = 7, MDXX = 1520.

Rule: add values unless a smaller numeral precedes a larger one, in which case subtract (XL = 40).

  • What are the edge cases? How many tests per edge case?

Design Questions from Testing

  • Store as string or another type?
  • Figure out the algorithm first, or as you test?
  • When to refactor common functionality?
  • Resist the temptation to gold-plate.
  • Choosing tests is hard — it forces you to think about the design.

Programming as Rock Climbing

  • Protection; the fall.
  • Kent Beck: Limbo and test && commit || revert
  • Little increments → fewer conflicts. How do we turn little increments into big ideas?

Testing OO Challenges

Reflection / Metaprogramming

Sometimes we must test private methods — they are units too.

  • Modify visibility to test? Don’t test? Show the InstructorTest class.
  • Java class loading underlies reflection; java.lang.Class is the basis.
  • Note problems with class loading and polymorphism.
  • Are there similar mechanisms in other languages?

Test Criteria

The book uses: Fast, Independent, Repeatable, Focused, Readable.

Testing Exceptions and Stubs

  • Testing exceptions: add the exception to the JUnit annotation.
  • Stubs/Mocks: Mockito vs. roll-your-own.
    • Issues: defining an interface for another class; drift between mock and reality over time.
  • Stubs pair with patterns like Strategy (e.g. NullMove — we care that it was delegated, not how).

Coverage

What to Test / CFGs

  • Functional (black-box) vs. structural (white-box) testing.
  • White-box: ensure all paths are exercised.
  • A test case is a complete path through the control-flow graph.

Coverage Metrics

  • Statement: all nodes visited
  • Decision/Branch: all edges visited
  • Condition: every condition takes each outcome (T/F) at least once
  • Path: every path traversed (loops multiply paths)
  • Tools like Emma give instruction, line, and branch metrics; path coverage needs formal verification.

New Advances in Testing

  • Facebook Sapienz: how to test millions of possible interactions?
  • A 1-in-a-million bug at 100M users happens ~100×/day.
  • Assume all tests are flaky at scale.
  • Build a model of the system and of the test space, then search for useful tests.

In-Class Activities

Day 1

  • Group discussion for EX4

Day 2

  • CI overview on GitHub — configuring GitHub Actions
  • CFGs; TDD with Roman numerals; Sapienz

Day 3

  • Office hours