Testing in Flutter: Unit, Widget and Integration Tests, Fake Repositories and Golden Tests

Version note: The examples in this post were run with Flutter 3.47.8 (Dart 3.13.7), flutter_test and integration_test (shipped with the Flutter SDK), mocktail 1.0.5, provider 6.1.5+1 and go_router 18.1.4. The output comes from flutter test runs. The integration_test file passed analyze, but it needs a device, an emulator or ChromeDriver, so it was not run on a device; the same flow ran as a widget test.

Guess First

In the tests of Bookstore, the online bookstore used as the shared example across this site's series, you create a TokenStore mock with mocktail and call await tokens.read() without stubbing anything. read() returns a Future<Tokens?>, so null is a valid result. What happens: do you get null, or does the test blow up? Is the "returns a default value" behavior you know from Moq there?

A Map of the Tests

Every post in the series added its own tests; the app now has 86 tests, and all of them run in about 10 seconds:

00:10 +85 ~1: All tests passed!

The ~1 is the live API test from the thirteenth post: it is skipped when API_BASE_URL is not given. Flutter has three kinds of tests, and each answers a different question:

Flutter's architecture recommendations have two items under testing, and both are labelled "Strongly recommend": test the layers both separately and together, and "Make fakes for testing". This post is what the two look like in Bookstore.

Fakes: Focusing on Inputs and Outputs

A fake is a small but working implementation of an interface, written to be used in tests. The FakeBookRepository we have used since the eleventh post returns a real catalog, records which pages were asked for and fails when told to:

/// A working BookRepository that records the pages asked for and can fail.
class FakeBookRepository implements BookRepository {
  FakeBookRepository({SampleCatalog? catalog, this.failuresLeft = 0})
    : _catalog = catalog ?? SampleCatalog(total: 60, pageSize: 20);

  final SampleCatalog _catalog;
  final calls = <int>[];
  int failuresLeft;
  Object error = Exception('timeout');

  @override
  Future<Result<List<Book>>> getPage(int page) async {
    calls.add(page);
    if (failuresLeft > 0) {
      failuresLeft--;
      return Result.failure(error);
    }
    return Result.ok(await _catalog.page(page));
  }

Tests written with this fake ask about behavior: "is the next page loaded at the end of the list?", "does try again work after an error?". The output of the catalog screen tests:

after start: calls [0]
after scrolling to book 20: calls [0, 1]
after the last book: calls [0, 1, 2, 3]
after pull to refresh: calls [0, 0]
error then retry: calls [0, 0]

In the previous post we added the error field; we tested that CachedBookRepository tells a network error from a server error by making the same fake return a NetworkException or an ApiException(500, ...). InMemoryCartRepository, SampleOrderRepository and SampleAuthRepository play the same role: in the production code they exist only for previews and tests, and the ViewModel tests need no mocking library. Fakes have one more advantage: since DriftCartRepository and InMemoryCartRepository implement the same contract, you can run the same tests against both and see that their behavior stays the same.

mocktail: Verifying Interactions

Some tests do not ask "what is the result?" but "was this call made?". Deleting the tokens when the user signs out is exactly that kind of question. For this we use mocktail; unlike mockito, it needs no code generation:

class MockTokenStore extends Mock implements TokenStore {}

void main() {
  // any() needs a value of the parameter's type; records are no exception
  setUpAll(() => registerFallbackValue((accessToken: '', refreshToken: '')));

  test('sign-out deletes the tokens exactly once', () async {
    // The question is "was clear() called?", not "what is stored?":
    // an interaction, which is what a mock is for
    final tokens = MockTokenStore();
    when(tokens.clear).thenAnswer((_) async {});
    final auth = ApiAuthRepository(
      ApiClient(baseUrl: Uri.parse('http://127.0.0.1:52813'), tokens: tokens),
      SharedPreferencesAsync(),
    );

    await auth.signOut();

    verify(tokens.clear).called(1);
    verifyNever(() => tokens.write(any()));
  });

Without the registerFallbackValue line, the test failed with this error:

Bad state: A test tried to use `any` or `captureAny` on a parameter of type `({String accessToken, String refreshToken})`, but
registerFallbackValue was not previously called to register a fallback value for `({String accessToken, String refreshToken})`.

Because of null safety, the any() argument matcher needs a real value of the parameter's type; for your own types and for records you register one once. The answer to "Guess First" is in the output too:

_TypeError: type 'Null' is not a subtype of type 'Future<({String accessToken, String refreshToken})?>'

An unstubbed method returns null on a mock; since the method's return type is a Future, null is not a Future and the test fails with a type error. Moq's "loose" behavior is not there: you have to define every async method with thenAnswer. Keep mocks to the places where the question is about an interaction; mocking every repository ties the tests to the implementation, and every refactoring means changing the tests too. The "When Not to Mock" post in the Unit Testing series makes the same decision on the C# side.

Widget Tests: the Whole App with provider and go_router

A widget test is not limited to a single widget. If you give the fakes to appProviders and pump BookstoreApp, you test a whole user flow with the real router, the real redirect, the real ViewModels and the real screens, without an emulator. We wrote the checkout flow as a function:

/// Sign in, add Dune to the cart, check out, see the order. The same steps
/// run as a widget test (test/flows) and on a device (integration_test/).
Future<void> checkoutFlow(WidgetTester tester) async {
  final orders = SampleOrderRepository();
  await tester.pumpWidget(
    MultiProvider(
      providers: appProviders(
        bookRepository: FakeBookRepository(),
        orderRepository: orders,
      ),
      child: const BookstoreApp(),
    ),
  );
  await tester.pumpAndSettle();

  // 1. Orders need a signed-in user: the tab redirects to the sign-in form
  await tester.tap(find.text('Orders'));
  await tester.pumpAndSettle();

The rest goes back to the catalog, adds Dune with find.byTooltip('Add to cart'), taps "Checkout" on the cart tab, fills in the address form and looks at the orders tab. The finders target what the user sees (text, tooltip, field label), which keeps the test from breaking when the inner structure of the widget tree changes. The output:

signed in, orders on screen: 2
snack bar: Order BK-1045 placed, cart empty: true
first order: BK-1045 · ₺489.00

The flow finished in under a second. The router's redirect, the order list refreshing by itself through OrderRepository's notification and the cart being cleared were all verified in the same test. In the official architecture example the router is also mocked with a MockGoRouter; that helps when testing a screen on its own, but when you want to test navigation itself, setting up the real router as we did catches more.

integration_test: the Same Flow on a Device

A widget test runs on a fake screen without a platform: no real keyboard, no real scroll physics, no platform side of the plugins, no drawing with Impeller. The integration_test package runs the test with the same WidgetTester API inside a real build, on a device or emulator:

void main() {
  // Runs the app on a real device or emulator instead of the test VM
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  testWidgets('checkout from sign-in to the order list', checkoutFlow);
}

This is where putting the flow in a separate function pays off: the same steps run in test/flows on every commit within seconds, and in integration_test/ on a device. These are the commands from the Flutter docs; we did not run any of them, because this post used no devices or emulators:

flutter test integration_test/checkout_flow_test.dart -d <device-id>
flutter drive --driver=test_driver/integration_test.dart --target=integration_test/checkout_flow_test.dart -d web-server

The first runs on an Android or iOS device; if you do not pass -d and more than one device is connected, Flutter asks which one to use. The web needs a running ChromeDriver and a small test_driver/integration_test.dart file that calls integrationDriver(). For a range of devices, the docs also describe uploading the Android test APK to Firebase Test Lab. In practice, integration tests are kept few: a handful of critical flows, run in CI at night or before a release.

Golden Tests and Platform Differences

A golden test compares an image of a widget pixel by pixel with an approved PNG. For BookCard:

await expectLater(
  find.byType(BookCard),
  matchesGoldenFile('goldens/book_card_dune.png'),
);

The first run creates the reference image with flutter test --update-goldens. Then we increased the space between the image and the title in the card from 12 to 16 pixels:

Golden "goldens/book_card_dune.png": Pixel test failed, 2.95%, 1382px diff detected.
Failure feedback can be found at
<work>/app/bookstore_app/test/ui/catalog/failures

The failures/ folder gets the old image, the new image and two images that show the difference; we added that folder to .gitignore. The first surprise when you open the golden is that the text is black squares: by default flutter_test uses a test font that draws every character as a square (the SDK docs call it Ahem). The reason is platform differences. Text drawn with real fonts, anti-aliasing and font rendering come out a few pixels different on macOS, Linux and Windows, and the same test passes on one machine and fails on another. The test font reduces that difference but does not remove it completely; that is why we tagged the golden tests and update and run them on one platform only (usually Linux in CI):

flutter test --exclude-tags golden,live
00:06 +84: All tests passed!

A golden test is a good tool for catching unintended changes in layout and colors; it is not for checking the text itself or how it looks with the real font. There are community packages that load font files in tests to produce goldens with real fonts; we did not try them.

Production Notes

Measure coverage, but do not make it a target. flutter test --coverage writes coverage/lcov.info; leaving out the generated .g.dart files, line coverage came out at 89.4%. Looking at the layers is more useful than the total: data/repositories 82.6%, ui/book_details 76.4%. Most of the missing lines are error branches; decide which ones are worth testing by looking at them line by line.

Keep the tests fast. Running 86 tests in 10 seconds is the reason they can run on every commit. Separate and tag tests that depend on the real network, real time (tests that wait with Future.delayed) or a real device; in CI let the fast group run on every pull request and the slow group at night. The workflow structure from the GitHub Actions series applies here too.

Inject time. OrdersScreen takes now, and CachedBookRepository takes a now function; that is how we tested scenarios like "35 minutes later" without waiting. Code that calls DateTime.now() directly produces tests that fail at midnight or at the end of the month.

Common Pitfalls

The first pitfall is calling pumpAndSettle() on a screen with an endless animation. A spinning CircularProgressIndicator never settles, so the test times out; check the loading state with pump().

The second pitfall is expecting real HTTP in a widget test. Requests inside testWidgets return 400; replace the network with a fake repository or MockClient.

The third pitfall is every developer updating the goldens on their own machine. Within a week the goldens flip between macOS and Windows, and nobody knows whether a difference is real.

The fourth pitfall is putting integration_test in place of all other tests. A slow test suite that needs devices and fails at random soon becomes something nobody runs.

A Small Experiment

★☆☆ Add a second golden for the dark theme (AppTheme.dark) to the BookCard golden test, and fail the test once by watching the price color change.

★★☆ Write a single contract test for InMemoryCartRepository and DriftCartRepository: run the same test group against both implementations through a factory function. Do both return TooManyCopies after the tenth copy?

★★★ Add the scenario where a user who is not signed in taps "Checkout" to the checkout flow. The address page turns into the sign-in page through the redirect; make the user return to the address form after signing in, and verify it with the checkoutFlow test.

In the next post we take the app to the stores: the Android App Bundle and the iOS build, signing, version numbers, flavors for dev/staging/prod with --dart-define-from-file, and the build and signing steps in GitHub Actions.