Then we temporarily reverted the corrections. Both tests failed for the original reasons. That second result was the one I cared about.

As the engineering manager on the consumer team, I was responsible for a wider product surface that included mobile, backend and web work. Mobile QA was a particular bottleneck: after implementation, we still had to assemble enough confidence to release. A new test could help, provided it caught the behaviour we had just fixed.

What a green test can hide

The test suite grew quickly as we made tests part of each change. In April 2026, the mobile app had 113 unit-test files. By 8 October, it had 2,257. That is a record of sustained investment, though a file count tells us little about whether a particular assertion would catch a regression.

This mattered more as agents began writing implementation and tests together. The agent could understand the intended behaviour, change the code and produce a passing test in one run. But a test that agrees with the new implementation might also agree with the old defect. It may check the wrong state, mock away the failing condition or assert only that a screen rendered.

I wanted a narrower answer at review: would this test have stopped the bug before the fix existed?

Run the test against the bug

For a regression, our request became specific. Write a test for the behaviour the defect violated. Confirm it fails before the correction and passes with the correction. When the history or setup made that hard to see, we sometimes temporarily reverted the fix and ran the new test again.

The last failure needs inspection. A broken import, a missing mock or an unrelated setup error proves nothing about the regression. We looked for the failure the test was meant to report. In the review with the two missed defects, both new specifications failed for their original reasons when we reverted the fixes. That gave us a stronger reason to keep them than the green checks alone.

This is a small extra step, but it changes the conversation in a pull request. “Tests pass” describes the current branch. “This test fails when the defect returns” describes the protection we are adding for the next branch.

Keep the evidence close to the change

The test still has to run in CI, and its result has to belong to the revision under review. If the code changes after a green run, we need fresh results for the affected checks. For a mobile UI change, unit tests are one part of the evidence; the team also captures the interaction on devices so a reviewer can see what happened in the actual screen.

I do not ask every test to go through a manual revert. A new feature test may have no previous bug to restore, and some changes need a different verification method. For a regression, though, the old failure is available as a useful challenge to the test. It is worth taking.

The two defects in that review had already made it through earlier passes. Their tests now had a reason to exist: they could recognise those failures if they returned. That was the standard I wanted as the test inventory grew.

The broader engineering process and the test history are in my Beyond the QA Bottleneck case study.