From an Hour to Six and a Half Minutes: Speeding Up Frontend Tests in CI

How we made our frontend component-test CI pipeline almost 10× faster in two phases — through sharding, Kubernetes limits, OOM kills, flaky tests, and finally a shared browser context that failed on the first attempt and, on the second, exposed four classes of leaks we had no idea about.

Phase	Pipeline
Starting point — one 30–65min job + OOM retries	≈ 60+ min
Phase 1 — 5 shards, start at t=0, flaky fixes	14.2 min
Phase 2 — shared browser context	6.2–6.8 min

Built with Claude Fable 5 in Claude Code + Superpowers · 2 phases, 3 merge requests · ~110M tokens · ~6 h of active work spread over two days.

Ten minutes of tests, fifty minutes of overhead

The frontend has ~2,500 component tests in ~800 files — Vitest Browser Mode, real Chromium via Playwright, MSW as the mock backend, pixel-level screenshot baselines. A solid setup, but in CI it ran as a single 30–65 minute job that frequently restarted from scratch because the pod died on OOM (out-of-memory — the job outgrew the memory limit of its Kubernetes pod and the kernel killed it). In practice the pipeline routinely took over an hour.

Yet the pure test time is ~10 minutes. The rest was overhead with one dominant item: every test file got a fresh iframe that re-evaluated the entire setup graph — Redux Store, TinyMCE, theme CSS. That’s ~4–5 s times ~800 files, i.e. 70–87 % of the whole run:

Duration  1246.9s  (setup 4739.9s, import 1606.8s, tests 6282.6s)
          ^^^^^^^ wall-clock       ^^^^^ this is what iframe-per-file costs

A browser worker’s day in isolated mode: 4–5 seconds evaluating the setup graph (Store, TinyMCE, theme), a few seconds running the file’s tests, throw it all away — and again, ×800 files. The ratio comes from a real measurement: 4,740 s of cumulative setup against ~10 min of actual tests. Most of the CPU time goes into preparing an environment the browser discards a few seconds later.

What the files looked like at the start

For context on the diffs below — this is what the key files looked like before we touched them. One CI job, no sharding, no worker limit. The most telling part is the comment on retry: OOM wasn’t being solved, it was being worked around by retrying the whole job:

.gitlab-ci.yml · original state

test-component:
  stage: test
  variables:
    KUBERNETES_CPU_REQUEST: "2"
    KUBERNETES_CPU_LIMIT: "3"
    KUBERNETES_MEMORY_REQUEST: "10Gi"
    KUBERNETES_MEMORY_LIMIT: "14Gi"
  retry:
    max: 2
    # script_failure added: Vitest Browser Mode + heavy memory usage causes OOM kills and
    # Playwright browser context crashes that appear as script_failure on first attempt.
    when:
      - runner_system_failure
      - stuck_or_timeout_failure
      - script_failure
  script:
    - npm ci
    - npm run test:component        # whole suite, one pod, 30–65 minutes

vite.config.ts · original test block

  plugins: [
    react({ ... }),
    svgr(),
    checker({ typescript: true }),   // tsc also ran inside test servers
  ],
  test: {
    testTimeout: 30000,
    include: ['src/**/*.test.{ts,tsx}'],   // one project, everything isolated
    setupFiles: ['./vitest.setup.tsx'],
    browser: {
      enabled: true,
      provider: playwright(),
      headless: true,                 // isolate: unset = true, iframe per file
      instances: [{ browser: 'chromium', viewport: { width: 1280, height: 720 } }],
      ...
    },
  },

vitest.setup.tsx · original state

// Everything at module top level — in the isolated world the file was
// evaluated once per iframe anyway, so "once" held automatically.
const layerOrderStyle = document.createElement('style');   // @layer ordering
document.head.prepend(layerOrderStyle);
tinymce.baseURL = `${import.meta.env.BASE_URL}js/tinymce`;
i18n.init({ ... });
document.head.appendChild(style);       // disable animations
console.error = (...args) => { ... };   // filter ResizeObserver noise

beforeAll(async () => {
  await server.start({ onUnhandledRequest: 'bypass', quiet: true });
});
afterAll(() => {
  server.stop();     // the MSW worker registered and unregistered for EVERY file
});

beforeEach(async () => {
  await userEvent.click(document.body);
  history.push('/');
  TestStoreSeeder.seedCache();
});                  // no axios cache or web storage cleanup

afterEach(() => {
  server.resetHandlers();
  cleanup();
});                  // no cleanup of DOM residue in body

Phase 1: sharding, Kubernetes, and two flaky tests

Phase one didn’t touch the tests themselves — it split the work and stabilized the environment. The job was split into 5 shards via GitLab parallel and stopped waiting for the build stage, since it is self-contained (its own npm ci):

.gitlab-ci.yml · impact: 1 job × 30–65 min → 5 × ~10 min, starting at t=0

 test-component:
   stage: test
+  # Sharded via GitLab `parallel`: wall-clock drops ~linearly with the shard count and
+  # each pod hosts fewer live browser contexts, which is what used to OOM the single
+  # 30-60min job into whole-job retries.
+  parallel: 5
+  # The job is self-contained (own npm ci, no build artifacts) — start it at pipeline
+  # creation instead of waiting for the build stage; it is the critical path.
+  needs: []
   variables:
     KUBERNETES_MEMORY_LIMIT: "14Gi"
+    # Pin the worker count to the CPU limit: os.availableParallelism() may see the
+    # node's cores instead of the cgroup quota, spawning more browser pages than
+    # 3 CPUs can drive and ballooning memory.
+    VITEST_MAX_WORKERS: "3"
   script:
+    - npx vitest run --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL ...

A foul in the very first pipeline: parallel without the shard

The first version deployed GitLab parallel: 3 — but the --shard flag never made it to vitest. Result: three pods dutifully ran the entire suite each. The pipeline looked “parallel”, but in reality it did the same work 3× and finished just as slowly as before. Only a look at the test counts per shard exposed it; the fix was one line:

   script:
     - npm ci
-    - npm run test:component
+    - npm run test:component -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

By the way: vitest splits shards by file order, not by their cost — so shards aren’t equally fast, and the slowest shard sets the pipeline’s duration.

That comment next to VITEST_MAX_WORKERS is the whole OOM-kill story in one sentence: Vitest derived its browser worker count from os.availableParallelism(), which in Kubernetes sees the whole node’s cores, not the pod’s cgroup quota. A pod limited to 3 CPUs was spawning browsers as if it ran on a 32-core machine; memory grew, the kernel killed — and GitLab restarted the entire 40-minute job.

On top of that, small things that added up: a typecheck plugin inside the test servers (it has its own CI job; here it only stole CPU from the browsers), a module-level axios cache that could serve a previous test’s response and bypass the MSW mock, and the MSW service worker registering and unregistering for every single file:

vite.config.ts · vitest.setup.tsx

 plugins: [
-  checker({ typescript: true }),
+  // No typecheck inside test servers: CI has a dedicated typecheck job.
+  ...(process.env.VITEST ? [] : [checker({ typescript: true })]),
 ],

 beforeEach(async () => {
+  // Module-level axios cache survives across tests within a file — a hit from an
+  // earlier test's identical request bypasses MSW entirely.
+  httpCacheStorage.data.clear();
 });

 beforeAll(async () => {
+  // The worker is registered on the origin, so it survives across files;
+  // re-registering it per file costs real time.
+  if (!window.__mswWorkerStarted) {
     await server.start({ onUnhandledRequest: 'bypass', quiet: true });
+    window.__mswWorkerStarted = true;
+  }
 });
-afterAll(() => { server.stop(); });

And one Windows peculiarity for developers: the default vitest browser API port (63315) falls, on some machines, into a TCP range reserved by Hyper-V — the tests then fail to start locally at all, dying with listen EACCES. Low ports tend not to be excluded:

     browser: {
+      // The stock port (63315) falls into a Hyper-V/WSL excluded TCP range on some
+      // Windows machines, failing the whole run with `listen EACCES`.
+      api: { port: 5123 },
       ...

Two flaky tests that cost 10 minutes per retry

After sharding, two tests remained that failed randomly and, through CI retries, stretched the run by ~10 minutes. Both had the same shape of problem — a fixed-time assumption where asynchronous work is running:

• Diagram element selection — a click dispatched during an asynchronous ELK relayout got silently lost. Fix: keep clicking until the model confirms a selectionRect (a repeated click is an idempotent re-select).
• TinyMCE init — a 1 s timeout for editor initialization wasn’t enough under load; raised to 15 s with polling.

The pattern for next time: don’t wait a fixed time, poll for a model-backed signal — and only retry idempotent interactions.

	Before	After
whole pipeline	60+ min	14.2 min
test-component	1 job, 30–65 min	5 shards, 7–13 min
OOM retries	routine	0

Interlude: the first shared-context attempt — green on Windows, red on Linux

Already in phase 1 we tried the biggest speedup: browser.isolate: false, i.e. one shared browser context per worker instead of an iframe per file. Locally on Windows the validation passed cleanly — and on Linux CI ~30 tests failed. We reverted the change and walked away with an inventory of leaks:

• DOM residue accumulates in the shared <body> until a scrollbar appears → the viewport narrows → ~26 screenshots shifted by a few px,
• localStorage spills across files (search rendered someone else’s “recent queries”),
• module state changes the word-cloud layout,
• late fetches from componentDidMount land as 404s after teardown and fail otherwise-green files.

Lesson #1 — Validating on the developer’s OS is not enough. Rendering leaks (scrollbar, fonts) only show up on the CI platform — every candidate configuration must go through a real pipeline before it’s called done.

Phase 2: shared browser context, this time with a plan

The second attempt started differently: first came an implementation plan with the complete leak inventory from the reverted attempt, a task with a fix for each one, and a validation method able to tell a real regression from noise. Only then was the code touched.

The core of the change: two vitest projects differing only in isolation. main shares the context, isolated keeps iframe-per-file for the files that genuinely need it — and a content scan decides the assignment, so a new “poisonous” file can never silently slip into the shared world:

vite.config.ts · impact: setup 4,740 s → ~380 s cumulative

-    include: ['src/**/*.test.{ts,tsx}'],
+    projects: [
+      {
+        test: {
+          name: 'main',
+          include: ['src/**/*.test.tsx'],
+          exclude: isolationRequiredTestFiles,   // content scan, see below
+          browser: { isolate: false },
+          sequence: { groupOrder: 0 },
+        },
+      },
+      {
+        test: {
+          name: 'isolated',
+          // Unit tests of module singletons (hydration cache, debounce
+          // timers, "once per page-load") need pristine modules.
+          include: ['src/**/*.test.ts', ...isolationRequiredTestFiles],
+          sequence: { groupOrder: 1 },
+          maxWorkers: 4,
+        },
+      },
+    ],

Modules are now evaluated once per worker and stay cached; between tests only state is reset (store, MSW handlers, DOM, storage). Cumulative setup: 4,740 s → ~380 s. The price: whatever a test doesn’t clean up is inherited by everyone after it — hence the rest of this chapter.

Each known leak from the inventory got its own fix in the setup hooks:

vitest.setup.tsx

 beforeEach(async () => {
+  // Web storage is origin-scoped — in a shared context it spills across files.
+  localStorage.clear();
+  sessionStorage.clear();
 });

 afterEach(() => {
   server.resetHandlers();
   cleanup();
+  // cleanup() unmounts React roots but not non-React residue (TinyMCE aux
+  // elements, orphaned portals). Accumulated residue grows a scrollbar, which
+  // narrows the viewport and shifts every centered screenshot off its baseline.
+  document.body.replaceChildren();
+  window.scrollTo(0, 0);
 });

The rest of the inventory

Late fetches from componentDidMount that settle with a 404 after the test’s teardown (the MSW handlers are already gone → the request falls through to the vite server) stopped failing green files in a targeted way — only 404s are suppressed, real errors stay visible:

     onUnhandledError(error) {
+      // Late component fetches settle with a 404 after the test tore down its
+      // MSW handlers. Only suppress 404s: real request bugs (500s, network)
+      // must stay visible.
+      if (error.name === 'AxiosError' && error.message?.includes('status code 404')) {
+        return false;
+      }
     },

Two smaller lessons from the same round:

• The word cloud had a seed, and it drifted anyway. The component has been passing a fixed random generator to the visualization library since spring — yet the layout in the shared context differed from the baseline. The source of the nondeterminism wasn’t the PRNG but warmed-up module state changing text measurement. When a seed doesn’t help, the pragmatic path remains: those files run in the isolated project.
• The two projects must not run at once. In parallel, each spins up its own Vite server + a full worker pool, and the overloaded CPU starts dropping module fetches (“Failed to fetch dynamically imported module”). That’s why sequence.groupOrder runs the projects one after the other, and the isolated project gets maxWorkers: 4.

Validation used a failure-set diff: the whole suite once with isolation forced on (baseline) and once with the new configuration, comparing the exact set of file :: test entries. Screenshot failures are expected noise on Windows (the baselines are chromium-linux) — identical on both sides, so they cancel out in the diff. The result had to be empty in both directions, twice in a row: 738 = 738. Only then did the code go to CI. And that’s where the more interesting part began.

Traps in the validation itself — Two things tried to quietly break the validation. With CI mode on, vitest retries failures and appends a (retry x2) suffix to the line — the regex parsing the failure set stopped matching on it, and the diff would have come out “clean” because both sets would have been empty. And the repo turned out to contain five accidentally committed Windows screenshot baselines (they slipped in with an earlier change into an otherwise Linux set) — on developer machines they silently turned five screenshot tests green that should have been failing. The regex got fixed, the baselines got deleted.

Case #1: a synthetic beforeunload kills the mocks for everyone after it

(pipeline 96237, shard 1/5)

Locally, the diff exposed 8 failures in a single file — a hook with “once per page-load” semantics suddenly never got its API response. Bisection led to a deterministic pair: it was enough for a test of an overlay component to run earlier in the worker, one that dispatches window.dispatchEvent(new Event('beforeunload')).

The mechanism: on beforeunload, the MSW client sends the service worker a CLIENT_CLOSED message — “the page is closing, stop mocking for me”. In the isolated world the iframe was dying anyway, so nobody cared. In a shared context this deregisters the long-lived iframe, and every subsequent file in that worker sends its requests straight to the vite server: 404.

vite.config.ts · content scan

       if (
         content.includes('vi.mock(') ||
         content.includes('vi.doMock(') ||
+        // A synthetic beforeunload = MSW CLIENT_CLOSED = end of mocks for
+        // the whole shared context. Such files run isolated.
+        content.includes('beforeunload')
       ) {

Case #2: wandering screenshots and two kinds of type

(pipeline 96237 → 96244)

CI was left with 5 screenshot mismatches. We regenerated the baselines from artifacts — and the next pipeline failed again, just on different files, and one of the regenerated ones failed anew. The diffs showed “ghosts”: the same content shifted by ~1 px.

A crop of one mismatch’s diff: content shifted by ~1 px due to different font metrics leaves red “ghosts” of text in the diff.

The breakthrough came from comparing two “actual” snapshots of the same test from different pipelines: one was typeset in a serif font, the other in a sans-serif. The tests don’t load the production index.html, so the Inter font doesn’t exist in tests at all and every baseline was created with the fallback font. But one test added a <link> to /css/index.css into document.head — including the @font-face for Inter — and never removed it. The head survives in a shared context: every file that ran after it rendered in Inter and no longer matched its baseline. Depending on which worker the file happened to land in, the mismatches “wandered” between runs.

Pipeline 96237 · sans-serif render — the file was preceded in its worker by a test with a leaked stylesheet (Inter).

Pipeline 96244 · clean render — the serif fallback font the original baselines were created with. The same test, two consecutive pipelines; crops magnified 3×. Beyond the typeface, the vertical geometry differs too — hence the ghosts shifted by a pixel.

From the same family: ModelViewer tests spied on CoreApiService.post via vi.spyOn(...).mockResolvedValue({ data: [] }) without a restore — and the files after them got empty tables.

vite.config.ts · RichTextEditorLinkStyles.test.tsx

+    // Spies on service singletons otherwise outlive the file that created them.
+    restoreMocks: true,
+    // Same for vi.stubGlobal (window.Prism, navigator, ...).
+    unstubGlobals: true,

+  afterEach(() => {
+    // index.css carries the Inter @font-face + global typography; the head
+    // outlives the file, so a leftover link recolors everyone's screenshots.
+    indexCssLink.remove();
+  });

The regenerated baselines were reverted (they had been chasing a poisoned render), and with the fix at the source the next three pipelines in a row were green.

	Before	After
test-component shard	7.9–18.3 min	3.6–6.1 min
whole pipeline	19–25 min	6.2–6.8 min
local full suite	20.8 min	12 min
validation	—	diff 738 = 738 · 3× green CI

Behind the scenes: how it was built with AI

Both phases were built in Claude Code with the Claude Fable 5 model and the Superpowers plugin (a brainstorm → plan → execution workflow with validation gates). Phase 1 was interactive — 15 prompts, continuous steering. Phase 2 ran from a single prompt over a prepared plan; a human stepped in only for git operations and at the end.

	Phase 1	Phase 2
input	15 prompts, interactive	1 prompt + the phase 1 plan
active time working with AI	~3.4 h	~2.4 h
tokens	~67M	~41M
API calls	587	463
output	sharding + stabilization in production	shared context in production

The prompts that drove it

The prompts are quoted in the original Czech, with translations in italics:

When	Prompt	Tokens
Jul 9, 11:29	”Dostal jsem feedback, že frontend testy běží hodinu…” — I got feedback that the frontend tests take an hour, that seems way too long to me…	kickoff
Jul 9, 11:40	”zkusme udelat vsechno” — let’s try to do everything	25.6M
Jul 9, 14:28	”uz to bezi nejak dlouho” — it’s been running for quite a while now — at that moment 3 pods were running, each with the full suite (the missing --shard)	8.7M
Jul 9, 14:50	”it failed” — this produced the entire leak inventory and the revert	9.4M
Jul 10, 6:57	”…mě zajímá celá pipeline. Klidně ten paralel můžeme zvednout i víc.” — …I care about the whole pipeline. We can raise the parallelism even further.	13.4M
Jul 10, 8:55	”mergnul jsem to, můžeš mi udělat větev na ten follow-up a nějakou spec, abych mohl clear context?” — I merged it; can you make me a branch for the follow-up and a spec, so I can clear context?	2.8M
Jul 10, 13:11	”pokračuj podle docs/superpowers/plans/2026-07-10-component-tests-shared-context.md” — continue according to [the plan]	41.4M

What the phase 2 plan looked like

That “spec” from the last phase 1 prompt is a markdown file: goal, architecture, the leak inventory from a real CI run (each item had to be addressed), and tasks with verification commands. An excerpt:

**Goal:** Enable `browser.isolate: false` for the bulk of the vitest suite,
cutting CI shard times from ~10–15 min to ~5 min.

| # | Leak                        | Symptom on Linux CI          | Status  |
|---|-----------------------------|------------------------------|---------|
| 1 | DOM residue in shared body  | scrollbar → ~26 screenshots  | Task 3  |
| 2 | localStorage leaks          | wrong dropdown branch        | Task 2  |
| 4 | late componentDidMount 404s | fail otherwise-green files   | Task 4  |

- Windows validation is NOT sufficient — every candidate config must go
  through a branch pipeline before being called done.
### Task 6: Local full-suite validation (failure-set diff)
### Task 7: Linux CI validation (the gate Windows cannot provide)

Worth noting: the plan did not predict the two new leaks (beforeunload/MSW, spy and stylesheet pollution) — but it prescribed a validation method that caught them reliably and a bisection procedure that tracked them down. A plan doesn’t have to know all the answers; it has to be able to tell that something is off.

What we’re taking away

• First measure where the time actually is. The “slow tests” were ~85 % environment overhead, not tests.
• Kubernetes limits ≠ what Node sees. os.availableParallelism() returns the node’s cores; the worker pool must be pinned to the cgroup quota, or OOM follows.
• Shared context = every uncleaned side effect is global. Storage, DOM, the document head, spies, the service worker — everything needs deterministic cleanup between tests.
• Enforce problem classes with configuration, not discipline. The content scan sends poisonous files into isolation automatically; restoreMocks/unstubGlobals wipe out a whole class of leaks at once.
• Validate with a failure set, not a feeling. A file :: test diff against a baseline separates regression from noise even with hundreds of expected failures.
• A flaky test is almost always fixed time vs. asynchronous reality. Poll for a model-backed signal; only retry idempotent actions.
• Retries in CI yes, locally no. CI retries absorb rare timing flakes (and vitest flags them); locally, failures must stay raw or you’ll never see the leaks.