Speed up your test suite by throwing computers at it

Start a watch party

Daniel Magliola

You've probably experienced this. CI times slowly creep up over time and now it takes 20, 30, 40 minutes to run your CI suite. Multi-hour runs are not uncommon. All of a sudden, all you're doing is waiting for CI all day, trying to context switch between different PRs. And then, a flaky test hits, and you start waiting all over again.

It doesn't have to be like this. In this talk I'll cover several tools, tips and techniques for improving your CI times by massively parallelizing your test runs, instead of having to painstakingly rewrite your tests.

You can find more information about them, more documentation, and full code examples in the supplementary repo.

The table of contents below should give you a more detailed idea of what i'll cover, and will allow you to jump to a specific section, if there's a topic you'd like to specifically hear about:

  • 00:00: Introduction and explanation of what i'll cover.
  • 05:20: How to manually split your test suite into multiple jobs. Caveats and pitfalls.
  • 09:10: How to run a single CI job in parallel on multiple machines.
  • 14:10: Why it's key to optimize startup times of CI jobs, and what to focus on.
  • 18:40: Speeding up bundle install / gems cache restore times.
  • 21:00: Speeding up git checkout / cloning your repo / shallow checkouts.
  • 22:00: Speeding up Container Spin Up / Boot Up time, by carefully choosing images to use.
  • 33:00: Building your own Docker image to run on CI. Advantages and how to approach it.
  • 35:30: Speeding up bundle install and git clone (building your own Docker image).
  • 38:40: Optimizing the layers in your CI Docker image for faster boot up times.
  • 42:30: Using observability to fight randomness and better understand CI performance over time.
  • 44:40: Fighting flaky tests with automation to help us track and fix them.
  • 48:00: Unevenly distributed tests: The problem, and some potential solutions.
  • 52:15: Too much of a good thing: The problem with parallelizing "too much".
  • 54:00: Using Critical Path Thinking to prioritize and choose what to optimize and what not to.