
We run them in both before- and after- warm-up modes, roughly ~20 times in each (40 times total), and then we take the mean and geometric mean for each mode. Each suite of benchmark tests includes around 1000 tests.

Each test is performed several times, and the geometric mean of these runs is reported. We run benchmark tests for code completion and highlighting for internal components on basic code snippets on every build. Our primary resources for measuring IDE performance are synthetic and real-life benchmarks, anonymous statistics, and your feedback. We use a variety of approaches to finding problems with IDE performance. If you’ve been reluctant to upgrade because of concerns about breaking changes, we think you’ll be excited by the improvements we’ve made. Our recent versions have fixed a lot of performance issues.

The performance of your tools, and especially your IDE, has a significant impact on your development experience.
