We currently have the possibility to add different parameters when running benchmarks. These result in a matrix how they are run, e.g.,:

common.createBenchmark(main, {
  source: ['array', 'buffer', 'string'],
  len: [10, 2048],
  n: [50, 2048]
});

This results in 12 variations of the specific benchmark being run (3x source * 2x len * 2x n). However, often we only require to test a specific subset of these parameters but it is not possible to define such groups.

Example:

common.createBenchmark(main, {
  groupA: {
    source: ['array'],
    len: [10, 2048],
    n: [50]
  },
  groupB: {
    source: ['buffer', 'string'],
    len: [2048],
    n: [50, 2048]
  }
});

Here we'd only run the benchmark 1 * 2 * 1 + 2 * 1 * 2 times (6x).

So if anyone would feel like implementing any such grouping functionality, that would be great! It will not be beneficial in all cases but at least in a couple. That way it's possible to reduce overall runtime.

// cc @mscdex