Home Original page

GitHub - expressjs/perf-wg: Performance Working Group

Charter

Mission Statement

The Express Performance Working Group establishes a framework for evaluating performance changes in Express, providing consistent measurement criteria, tooling, and guidance to ensure performance-related decisions are made with data and context.

Core Philosophy

We focus on evaluation, not prescription. Rather than dictating how to write fast code, we:

  • Define how Express evaluates performance work
  • Measure whether changes are meaningful
  • Ensure performance PRs are evaluated consistently and in context

Responsibilities

Performance Evaluation Framework

  • Establish evaluation criteria (latency, throughput, memory usage)
  • Define what constitutes "meaningful" performance changes
  • Create standards for testing under various conditions (concurrency, payload size, middleware stack)

Tooling & Infrastructure

  • Tooling: see Perf CLI Readme.md
  • Provide environments and tools to make measurements possible
  • Help contributors run performance tests and interpret results
  • Support dependency version overrides for testing performance across different Express versions
  • Maintain two categories of tests:
    • Focused unit benchmarks (in package repos)
    • E2E load tests of realistic applications (in express repo)

Key Questions We Answer

  • Does this proposed change meaningfully improve performance under realistic load?
  • By how much? (p95 latency, req/sec, CPU usage)
  • Under what conditions?

Ecosystem Collaboration

  • Work with Node.js core on unlocking performance improvements
  • Maintain relationships with monitoring providers through OSS programs
  • Support public benchmark maintenance

Current Initiatives

Initiative Champion Status Links
Setup Charter @wesleytodd In progress #3

Members

The Performance WG is made up of volunteers, you do not need to be a member to participate, but once you participate please open a PR to add youself here.

Two teams exist for mentioning the group and managing access:

  • @expressjs/perf-wg: everyone participating in the WG has write access to this repo
  • @expressjs/perf-wg-captains: the repository captians as per Express project guidelines

Team Members

Meetings

The Performance Working Group meets bi-weekly (see meeting issues). The meeting is open to the public. The agenda and meeting notes are published in this repository. You can find the calendar entries in the OpenJS Foundation calendar.

Offline Discussions

The Performance Working Group uses GitHub issues for offline discussion. The discussions are open to the public and anyone may participate. Also, the group uses the channel #express-perf-wg in the OpenJS Foundation Slack for realtime discussions.

How to Run the Load Tests

Requirements

  • Node.js v20 or newer
  • Docker for containerized runners
  • Load cli's on your PATH
    • autocannon for local runner (optional, can be used via PATH="$(pwd)/node_modules/.bin:${PATH}" npm run test:load)
    • wrk2 for local runner (optional)

Quick Start

  1. Install dependencies

  2. Run basic test with vanilla Node.js

CLI Usage

CLI Options

The load command supports the following options:

Option Short Default Description
--cwd - Current directory Working directory
--runner -u @expressjs/perf-runner-local Runner module to use
--repo - https://github.com/expressjs/perf-wg.git Git repository URL
--repo-ref - master Git branch/tag to use
--test -t @expressjs/perf-load-example Test module to run
--node -n lts_latest Node.js version to use
--duration -d 60 Test duration in seconds
--overrides -o - JSON string with package overrides
--config -c - JSON string with package overrides
--[no-]write - - Write results file to disk
--force-rebuild - - For use with docker based runners, forces rebuilding container

Basic Commands

# Run with default local Node.js runner
node --run load -- --test="@expressjs/perf-load-example"

# Run with specific runner
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid"

# Run with overrides
node --run load -- --test="@expressjs/perf-load-example" --overrides='{"express":"5.0.0"}'

# Run with custom duration (default: 60 seconds)
node --run load -- --test="@expressjs/perf-load-example" --duration=30

# Run with specific runner and overrides
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid" --overrides='{"express":"5.0.0"}'

# Run with specific runner, overrides, and custom duration
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid" --overrides='{"express":"5.0.0"}' --duration=120

# Run with specific runner and overrides and pass api key env
NSOLID_SAAS="XXX" node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid" --overrides='{"express":"5.0.0"}'

# Compare results
node --run compare -- result-A.json result-B.json

Runner-Specific Examples

# Vanilla Node.js (baseline performance)
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-vanilla"

# Vanilla Node.js with --force-rebuild
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-vanilla" --force-rebuild

# N|Solid with built-in monitoring
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid"

# Node.js with Datadog APM (when available)
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-datadog"

# Node.js with New Relic APM (when available)
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-newrelic"

Programmatic Usage

Basic Import and Usage

// Import specific runner
import vanillaRunner from '@expressjs/perf-runner-vanilla';
import nsolidRunner from '@expressjs/perf-runner-nsolid';

// Run single test
const results = await vanillaRunner({
  test: '@expressjs/perf-load-example',
  overrides: { express: '5.0.0' }
});

console.log(`Requests/sec: ${results.clientResults.requests.mean}`);

Performance Comparison Script

import vanillaRunner from '@expressjs/perf-runner-vanilla';
import nsolidRunner from '@expressjs/perf-runner-nsolid';

async function compareRuntimes() {
  const testConfig = {
    test: '@expressjs/perf-load-example',
    overrides: { express: 'latest' }
  };

  console.log('Running performance comparison...');

  // Run tests in parallel
  const [vanillaResults, nsolidResults] = await Promise.all([
    vanillaRunner(testConfig),
    nsolidRunner(testConfig)
  ]);

  // Compare results
  const vanillaRPS = vanillaResults.clientResults.requests.mean;
  const nsolidRPS = nsolidResults.clientResults.requests.mean;
  const overhead = ((vanillaRPS - nsolidRPS) / vanillaRPS * 100).toFixed(2);

  console.log('Performance Comparison:');
  console.log(`Vanilla Node.js: ${vanillaRPS.toFixed(0)} req/sec`);
  console.log(`N|Solid:        ${nsolidRPS.toFixed(0)} req/sec`);
  console.log(`Overhead:       ${overhead}%`);
}

compareRuntimes().catch(console.error);

Advanced Configuration

import { createRunner } from '@expressjs/perf-runner-shared';

// Create custom runner configuration
const customRunner = createRunner({
  type: 'custom',
  runtime: 'node.js',
  apm: 'custom-monitoring',
  capabilities: ['profiling', 'tracing'],
  env: {
    RUNTIME_TYPE: 'custom',
    CUSTOM_CONFIG: 'enabled'
  }
});

// Use with abort controller for cancellation
const controller = new AbortController();
const results = await customRunner({
  test: '@expressjs/perf-load-example',
  signal: controller.signal
});

Available Runners

Runner Description Use Case Status
@expressjs/perf-runner-local Local Node.js Default Available
@expressjs/perf-runner-vanilla Standard Node.js runtime Baseline performance measurements Available
@expressjs/perf-runner-nsolid N Solid runtime with built-in monitoring Enterprise monitoring capabilities
@expressjs/perf-runner-datadog Node.js + Datadog APM agent Datadog APM overhead analysis Help Wanted
@expressjs/perf-runner-newrelic Node.js + New Relic APM agent New Relic APM overhead analysis Help Wanted

Performance Analysis Examples

CLI-based Comparison

# Step 1: Run baseline test
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-vanilla"
# Output: results/vanilla-result-123456789.json

# Step 2: Run N|Solid test  
node --run load -- --test="@expressjs/perf-load-example" --runner="@expressjs/perf-runner-nsolid"
# Output: results/nsolid-result-123456790.json

# Step 3: Compare results
node --run compare -- results/vanilla-result-123456789.json results/nsolid-result-123456790.json

Programmatic Analysis

import fs from 'node:fs/promises';

async function runPerformanceAnalysis() {
  // Test configuration
  const tests = [
    { name: 'Express Hello World', test: '@expressjs/perf-load-example' },
    { name: 'Express with Query', test: '@expressjs/perf-load-extended-query' }
  ];

  const runners = [
    { name: 'Vanilla', runner: vanillaRunner },
    { name: 'N|Solid', runner: nsolidRunner }
  ];

  const results = {};

  // Run all combinations
  for (const test of tests) {
    results[test.name] = {};
    
    for (const { name, runner } of runners) {
      console.log(`Running ${test.name} with ${name}...`);
      
      const result = await runner({ test: test.test });
      results[test.name][name] = {
        requestsPerSec: result.clientResults.requests.mean,
        latencyP95: result.clientResults.latency.p95,
        memoryUsage: result.serverMetadata.memory
      };
    }
  }

  // Save comprehensive report
  await fs.writeFile('performance-report.json', JSON.stringify(results, null, 2));
  console.log('Performance analysis complete. Results saved to performance-report.json');
}

### Runner Development

To create a new runner, extend the shared runner infrastructure:

```javascript
// packages/runner-custom/index.mjs
import { createRunner } from '@expressjs/perf-runner-shared';

export default createRunner({
  type: 'custom',
  runtime: 'node.js',
  apm: 'custom-apm',
  capabilities: ['custom-profiling'],
  env: {
    RUNTIME_TYPE: 'custom',
    CUSTOM_SETTING: 'enabled'
  }
});

Code of Conduct

The Express Project's CoC applies to this repo.