artillery

Cloud-scale load testing. https://www.artillery.io

weekly downloads165K

versionv2.0.30

licenseMPL-2.0

categorytesting

load testingstress testingbenchmarkperformanceblackbox testing

Overview

Artillery is an open-source, npm-based load testing framework designed to simulate high traffic loads on APIs, web services, microservices, and stateful systems. Unlike GUI-heavy tools from the Java ecosystem, Artillery uses YAML or JSON configuration files to define test scenarios, making it particularly accessible for JavaScript developers who prefer declarative setups or programmatic control via Node.js plugins.

The toolkit operates by spawning virtual users (VUs) that execute predefined flows—sending HTTP requests, opening WebSocket connections, or interacting with Socket.io servers—while collecting real-time metrics like response times, throughput, and error rates. It supports multiple protocols out-of-the-box including HTTP/1.1, WebSockets, Socket.io, AWS Kinesis, and HLS streaming. Artillery is protocol-agnostic at the target level, meaning you can test backends written in any language (Go, Python, Ruby, etc.) as long as they expose the supported interfaces.

Developers use Artillery throughout the development lifecycle: locally during feature development to catch performance regressions, in CI/CD pipelines with SLO enforcement gates, and in production for continuous validation. Its plugin ecosystem extends core functionality with assertion libraries, custom metrics publishers, fuzzing capabilities, and per-endpoint statistics—all distributed as standard npm packages. For large-scale tests exceeding single-machine capacity, Artillery integrates with AWS Lambda and Fargate to distribute load generation without managing infrastructure.

Quick Start

typescript

// test-config.yml
config:
  target: "https://api.example.com"
  phases:
    - duration: 60
      arrivalRate: 10
      name: "Warm up"
    - duration: 120
      arrivalRate: 50
      name: "Sustained load"
  payload:
    path: "users.csv"
    fields: ["email", "password"]
  ensure:
    p95: 800
    p99: 1500

scenarios:
  - name: "User login and fetch profile"
    flow:
      - post:
          url: "/auth/login"
          json:
            email: "{{ email }}"
            password: "{{ password }}"
          capture:
            - json: "$.token"
              as: "authToken"
      - get:
          url: "/users/me"
          headers:
            Authorization: "Bearer {{ authToken }}"
          expect:
            - statusCode: 200
            - contentType: json

// Run with: npx artillery run test-config.yml
// Or programmatically:
const artillery = require('artillery');
const { run } = artillery;

run({
  config: {
    target: 'http://localhost:3000',
    phases: [{ duration: 30, arrivalRate: 5 }]
  },
  scenarios: [{
    flow: [
      { get: { url: '/health' } },
      { think: 1 },
      { post: { url: '/orders', json: { item: 'widget', qty: 2 } } }
    ]
  }]
}).then(result => {
  console.log(`Requests: ${result.aggregate.counters['http.requests']}`);
  console.log(`P95 Latency: ${result.aggregate.summaries['http.response_time'].p95}ms`);
});

Use Cases

API regression testing in CI/CD pipelines: Enforce performance SLOs by running Artillery tests on every commit, failing builds when latency percentiles exceed defined thresholds (e.g., p95 > 500ms). The ensure directive gates deployments based on measurable performance criteria rather than hoping nothing broke.

E-commerce checkout flow validation: Simulate realistic user journeys through multi-step processes—browsing products, adding to cart, completing payment—using CSV payload injection to vary test data (user credentials, product IDs) across thousands of virtual users. Artillery's transaction tracking measures end-to-end flow duration rather than isolated endpoint times.

WebSocket/real-time service stress testing: Test chat applications, live dashboards, or IoT message brokers under sustained connection load. Artillery maintains persistent WebSocket connections while sending/receiving messages according to scenario logic, measuring message delivery latency and connection stability under concurrent user spikes.

Capacity planning before traffic events: Determine infrastructure limits before Black Friday sales or product launches by ramping virtual users from baseline to peak expected load. Phase-based load profiles (ramp-up, sustain, ramp-down) reveal bottlenecks and validate auto-scaling behavior without risking production outages.

Third-party API SLA verification: Continuously monitor external dependencies by running scheduled Artillery tests against partner APIs, tracking their actual performance against contractual SLAs. JSON reports feed into observability platforms via plugins, alerting when upstream services degrade.

Pros & Cons

Pros

+Native JavaScript ecosystem integration with npm plugins and programmatic extensibility using any Node.js module
+YAML-based scenario definitions reduce boilerplate compared to code-heavy alternatives, lowering barrier for non-specialists
+Serverless scaling via AWS Lambda/Fargate eliminates infrastructure management for hyperscale tests
+Protocol versatility supports HTTP, WebSockets, Socket.io, Kinesis, and HLS without separate tooling
+Real-time metrics with CI/CD-friendly threshold enforcement fails pipelines on SLO violations automatically

Cons

−No native browser automation or JavaScript execution inside rendered pages—requires separate tooling or plugins for browser-based flows
−Serverless scaling locked to AWS ecosystem; other cloud providers require custom distributed setups
−YAML configuration can become unwieldy for extremely complex scenarios with deep branching logic
−Performance overhead from Node.js event loop may limit single-machine throughput compared to Go-based tools like k6
−Plugin quality varies as community packages lack centralized vetting or consistent maintenance standards

Alternatives

k6v0.0.0

Dummy package for autocompleting k6 scripts.

testing

155K / week

Comparisons

artillery vs k6

testing

Install

bash

npm install artillery

bash

pnpm add artillery

bash

bun add artillery

Overview

Quick Start

typescript

// test-config.yml
config:
  target: "https://api.example.com"
  phases:
    - duration: 60
      arrivalRate: 10
      name: "Warm up"
    - duration: 120
      arrivalRate: 50
      name: "Sustained load"
  payload:
    path: "users.csv"
    fields: ["email", "password"]
  ensure:
    p95: 800
    p99: 1500

scenarios:
  - name: "User login and fetch profile"
    flow:
      - post:
          url: "/auth/login"
          json:
            email: "{{ email }}"
            password: "{{ password }}"
          capture:
            - json: "$.token"
              as: "authToken"
      - get:
          url: "/users/me"
          headers:
            Authorization: "Bearer {{ authToken }}"
          expect:
            - statusCode: 200
            - contentType: json

// Run with: npx artillery run test-config.yml
// Or programmatically:
const artillery = require('artillery');
const { run } = artillery;

run({
  config: {
    target: 'http://localhost:3000',
    phases: [{ duration: 30, arrivalRate: 5 }]
  },
  scenarios: [{
    flow: [
      { get: { url: '/health' } },
      { think: 1 },
      { post: { url: '/orders', json: { item: 'widget', qty: 2 } } }
    ]
  }]
}).then(result => {
  console.log(`Requests: ${result.aggregate.counters['http.requests']}`);
  console.log(`P95 Latency: ${result.aggregate.summaries['http.response_time'].p95}ms`);
});

Use Cases

Pros & Cons

Pros

+Native JavaScript ecosystem integration with npm plugins and programmatic extensibility using any Node.js module
+YAML-based scenario definitions reduce boilerplate compared to code-heavy alternatives, lowering barrier for non-specialists
+Serverless scaling via AWS Lambda/Fargate eliminates infrastructure management for hyperscale tests
+Protocol versatility supports HTTP, WebSockets, Socket.io, Kinesis, and HLS without separate tooling
+Real-time metrics with CI/CD-friendly threshold enforcement fails pipelines on SLO violations automatically

Cons

−No native browser automation or JavaScript execution inside rendered pages—requires separate tooling or plugins for browser-based flows
−Serverless scaling locked to AWS ecosystem; other cloud providers require custom distributed setups
−YAML configuration can become unwieldy for extremely complex scenarios with deep branching logic
−Performance overhead from Node.js event loop may limit single-machine throughput compared to Go-based tools like k6
−Plugin quality varies as community packages lack centralized vetting or consistent maintenance standards