Run multiple promise-returning & async functions with limited concurrency
p-limit is a concurrency control utility that restricts how many promise-returning or async functions execute simultaneously. When dealing with operations like batch API requests, file system operations, or database queries, running hundreds or thousands of promises concurrently can overwhelm servers, hit rate limits, or exhaust system memory. This package provides a simple limiter pattern to queue excess tasks until slots become available.
The package has become a foundational dependency in the JavaScript ecosystem with over 185 million weekly downloads. It's used extensively in build tools, CLI utilities, and backend services that need to process large batches of I/O operations efficiently. The implementation is deliberately minimal—no external dependencies—and focuses solely on controlling execution flow rather than providing a full-featured queue system.
p-limit is ESM-only as of version 4.0.0 and requires Node.js 20+ for the latest version 7.x. The API revolves around creating a limiter function that wraps your async operations, automatically queuing them when the concurrency limit is reached. It works seamlessly with Promise.all() and similar patterns, making it easy to integrate into existing codebases without major refactoring.
This package is specifically designed for I/O-bound operations where the bottleneck is external resources, not CPU cycles. For CPU-intensive parallel processing, Node.js worker threads are the appropriate solution. The recommended concurrency values typically range from 2-10 depending on the target system's constraints, and should be determined through empirical testing rather than arbitrary large numbers.
import pLimit from 'p-limit';
import fetch from 'node-fetch';
const limit = pLimit(3);
const urls = [
'https://api.github.com/users/sindresorhus',
'https://api.github.com/users/tj',
'https://api.github.com/users/addyosmani',
'https://api.github.com/users/paulirish',
'https://api.github.com/users/getify',
'https://api.github.com/users/kentcdodds'
];
const fetchUser = async (url) => {
console.log(`Fetching: ${url}`);
const response = await fetch(url);
const data = await response.json();
return { login: data.login, followers: data.followers };
};
const tasks = urls.map(url => limit(() => fetchUser(url)));
console.log(`Active: ${limit.activeCount}, Pending: ${limit.pendingCount}`);
const results = await Promise.all(tasks);
results.forEach(user => {
console.log(`${user.login}: ${user.followers} followers`);
});
console.log(`Completed. Active: ${limit.activeCount}, Pending: ${limit.pendingCount}`);Batch API requests with rate limiting: When fetching data from REST APIs that limit requests per second, p-limit prevents overwhelming the server while maximizing throughput. Instead of sending 1000 simultaneous requests and getting blocked, you can limit to 5 concurrent requests that complete reliably.
Parallel file system operations: Processing hundreds of files (reading, transforming, writing) benefits from concurrency control to avoid file descriptor exhaustion and memory spikes from loading too many files simultaneously. Limiting to 3-5 concurrent file operations typically provides good performance without system strain.
Database batch operations: When inserting or updating thousands of records, limiting concurrent queries prevents connection pool exhaustion and database overload. This is especially important for cloud databases with connection limits or when sharing a database across services.
Web scraping workflows: Crawling multiple pages while respecting server resources and avoiding IP bans requires controlled concurrency. p-limit allows polite scraping that won't trigger anti-bot measures while still processing pages efficiently.
Image or video processing pipelines: When processing media files through external services or local libraries, limiting concurrent operations prevents memory overflow from loading too many large files into RAM simultaneously while maintaining reasonable throughput.
npm install p-limitpnpm add p-limitbun add p-limit