Parallelization and distribution

Affected calculations and remote caching remove work that doesn't need to run. Nx still needs to execute cache misses, and the task graph determines which of those tasks can run at the same time.

Nx can run independent tasks as parallel processes on one machine or distribute them across multiple machines. Both approaches preserve task dependencies.

Parallelization on one machine

Nx parallelizes ready tasks whenever you run a target. This applies to individual project commands, run-many, and affected commands. For example, Nx can build independent project dependencies concurrently before building the project that depends on them.

Set the workspace-wide process limit with parallel in nx.json. Use --parallel=<number> to change it for one command:

nx affected -t build --parallel=2

A single machine is the least complex option, and its logs and artifacts stay in one place. Its CPU and memory limit the number of useful parallel processes, so CI duration grows once the task graph exceeds that capacity.

Characteristic	Result	Notes
Configuration	Pro	CI uses the same Nx commands as local development.
Debugging	Pro	Logs and artifacts stay on one machine.
Scale	Con	One machine limits CPU, memory, and useful parallelism.

Distribution across machines

Distribution adds compute by running tasks on multiple CI jobs. The main job determines the tasks to run, while agent jobs execute assigned tasks. You can maintain that assignment yourself or use Nx Agents.

Manual distribution

A common manual strategy divides work into fixed bins, often by target:

# Get affected projects and make the list available to agent jobs.
- nx show projects --affected --json > affected-projects.json
- node store-affected-projects.js

- nx run-many -t lint --projects=$PROJECTS

- nx run-many -t test --projects=$PROJECTS

- nx run-many -t build --projects=$PROJECTS

Fixed bins often finish at different times, leaving some machines idle. They can also duplicate work when a task in one bin depends on a task assigned to another bin. Remote caching reduces some duplication, but scripts still need to coordinate task order and cache availability.

The ideal bins change as the project graph and affected set change. Maintaining that logic becomes a CI responsibility.

Characteristic	Result	Notes
Configuration	Con	Custom scripts assign tasks and require maintenance.
Debugging	Con	Logs and artifacts start on separate machines.
Scale	Pro	More machines provide more CPU and memory.

Distribution with Nx Agents

Nx Agents dynamically assign ready tasks from the task graph to available agent machines. The main job keeps the same Nx commands used for a single-machine pipeline:

# Start eight agents and stop them after build tasks finish.
- npx nx start-ci-run --distribute-on="8 linux-medium-js" --stop-agents-after=build
- nx affected -t lint test build

Nx Agents account for task dependencies and the affected set on each CI run. They use remote caching to move artifacts between agents and collate results on the main job. Dynamic assignment reduces the idle time caused by fixed bins without requiring a custom scheduler.

Characteristic	Result	Notes
Configuration	Pro	Existing Nx task commands remain unchanged.
Debugging	Pro	Nx collates logs and artifacts on the main job.
Scale	Pro	Dynamic assignment uses available agents across the current task graph.

Use Nx Agents when one machine no longer provides enough parallelism and maintaining manual distribution isn't worthwhile. You can generate a workflow with the CI workflow generator or follow a CI setup guide.