Manual Distributed Task Execution on Github Actions

Using Nx Agents is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up distributed task execution on your own CI provider using the recipe below.

Run Custom Agents on GitHub

Our reusable GitHub workflow represents a good set of defaults that works for a large number of our users. However, reusable GitHub workflows come with their limitations.

If the reusable workflow above doesn't satisfy your needs you should create a custom workflow. If you were to rewrite the reusable workflow yourself, it would look something like this:

.github/workflows/ci.yml
1name: CI 2on: 3 push: 4 branches: 5 - main 6 pull_request: 7 8# Needed for nx-set-shas when run on the main branch 9permissions: 10 actions: read 11 contents: read 12 13env: 14 NX_CLOUD_DISTRIBUTED_EXECUTION: true # this enables DTE 15 NX_BRANCH: ${{ github.event.number || github.ref_name }} 16 NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }} 17 NPM_TOKEN: ${{ secrets.NPM_TOKEN }} # this is needed if our pipeline publishes to npm 18 19jobs: 20 main: 21 name: Nx Cloud - Main Job 22 runs-on: ubuntu-latest 23 steps: 24 - uses: actions/checkout@v4 25 name: Checkout [Pull Request] 26 if: ${{ github.event_name == 'pull_request' }} 27 with: 28 # By default, PRs will be checked-out based on the Merge Commit, but we want the actual branch HEAD. 29 ref: ${{ github.event.pull_request.head.sha }} 30 # We need to fetch all branches and commits so that Nx affected has a base to compare against. 31 fetch-depth: 0 32 filter: tree:0 33 34 - uses: actions/checkout@v4 35 name: Checkout [Default Branch] 36 if: ${{ github.event_name != 'pull_request' }} 37 with: 38 # We need to fetch all branches and commits so that Nx affected has a base to compare against. 39 fetch-depth: 0 40 filter: tree:0 41 42 # Set node/npm/yarn versions using volta 43 - uses: volta-cli/action@v4 44 with: 45 package-json-path: '${{ github.workspace }}/package.json' 46 47 - name: Use the package manager cache if available 48 uses: actions/setup-node@v3 49 with: 50 node-version: 20 51 cache: 'npm' 52 53 - name: Install dependencies 54 run: npm ci 55 56 - name: Check out the default branch 57 run: git branch --track main origin/main 58 59 - name: Initialize the Nx Cloud distributed CI run and stop agents when the build tasks are done 60 run: npx nx-cloud start-ci-run --distribute-on="manual" --stop-agents-after=e2e-ci 61 62 - name: Check the formatting 63 run: npx nx-cloud record -- nx format:check 64 65 - name: Lint, test, build, and run e2e 66 run: npx nx affected -t lint,test,build,e2e-ci --configuration=ci 67 68 agents: 69 name: Agent ${{ matrix.agent }} 70 runs-on: ubuntu-latest 71 strategy: 72 matrix: 73 # Add more agents here as your repository expands 74 agent: [1, 2, 3] 75 steps: 76 - name: Checkout 77 uses: actions/checkout@v4 78 79 # Set node/npm/yarn versions using volta 80 - uses: volta-cli/action@v4 81 with: 82 package-json-path: '${{ github.workspace }}/package.json' 83 84 - name: Use the package manager cache if available 85 uses: actions/setup-node@v3 86 with: 87 node-version: 20 88 cache: 'npm' 89 90 - name: Install dependencies 91 run: npm ci 92 93 - name: Start Nx Agent ${{ matrix.agent }} 94 run: npx nx-cloud start-agent 95 env: 96 NX_AGENT_NAME: ${{ matrix.agent }} 97

There are comments throughout the workflow to help you understand what is happening in each section.