GitLab CI: How can I reuse installed npm packages between jobs?

Question

I have a GitLab Pages site that uses Gulp for building. My .gitlab-ci.yml file looks similar to this:

image: node:latest

before_script:
  - npm install gulp-cli -g
  - npm install gulp [...and a whole bunch of packages] --save-dev

build:
  stage: build
  script:
  - gulp buildsite
  artifacts:
    paths:
    - public

pages:
  stage: deploy
  script:
  - gulp
  artifacts:
    paths:
    - public

cache:
  paths:
  - node_modules/

Before both the build and pages jobs, the npm install commands are executed (once before each job). Since I have quite a few packages, this usually takes a while.

Is there a way to only do the installs once across the entire build?

I thought that's what cache was supposed to help with, but it still seems like it still redownloads everything.

Patronymic

The cache system doesn't have any guarantees, if you need the node_modules to be sent between jobs 100% of the time, use artifacts https://mcmap.net/q/539901/-gitlab-deploy-job-fetches-changes-from-git You can also add an expiry to the artifacts so they're not kept on your server forever – Gangster

Answer 1

Though the answer in the comments is essentially correct. I think a specific answer for your case would be good though. One approach you could use is adding a third stage that would bear the load of installing node modules plus you could also cache them to speed up subsequent builds:

image: node:latest

stages:
  - prep
  - build
  - deploy  

before_script:
  - npm install gulp-cli -g  

prep:
  stage: prep
  script:
  - npm install gulp [...and a whole bunch of packages] --save-dev
  artifacts:
   paths:
   - node_modules 
  cache:
   paths:
   - node_modules

build:
  stage: build
  script:
  - gulp buildsite
  artifacts:
    paths:
    - public

pages:
  stage: deploy
  script:
  - gulp
  artifacts:
    paths:
    - public

This solution will only perform the installation once and will cache the result for future ci pipelines, you may also place an expiry on the node modules artifacts.

Presumptuous

BUG: artifacts will be uploaded by HTTP, so a huge node_modules will result to CI failed by uploading entity too large. – Royalroyalist

Answer 2

You need to set cache: untracked: true to really cache the files not tracked by Git.

cache:
  untracked: true
  paths:
      - node_modules/

Bullshit