Customize AWS ElasticBeanstalk NodeJS Install (use yarn)

Question

Isit possible to configure EBS to install my NodeJS application using yarn package manager instead of NPM?

Urbanity

I've decided to install Node packages using Yarn before uploading the archive to Elastic Beanstalk and include node_modules to the archive. The presence of node_modules will make NPM skip installation. It's easier and more reliable than the solutions in the answers. – Offer

@Offer does this prevent Elastic Beanstalk to run npm install and override content of your node_modules folder? – Pakistan

@Pakistan It doesn't prevent Elastic Beanstalk from running npm install. It makes the NPM do nothing. – Offer

Answer 1

I've figured out a way, but it is a little hacky.

1. Create a .ebextensions/yarn.config file. (The name does not have to be 'yarn'.)

2. Put this content into the file:

   files:
   # Runs right before `npm install` in '.../50npm.sh'
   "/opt/elasticbeanstalk/hooks/appdeploy/pre/49yarn.sh" :
       mode: "000775"
       owner: root
       group: users
       content: |
           #!/bin/bash
   
           app="$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)";
   
           # install node
           curl --silent --location https://rpm.nodesource.com/setup_8.x | bash -;
   
           # install yarn
           curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | tee /etc/yum.repos.d/yarn.repo;
           yum -y install yarn;
   
           # install node_modules with yarn
           cd "${app}";
           yarn --production;
   

This ebextension creates a file which does 3 things:

1. Installs node.
2. Installs yarn.
3. Installs node_modules with yarn.

In order to make Elastic Beanstalk run yarn install before it runs npm install, the file is created under /opt/elasticbeanstalk/hooks/appdeploy/pre. This turns the file into a pre-deployment hook, which means that Elastic Beanstalk will run it during the first phase of deployment. By default, there is another file in this directory called 50npm.sh, which runs npm install. Since Elastic Beanstalk runs the files in this directory alphabetically, 49yarn.sh (our file) will run before 50npm.sh (the default file), resulting in yarn install running before npm install.

One potential problem is that the environment variables set in the Elastic Beanstalk UI (under Configuration > Software Configuration) are not available at this point of the deployment phase. This is a big problem if you have an npm auth token there which you use to install private npm modules.

Another potential problem is that this installs node manually, so the "Node version" you specify in the Elastic Beanstalk UI (under Configuration > Software Configuration) will have no effect on the version of node your application uses; you need to specify it in this ebextension. Elastic Beanstalk's 50npm.sh both installs node and runs npm install. Since we have to run yarn install before that file runs, we also have to install node manually. Then, when Elastic Beanstalk goes to install node, it detects that node is already installed but does not verify that it is the correct version, so it skips the node installation.

For reference, the yarn installation instructions came from here: https://yarnpkg.com/docs/install#linux-tab

Ostrogoth

Is "/tmp/deployment/application" just a placeholder or is it an actual directory involved in the beanstalk deployment? – Bastogne

Yeah /tmp/deployment/application is where beanstalk puts the project code. Beanstalk deletes the folder and recreates it during every deployment. Amazon probably chose to put it in /tmp because, as a UNIX convention, /tmp is deleted whenever the system shuts down. – Ostrogoth

This works like a charm! The only suggestion I have is instead of hardcoded /tmp/deployment/application to use /opt/elasticbeanstalk/bin/get-config container -k app_staging_dir to get the temporary folder containing the project. – Fortuitism

I updated the post to get the application directory dynamically by using that command. – Ostrogoth

This looks good, but I don't understand why you are running the Node installation command from rpm.nodesource.com. Are you just trying to upgrade to the latest version of Node? EB will have already installed Node by the time your 49yarn.sh runs. – Unvoiced

I did that because 50npm.sh both installs node and runs npm install; EB didn't split those 2 processes into 2 scripts. So if you run the ebextension above after 50npm.sh, the app will install the node modules once in 50npm.sh and again in the ebextension, which defeats the purpose of using yarn in the first place. If EB were to split those two processes into multiple scripts, then yes it would be much better to let EB install node instead of doing it in this ebextension. – Ostrogoth

To add to the above, it should be pointed out that this will override EB's NodeVersion config, since you're installing Node 6. I just spent ages trying to work out why it wasn't using Node 8, only to realise it's that curl line! – Pediculosis

Thanks, I updated the answer to include a warning for that. – Ostrogoth

Answer 2

I did this following instructions on https://yarnpkg.com/lang/en/docs/install/

commands:
  01_install_yarn:
    command: "sudo wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo && curl --silent --location https://rpm.nodesource.com/setup_6.x | sudo bash - && sudo yum install yarn -y"

Childbearing

Which file do you add this to? – Cimabue

Answer 3

This way that i came up with lets you still control the node version via the Elastic Beanstalks Dashboard.

Thanks for this question! couldn't have come to this solution without it :D

 "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh":
        mode:    "000755"
        owner:   root
        group:   users
        content: |
           #!/usr/bin/env bash
           #
           # Prevent installing or rebuilding like Elastic Beanstalk tries to do by
           # default.
           #
           # Note that this *overwrites* Elastic Beanstalk's default 50npm.sh script
           # (https://gist.github.com/wearhere/de51bb799f5099cec0ed28b9d0eb3663).
    "/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh":
        mode:    "000755"
        owner:   root
        group:   users
        content: |
           #!/usr/bin/env bash
           #
           # Prevent installing or rebuilding like Elastic Beanstalk tries to do by
           # default.
           #
           # Note that this *overwrites* Elastic Beanstalk's default 50npm.sh script.
           # But their default script actually doesn't work at all, since the app
           # staging dir, where they try to run `npm install`, doesn't exist during
           # config deploys, so ebnode.py just aborts:
           # https://gist.github.com/wearhere/de51bb799f5099cec0ed28b9d0eb3663#file-ebnode-py-L140
    "/opt/elasticbeanstalk/hooks/appdeploy/pre/49yarn.sh" :
        mode:    "000775"
        owner:   root
        group:   users
        content: |
           tmp="$(mktemp || bail)";
           app="$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)";
           version="$(/opt/elasticbeanstalk/bin/get-config optionsettings -n aws:elasticbeanstalk:container:nodejs -o NodeVersion)";
           echo $version
           major="$(cut -d'.' -f1 <<<${version})"
           yum -y install python26 python26-libs
           wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo;

           wget "https://rpm.nodesource.com/pub_${major}.x/el/7/x86_64/nodejs-${version}-1nodesource.x86_64.rpm" -O "${tmp}";
           rpm -i --nosignature --force "${tmp}";
           rm -f "${tmp}";

           yum -y install yarn;

           cd "${app}";
           yarn --production;

Tillo

Answer 4

Had to revisit this as we couldn't figure out why we were stuck on node 8 even though we set it to node 12 in the EB UI. Seems that if you install a global node it overrides the version setting. Instead of installing a global node, this uses the Elastic Beanstalk node install and adds it to the path. You have to add the PATH in again at the start of your yarn install script but it seems to be the least invasive way to use yarn.

content: |
  #!/usr/bin/env bash
  set -euxo pipefail

  EB_NODE_VERSION=$(/opt/elasticbeanstalk/bin/get-config optionsettings -n aws:elasticbeanstalk:container:nodejs -o NodeVersion)
  echo "EB node version: $(EB_NODE_VERSION)"

  # Make sure Node binaries can be found (required to run npm).
  # And this lets us invoke npm more simply too.
  export PATH=/opt/elasticbeanstalk/node-install/node-v$EB_NODE_VERSION-linux-x64/bin:$PATH

  if yarn -v; then
    echo 'Yarn already installed.'
  else
    echo 'Installing yarn...'
    npm install yarn -g
  fi

Desuetude

Where can i put this file? Does it need any special filename? – Associationism

Answer 5

Since get-config is no longer present in the new Amazon Linux 2 platform, we had to figure another clean way to do this, and came up with the following :

container_commands:
  01_npm_install_yarn:
    command: "npm install -g yarn"
  10_yarn_install:
    command: 'PATH="$PATH:$(dirname $(readlink $(which node)))" yarn install'

You may want to put the PATH= logic in a script and call it before every yarn command, to have clean command: instructions in your extentions.

Also, note that if you install yarn using the yum package manager, you completely break the NodeJS version management provided by Beanstalk (since it the black magic running behind make some symlinks in /bin and /usr/bin).

Jus

Answer 6

An easy way to prevent EB from running npm install is to create an empty node_modules folder in a prebuild hook:

Edit your_project/.platform/hooks/prebuild/prevent-npm.sh:

#!/bin/bash
# EB build scripts will not install using npm if node_modules folder exists
mkdir node_modules

This way, EB will still install Node.js, so you don't have to install that yourself. But it will skip npm install when node_modules exists.

Then inside the predeploy hook, you can run yarn. If you are running Node.js 16 or newer, you can use corepack yarn, and it will just work (you don't have to install yarn from yum, npm or anything else.)

Edit your_project/.platform/hooks/predeploy/yarn.sh:

#!/bin/bash
corepack yarn

Update: As of Amazon Linux 2023, corepack is no longer included, and needs to be explicitly installed:

Edit your_project/.platform/hooks/prebuild/corepack.sh:

#!/bin/bash
npm i -g corepack

Be sure to also create symlinks under confighooks in your project, or else it will fail to build when changing config (e.g. updating environment values):

mkdir -p .platform/confighooks/{prebuild,predeploy}
ln -s ../../hooks/predeploy/yarn.sh .platform/confighooks/predeploy/yarn.sh
ln -s ../../hooks/prebuild/prevent-npm.sh .platform/confighooks/prebuild/prevent-npm.sh
ln -s ../../hooks/prebuild/corepack.sh .platform/confighooks/prebuild/corepack.sh

Journalese

Answer 7

# place in .platform/hooks/prebuild/yarn.sh

#!/bin/bash

# need to install node first to be able to install yarn (as at prebuild no node is present yet, node version: https://github.com/nodesource/distributions)
sudo curl --silent --location https://rpm.nodesource.com/setup_12.x | sudo bash -
sudo yum -y install nodejs

# install yarn
sudo wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo
sudo yum -y install yarn

# install
cd /var/app/staging/

# debugging..
ls -lah

yarn install --prod

chown -R webapp:webapp node_modules/ || true # allow to fail

Edik