What is resetting the PATH variable at the last second during an OpenShift v2 push hook?

Question

TL;DR: Working app, cloned it, clone doesn't start correctly from push hook (but works fine manually if I ssh in.) PATH has correct Node version added to it, but somewhere right in the last step, the incorrect Node version is prepended to the PATH again.

Path is correct here:

remote:     PATH = /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin:/var/lib/openshift/.../app-root/runtime/repo/node_modules/.bin:/var/lib/openshift/...//.node_modules/.bin:/opt/rh/nodejs010/root/usr/bin:/bin:/usr/bin:/usr/sbin

Then incorrect immediately after, somewhere in here:

remote: Starting NodeJS cartridge
remote: Tue Aug 22 2017 15:39:10 GMT-0400 (EDT): Starting application 'staging' ...

So what scripts and hooks are represented in or before these last two lines? PATH isn't just adding lines to itself...

---

I have a working OpenShift v2 app running a NodeJS version new enough to support fat arrow notation.

It appears that it was set up per Custom node.js version on Openshift because the scripts from that repo (for using a marker file) are present in .openshift.

I set up a second one using rhc create --from-app based on the working one, reset the repo, then re-deployed onto it. The second one worked great, except for the final step of starting node:

remote: npm info ok 
remote: NOTE: The .openshift/action_hooks/build hook is not executable, to make it executable:
remote:       On Windows run:   git update-index --chmod=+x .openshift/action_hooks/build
remote:       On Linux/OSX run: chmod +x .openshift/action_hooks/build
remote: Preparing build for deployment
remote: Deployment id is cedf7f51
remote: Activating deployment
remote: NOTE: The .openshift/action_hooks/deploy hook is not executable, to make it executable:
remote:       On Windows run:   git update-index --chmod=+x .openshift/action_hooks/deploy
remote:       On Linux/OSX run: chmod +x .openshift/action_hooks/deploy
remote: 
remote:   - pre_start_nodejs: Adding Node.js version 4.x.x binaries to path
remote:   - PATH set to include custom node version (4.x.x) from
remote:        /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin 
remote:     PATH = /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin:/var/lib/openshift/.../app-root/runtime/repo/node_modules/.bin:/var/lib/openshift/...//.node_modules/.bin:/opt/rh/nodejs010/root/usr/bin:/bin:/usr/bin:/usr/sbin
remote: Starting NodeJS cartridge
remote: Tue Aug 22 2017 15:39:10 GMT-0400 (EDT): Starting application 'staging' ...
remote: Waiting for application port (8080) become available ...

(Everything up to this point is exactly as it is on the working app, except for names.)

remote: Application 'staging' failed to start (port 8080 not available)
remote: -------------------------
remote: Git Post-Receive Result: failure
remote: Activation status: failure
remote: Activation failed for the following gears:
remote: ... (Error activating gear: CLIENT_ERROR: Failed to execute: 'control start' for /var/lib/openshift/.../nodejs
remote: #<IO:0x00000001cd42d0>
remote: #<IO:0x00000001cd4258>
remote: )
remote: Deployment completed with status: failure
remote: postreceive failed

rhc env and rhc app show show the settings to be identical in all the ways that they can be.

I know from other questions that the "port 8080" part above is a red herring. Plus, if I rhc ssh in and manually node www.js, it uses that port just fine and I can reach the app via browser.

So I investigated using rhc tail. I can see it failing to start, repeatedly, due to fat arrow notation:

pg.on('error', (err) => {
                      ^
SyntaxError: Unexpected token >
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
...
DEBUG: Program node ./www.js exited with code 8
DEBUG: Starting child process with 'node ./www.js'

Yet, if I rhc ssh into this same server and run node --version, I get the newer version (the same version as the other server, which both pull it from the marker file in the .openshift directory I deployed.)

I'm guessing somehow the final step in the push hook is using Node 0.10, since that's what the cartridge is named on both apps. However, if I check the path being added to PATH, a newer Node version does in fact live there.

However, this is where things get interesting. The PATH reported above (where 4.x.x is prepended) is no longer the path by the time Node is launched. I changed www.js to just spit out process.env.PATH and compared the two. The latter has these two paths added to the beginning!

/opt/rh/nodejs010/root/usr/bin
/opt/rh/v8314/root/usr/bin

What is doing that and how can I stop it? What even has a chance during these lines of output?

remote:     PATH = /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin:/var/lib/openshift/.../app-root/runtime/repo/node_modules/.bin:/var/lib/openshift/...//.node_modules/.bin:/opt/rh/nodejs010/root/usr/bin:/bin:/usr/bin:/usr/sbin
remote: Starting NodeJS cartridge
remote: Tue Aug 22 2017 15:39:10 GMT-0400 (EDT): Starting application 'staging' ...

(And why just on my second app, when I used --from-app to create it and all other settings seem to be equal between the two?)

Bushranger

I have no idea about most of this technology, but interactive ssh sessions generally parse more of ~/.bashrc etc. Perhaps PATH is only being set in that case? – Sarsen

@Sarsen Thanks, but per the OP, the push hook is correctly doing what it says, and adding the 4.x.x node bin dir to the path. The problem is that then the original node bin dir is duplicated and added to the beginning of the path right after that. – Bushranger

Answer 1

Among one of the 129 forks of the repo used originally in the app to customize it's NodeJS version, there was a comment on an issue that solved it:

Add a line to .openshift\lib\utils:

#  Add the node binary path to the PATH.
export OPENSHIFT_NODEJS_VERSION=0.6 ### this is the new line
export PATH="$node_bin_path:${PATH}"

"It's important, like you mentioned, that it's set to 0.6, EVEN IF YOU'RE NOT RUNNING 0.6."

It's also important that it's set to 0.6, even if the original Node version you're getting reverted back to isn't 0.6, either. (I tried with 0.10, with no luck. 0.6 is the magic answer, here.)

Bushranger

Answer 2

Node.js (Latest) Cartridge

@hub.openshift : Node.js Latest

By default, the Node.js version is determined by querying semver.io/node/stable.

A different URL can be specified either via NODE_VERSION_URL environment variable or by setting .openshift/NODE_VERSION_URL marker in your application repository.

For instance, you'd get the latest 0.10.x by putting this in NODE_VERSION_URL variable or .openshift/NODE_VERSION_URL marker:

https://semver.io/node/resolve/0.10

If you're using a non-default Node.js version and you're planning to scale the application across multiple gears, you must use the environment variable (learn here why).

---

Other Possible Methods

pre_build action hook workaround :

#!/bin/bash

#NODE_VERSION_URL
TARGET_NODE_VERSION=$(node -e "var p = require('$OPENSHIFT_REPO_DIR/package.json')||{engines:{}}; console.log(encodeURI(p.engines.node||''));")
echo "https://semver.io/node/resolve/$TARGET_NODE_VERSION" > ${HOME}.env/user_vars/NODE_VERSION_URL

#NPM_VERSION_URL
TARGET_VERSION=$(node -e "var p = require('$OPENSHIFT_REPO_DIR/package.json')||{engines:{}}; console.log(encodeURI(p.engines.npm||''));")
echo "https://semver.io/npm/resolve/$TARGET_VERSION" > ${HOME}.env/user_vars/NPM_VERSION_URL

---

package.json engines workaround :

Adding the .openshift folder from this repo will let you run any version of nodejs that you need using the engines section of your package.json file (see the README for usage):

"engines": {
    "node": ">= 0.12.0"
 },

Terracotta

Sorry, but I don't want to run the latest Node. I have a specific version running in an already-working rhc app that I want to work similarly in my other rhc app. So only your last suggestion applies, and in that case, the repo is the one mentioned in the question I linked to ( #27955212 ) which is already how both apps are set up--and I would like to keep it that way. The problem comes down to PATH being overridden incorrectly at the last moment in the push hook, and that's what I need help with. – Bushranger

(The reason I'm not up for changing cartridges is, if in this relatively uncomplicated clone operation things have gone so unexpectedly, I don't want to be debugging larger changes to the already-working app.) – Bushranger

(Also, the current setup almost works--the correct Node version is installed, and added to the path. It's just that the original Node version is readded to the path right after that.) – Bushranger

So I give you 3 solutions / methods to test, and it feels like you haven't even read them... (none of them force you to the latest nodejs version, all is for you to define what version...) Oh well, Sorry none of them "fit your minimum difficulty level". Maybe they will help someone else. – Terracotta

Sorry, I do appreciate your answer attempt, but on the contrary, my impression was that you hadn't read the whole OP / the title. I wanted help with the PATH issue, since that's what it boiled down to. Your third solution is a repo I'm already using (a different branch of) so I'm obviously already aware of it. Your other two are actually two parts of the same solution, and it's a solution that involves installing a different nodejs cartridge. That'd be fine if I were first setting this up, but it's clear from the OP that I want the second cloned app to work the same as the first. Thanks tho – Bushranger