Using envsubst for a Meteor app with bitbucket pipelines

Question

I have the following bitbucket pipeline, which is for deploying a Meteor app with Mup.

Everything is working up until the final mup deploy command it seems to not to be able to read the generated config.json file that is supposed to have my secure mongo url injected into it.

this is what the secure_mongo.json file looks like

{
    "secret": $STAGING_MONGO_URL
}

In the mup file I access like

var mongo = require('./config.json');

module.exports = {
  MONGO_URL: mongo.secret,
}

image: node:14.16.0

pipelines:
  branches:
    staging:
      -  step:
           name: Deploy to staging CI/CD Environment
           script:
             - mkdir -p ~/.ssh
             - apt-get update && apt-get install gettext-base
             - curl https://install.meteor.com/ | sh
             - export METEOR_ALLOW_SUPERUSER=true
             - cd .bot-staging-ci-cd
             - (umask 077 ; echo $DO_STAGING_CICD_SSH_KEY | base64 --decode > ~/.ssh/id_rsa)
             - cat secure_mongo.json | envsubst > config.json
             - cat config.json && realpath config.json
             - npm install -g mup
             - npm install -g ssh2
             - mup deploy

And I do have a secure variable named $STAGING_MONGO_URL in the repository. So Not entirely sure what is going wrong. Any help would be great.

This question is related and helped me get this far Storing secrets into Bitbucket Pipelines and then deploy on App Engine? but it is not the same question.

To be clear the error I am getting is Mup deploy is saying that the config.json file is reaching an Unexpected token

exact error

Error loading config file:
SyntaxError: /opt/atlassian/pipelines/agent/build/.bot-staging-ci-cd/config.json: Unexpected token m in JSON at position 13

UPDATE:

So I thought the json error had to do with the require of the config.json file and tried to write use envsubst directly with a the module.exports .js file but it doesn't seem to replace the variable at all, Mup fails directly at the position where the variable should be replaced.

Thanks

Rackety

Does the config.json file look like it should? What is the full error message? – Fleenor

Well the config.json file looks identical to the secure_mongo.json file when I download the raw and view the cat I read in that question I linked in a comment that even with the cat command it wont display the secret with bitbucket pipelines, but I thought it would since it is a brand new written file. I will add the exact error in the question since it is too long for this comment. – Rackety

Answer 1

Unless your STAGING_MONGO_URL variable starts and ends with double quotes, once it is rendered into config.json by envsubst, that file is no longer valid json.

Wrap the value placeholder with double quotes in your secure_mongo.json template

{
    "secret": "$STAGING_MONGO_URL"
}

Also, be careful to avoid actual double quotes in the secret value, because once rendered it would break json parsing again.

Even if you successfully render the config.json file it is useless to print it to stdout: a Bitbucket security feature will prevent you from printing your secret variable in the pipelines logs E.g. see:

• Bitbucket pipeline replacing text with variable?
• Bitbucket: Show value of variables marked as secret

Orban

So I ended up putting the string wrap in the environment variable in bitbucket. I guess that is the wrong way to do it but it worked. – Rackety

Yeah, that is a quite bad solution. If the app accidentally prints the parsed URL in the logs, Bitbucket won't mask the value because it will not match the one stored with wrapping quotes. – Orban

Ok very good to know I will fix that. Thank you – Rackety

Answer 2

Also for anyone trying to get this to work with Meteor this worked for me

image: node:14

pipelines:
  branches:
    staging:
      -  step:
           name: Deploy to staging CI/CD Environment
           script:
             - mkdir -p ~/.ssh
             - apt-get update && apt-get install gettext-base
             - curl https://install.meteor.com/ | sh
             - export METEOR_ALLOW_SUPERUSER=true
             - cd .bot-staging-ci-cd
             - (umask 077 ; echo $DO_STAGING_CICD_SSH_KEY | base64 --decode > ~/.ssh/id_rsa)
             - cat mup-secure.js | envsubst > mup.js
             - cat settings-secure.json | envsubst > settings.json
             - rm -rf node_modules
             - npm install -g mup
             - npm install -g ssh2
             - meteor npm i
             - mup setup
             - mup deploy --settings settings.json

Rackety