How to use private, self hosted NPM package with Google App Engine node, standard environment

Question

I have an NPM package hosted on a private Bitbucket git repo (not in the official NPM registry).

I have this in my package.json, under the "dependencies" key:

"a-private-package" git+ssh://[email protected]:myusername/a-private-package.git

It works when I run npm install locally as my SSH keys are used.

But when I use gcloud app deploy to deploy to the app engine standard environment for node, I get a Host key verification failed from Google Cloud Build.

I have tried:

Adding a custom SSH key to Cloud Build.

https://cloud.google.com/cloud-build/docs/access-private-github-repos

Issue: No access to cloudbuild.yaml for GAE standard; cannot tell git to use the SSH key.

Adding my private git repo to Google Sources.

Issue: No access to cloudbuild.yaml for GAE standard; cannot tell git to use the SSH key.

npm pack; npm install

Issue: Does not keep repo history/URL.

Is it actually possible?

Singultus

Have you taken a look at this Article? It mentions the usage of the cloudbuild.yaml with GAE standard and linking Bitbucket to it. – Officious

I haven't, but will take a look. I do not think it would work as the example calls gcloud app deploy from the build, which would start another fresh build without the SSH key installed in the layers before it? – Singultus

Well, the article is running the gcloud app deploy after he defines the cloudbuild.yaml. I see you are obtaining a Host key verification failed error from Cloud Build and adding the SSH keys to the .yaml file could fix this but you also say you can not modify the cloudbuild.yaml. Why are you not able to modify the cloudbuild.yaml? – Officious

gcloud app deploy will trigger a new fresh build (from the first build), with an empty cloudbuild.yaml. I think this is the core issue (that you cannot customize the cloudbuild.yaml that gcloud app deploy uses). If you view the comments of that article, someone else is having this same issue. Thanks for the link though. – Singultus

@Singultus did you ever find a solution to this? – Nutshell

Answer 1

It is not possible to modify the cloudbuild.yaml when you are running gcloud app deploy. Instead, you must create a new cloudbuild.yaml and execute it with gcloud builds submit --config=cloudbuild.yaml . In this case, the gcloud app deploy is going to be executed inside the cloudbuild.yaml.

I have tried the steps described for connecting to a private Github repository and changing the values to fit it with bitbucket, but was not able to. Thus, I have created this Feature Request for better documentation

---

Using Cloud Source Repositories

I believe that, as you already have a dependency on the private repo, then it will be simpler to host you entire app on it. Given this, you will have to clone the entire repo, run npm install and deploy.

In this case, Cloud Source Repositories has a built in feature to mirror directly to Bitbucket (public and private repos).

Steps:

1) Create on your app root folder a cloudbuild.yaml with the following code:

steps:
# NPM install
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
#Test
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
#Deploy
- name: "gcr.io/cloud-builders/gcloud"
  args: ["app", "deploy"]

2) Connect Cloud Source Repositories to Bitbucket

3) Create a Cloud Build Trigger (so new code pushed to the repo will be automatically deployed)

4) Push the root folder containing the app.yaml and the cloudbuild.yaml to the repo

It should now be Synced to Cloud Source Repositories and it should trigger the Cloud Build for the deploy.

Officious

Answer 2

Unfortunately you will need to embed a username/password in the package.json, but you can probably use the https endpoint:

"a-private-package": "git+https://myusername:[email protected]/myusername/a-private-package.git"

If this works for you I'd suggest creating a separate account on bitbucket and restricting it to view-only on that repo.

Danas