Deployed a Next.js application to App Engine Standard [Nodejs] and got a 500 error

Question

I deployed to app engine with nodejs8 runtime and got a 500. Im deploying a next.js application, and upon reviewing StackDriver I get. It appears .next might be getting ignored. The error is as follows:

throw new Error("Could not find a valid build in the '".concat(this.distDir, "' directory! Try building your app with 'next build' before starting the server."));

Error: Could not find a valid build in the '/srv/build' directory! Try building your app with 'next build' before starting the server. at Server.readBuildId (/srv/node_modules/next/dist/server/next-server.js:753:15) at new Server (/srv/node_modules/next/dist/server/next-server.js:80:25) at module.exports (/srv/node_modules/next/dist/server/next.js:6:10) at Object.<anonymous> (/srv/server.js:10:13) at Module._compile (module.js:653:30) at Object.Module._extensions..js (module.js:664:10) at Module.load (module.js:566:32) at tryModuleLoad (module.js:506:12) at Function.Module._load (module.js:498:3) at Function.Module.runMain (module.js:694:10)

My package.json file looks like:

   {
  "name": "emails",
  "private": true,
  "version": "1.0.0",
  "main": "server.js",
  "scripts": {
    "dev": "NODE_ENV=development node server.js",
    "build": "next build",
    "lint": "standard",
    "prestart": "next build", 
    "start": "NODE_ENV=production node server.js",
    "appspot-deploy": "gcloud app deploy --project=email-app-219521",
    "deploy": "gcloud app deploy"
  },
  "standard": {
    "parser": "babel-eslint"
  },
  "license": "ISC",
  "dependencies": {
    "@firebase/app-types": "^0.3.2",
    "@material-ui/core": "^3.2.0",
    "@material-ui/icons": "^3.0.1",
    "@zeit/next-sass": "^1.0.1",
    "body-parser": "^1.18.3",
    "express": "^4.16.3",
    "express-rate-limit": "^3.2.1",
    "express-session": "^1.15.6",
    "firebase-admin": "^6.0.0",
    "isomorphic-unfetch": "^3.0.0",
    "memorystore": "^1.6.0",
    "next": "^7.0.1",
    "node-sass": "^4.9.3",
    "react": "^16.5.2",
    "react-dom": "^16.5.2",
    "styled-jsx-plugin-sass": "^0.3.0"
  },
  "devDependencies": {
    "babel-eslint": "^10.0.1",
    "eslint": "^5.6.1",
    "webpack": "^4.20.2"
  },
  "engines": {
    "node": "8.x.x"
  }
}

And my app.yaml file looks like:

runtime: nodejs8
env_variables:
  NODE_ENV: production
handlers:
- url: /.*
  script: server.js

And Im serving up my project with express on port 8080.

Subminiature

Can you look at the stdout and stderr logs of your application in Stackdriver Loggingat console.cloud.google.com/logs/viewer? Make sure to select "GAE Application" in the resource picker. Please update your post with what you see in the logs. gcloud alpha services list --available and your screenshots do not seem to be related to App Engine. – Searby

We had similar problems, and this more recent answer from vercel did the job: github.com/vercel/next.js/discussions/12474 Turns out GAE does not like the .next directory, so it needs to look for a build directory instead. Check the link for additional setup. – Assail

Answer 1

When your application is returning 500 errors, make sure to look at the stdout and stderr logs of your application in Stackdriver Logging at https://console.cloud.google.com/logs/viewer. Double check that you are looking at "GAE Application" resource picker.

Looking at the error message, it seems that the .next folder does not exist in your app. This .next folder is a folder that is usually generated via a "build step", and I see indeed that you have "build": "next build" as a script in your package.json.

You should not be using prestart to perform this build step, first because App Engine does not run prestart on instance startup, but also because in general this would be bad for performances.

You have two ways to create this folder:

1. Generate .next on your machine before deploying, to do so, you can change your deploy script to be: "deploy": "npm run build && gcloud app deploy". (And also make sure that the .gcloudignore file does not contain .next or does not include the content of the .gitignore file)

2. Run this build step on Google Cloud's servers after deploying: The Node.js App Engine runtime will execute any gcp-build script if present. This means that you can add this script: "gcp-build": "npm run build" to your package.json. When doing so, I recommend you to add .next to the .gcloudignore file so that the .next folder does not get uploaded at deployment time.

In addition, note that you can simplify your app.yaml to just runtime: nodejs8:

• NODE_ENV is automatically set to production, you can remove it
• Your handlers section is equivalent to the default one.

Searby

Thanks Steren, Im up and running now after changing my deploy script to what you suggested and cleaned up the other items you mentioned. I've seen various examples out there to include one person using the prestart build stepp for GAE as well as another stating that .next is ignored and requires a next.config key value of distDir: 'build'. Neither of those solutions worked. – Subminiature

I ended up here because I can't quite get the gcp-build & next.js build working either. In fact, I can't even determine from looking at verbose deploy logs and stackdriver logging that gcp-build even executes. It would be fairly valuable if this works as described in cloud.google.com/blog/products/application-development/… but i'm not sure how to debug it further given the limited amount of documents about it online. – Attainture

Answer 2

I've just managed to make it work. Here's what I found:

First of all, it doesn't work with Yarn. It seems gcp-build command does not run when there is yarn.lock file.

Now in package.json, use following scripts:

  "scripts": {
    "gcp-build": "next build",
    "start": "next start -p $PORT"
  }

And make sure to declare Next and all modules needed in next.config.js as dependencies (not devDependencies)

FYI I only have runtime: nodejs10 in my app.yaml and only scripts I have are in pages folder and next.config.js

Beckybecloud

Answer 3

This happened to me when I removed a file that one of my pages (in the /pages folder) happened to be referencing.

Either restore that file, or remove the file that is referencing the missing file.

Nephritis