ChunkLoadError: Loading chunk XY failed. - Randomly getting fatal on PRODUCTION

Question

we have got our ecommerce platform already in production and we are experiencing weird ChunkLoadError. This error happens randomly, and is not replicable. When we are trying to open failed file it is there and can be loaded normaly.

If user get's this error, he get's white screen (logicaly) but after refresh everything is fine.

We are running SSR ecommerce on React (latest), Express (latest)

our webpack / razzle config

const path = require('path');
const autoprefixer = require('autoprefixer');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const CompressionPlugin = require('compression-webpack-plugin');
const LoadablePlugin = require('@loadable/webpack-plugin');

const LodashModuleReplacementPlugin = require('lodash-webpack-plugin');

const BundleAnalyzerPlugin = require('webpack-bundle-analyzer').BundleAnalyzerPlugin;
const TerserPlugin = require('terser-webpack-plugin');
// const DuplicatePackageCheckerPlugin = require('duplicate-package-checker-webpack-plugin');

module.exports = {
  modify: (baseConfig, env, webpack) => {
    const { target, dev } = env;
    const appConfig = { ...baseConfig };
    // Setup SCSS
    if (target === 'web') {
      const filename = path.resolve(__dirname, 'build');

      const cssLoader = {
        loader: 'css-loader',
        options: {
          minimize: !dev,
          sourceMap: false,
          importLoaders: 1
        }
      };

      const postCSSLoader = {
        loader: 'postcss-loader',
        options: {
          ident: 'postcss',
          sourceMap: false,
          plugins: () => [
            autoprefixer({
              browsers: [
                '>1%',
                'last 4 versions',
                'Firefox ESR',
                'not ie < 9' // React doesn't support IE8 anyway
              ]
            })
          ]
        }
      };

      const sassLoader = {
        loader: 'sass-loader',
        options: {
          minimize: !dev,
          sourceMap: false,
          importLoaders: 1
        }
      };

      if (dev) {
        appConfig.output.filename = 'static/js/[name].js';
        appConfig.module.rules.push({
          test: /\.scss$/,
          use: ['style-loader', cssLoader, postCSSLoader, sassLoader]
        });
      } else {
        appConfig.output.filename = 'static/js/[name].[chunkhash:8].js';

        // For production, extract CSS
        appConfig.module.rules.push({
          test: /\.scss$/,
          use: [MiniCssExtractPlugin.loader, cssLoader, postCSSLoader, sassLoader]
        });

        appConfig.plugins.push(
          new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/),
          new webpack.IgnorePlugin(/moment/, /react-kronos/),
          new webpack.optimize.OccurrenceOrderPlugin(),
          // new webpack.optimize.LimitChunkCountPlugin({maxChunks: 50}),
          new CompressionPlugin()
          ,new BundleAnalyzerPlugin({
            analyzerMode: 'static',
            generateStatsFile: true,
            openAnalyzer: false
          })
          // ,new DuplicatePackageCheckerPlugin()
        );
      }

      // optimization
      appConfig.optimization = {
        ...baseConfig.optimization,
        minimize: !dev,
        minimizer: [new TerserPlugin({
          parallel: true,
        })],
        splitChunks: {
          chunks: 'initial',
          minSize: 30000,
          // minRemainingSize: 0,
          maxSize: 0,
          minChunks: 1,
          maxAsyncRequests: 6,
          maxInitialRequests: 4,
          automaticNameDelimiter: '~',
          automaticNameMaxLength: 30,
          cacheGroups: {
            defaultVendors: {
              test: /[\\/]node_modules[\\/]/,
              priority: -10
            },
            default: {
              minChunks: 2,
              priority: -20,
              reuseExistingChunk: true
            }
          }
        },
        moduleIds: 'total-size', //added in future deterministic
        chunkIds: 'total-size', //added
        mangleWasmImports: !dev, //added
        removeAvailableModules: !dev, //added
        mergeDuplicateChunks: !dev, //added
        flagIncludedChunks: !dev,
        occurrenceOrder: false,
        usedExports: !dev,
        // namedModules: true,
        // namedChunks: true,
        runtimeChunk: 'single'
        // runtimeChunk: {
        //   name: entrypoint => `runtimechunk~${entrypoint.name}`
        // }
      };

      appConfig.plugins.push(
        new LoadablePlugin({
          outputAsset: false,
          writeToDisk: { filename }
        }),
        new LodashModuleReplacementPlugin({
          collections: true,
          cloning: true,
          deburring: true,
          // coercions: true,
          flattening: true,
          paths: true,
          // placeholders: true
          shorthands: true
          // caching: true
        })
      );
    } else {
      appConfig.module.rules.push({
        test: /\.(scss)$/,
        use: ['ignore-loader']
      });
    }
    return appConfig;
  },
  modifyBabelOptions: mainBabelOptions => {
    return {
      ...mainBabelOptions,
      ...{ plugins: [].concat(mainBabelOptions.plugins ? mainBabelOptions.plugins : [], ['lodash']) }
    };
  }
};

multiple errors

Here is randomly picked trace

(error: https://www.freshbox.sk/static/js/common-blocks-functional-userButton.3074d9ca.chunk.js)
  at m.e(webpack/bootstrap:170:18)
  at importAsync(./src/common/blocks/header/HeaderVariant2.jsx:35:9)
  at requireAsync(./src/common/blocks/header/HeaderVariant2.jsx:34:28)
  at loadAsync(./node_modules/@loadable/component/dist/loadable.esm.js:217:31)
  at componentDidMount(./node_modules/@loadable/component/dist/loadable.esm.js:147:16)
  at Ji(./node_modules/react-dom/cjs/react-dom.production.min.js:212:132)
  at b(./node_modules/react-dom/cjs/react-dom.production.min.js:255:229)
  at If(./node_modules/scheduler/cjs/scheduler.production.min.js:19:467)
  at cg(./node_modules/react-dom/cjs/react-dom.production.min.js:122:325)
  at Jj(./node_modules/react-dom/cjs/react-dom.production.min.js:248:370)
  at yj(./node_modules/react-dom/cjs/react-dom.production.min.js:239:376)
  at Ig(./node_modules/react-dom/cjs/react-dom.production.min.js:230:137)
  at bk(./node_modules/react-dom/cjs/react-dom.production.min.js:281:43)
  at a(./node_modules/react-dom/cjs/react-dom.production.min.js:284:301)
  at Nj(./node_modules/react-dom/cjs/react-dom.production.min.js:240:120)
  at ik(./node_modules/react-dom/cjs/react-dom.production.min.js:284:287)
  at hydrate(./node_modules/react-dom/cjs/react-dom.production.min.js:290:206)
  at done(./src/client/index.js:81:3)
  at checkReadyState(./node_modules/@loadable/component/dist/loadable.esm.js:428:11)
  at E/</n.push(./node_modules/@loadable/component/dist/loadable.esm.js:435:7)
  at ? (/static/js/common-components-category-listing-_default-LayoutSwitcher.869947cb.chunk.js:1:75)```

Fribble

Answer 1

If the code is split into chunks in order to optimise the loading, the index file usually contains the names and path to all the chunks according to the current webpack build. Making changes to the code and building again can rename the chunks with edited code. The browser however loads the index first along with certain required chunks, and then the rest of the chunks are fetched on demand, to make the whole code-splitting optimisation work.

I suspect that in this case, there is a production deploy after the index is loaded by the browser, and some of the chunks get renamed by the changes next build. Which results in invalid addresses for certain chunks in the stale index. Afterwards when the old index which was in the browser tries to load those non-existent chunks when the user navigated to that part of the website, it throws Loading chunk XY failed. Refreshing should update the index and resolve the issue.

One way to resolve this would be to use service workers. The way it could work is as follows:

• The browser loads the index and the specific required chunks to render the requested section.
• As the user views/uses the loaded section, the service worker installs in the background and loads all the remaining chunks into memory, so that all the chunks can be served offline when requested. Basically creating a copy of the current state of the server in the browser memory. This not only preserves files even after the server updates, but also improves load times of other chunks drastically.
• When there is a new production deploy, the installed service worker can serve the locally cached files initially to prevent any crash.
• The service worker can update in the background and serve the new version of the site when the user visits it next, or show a message that a new version is available and ask the user to reload.

Using the stale-while-revalidate or cache-first logic for service workers should work here. Hope that helps.

Fruitage

thank you Rishit, i appreciate your response. I totally understand your point for creating service worker, i guess that we will get to it. What i do not understand is, that according to your words, this is the natural behaviour of reactjs without service workers? that is a bit concerning – Ambry

@IvanKopčík This issue is a side effect of code splitting, which is more of a webpack/deployment thing imo. React doesn't have anything to do with it. And CRA by default does generate a single huge bundle which, although not optimal for production, doesn't have this issue. Plus service-workers, which resolve this problem with splitting, are very easy to implement with CRA at least. – Fruitage

actually a few days back I was speaking to a few devOps guys about this and what they suggested sounded like a viable option. what they said was - normally with webpack what happens is that index.html carries the info for the scripts which are the chunks. so what one can do is and this is specifically for the hosting done in S3 buckets is that you can create folder for each deployment version and in the folder keep all the build files except index.html. index will be at the root level. in that way in the event of new deployment the old files will still remain in the folder. – Gerlach

That makes sense. The general solution is to make the old chunks available to the old index.html file. The method I proposed did so by storing those files on the frontend, whereas your proposal is a server side solution of the same. It isn't limited to just s3 buckets, it should work with a lot of other deployment setups as well. You can post this as a separate answer as it might be a better alternative for a lot of devs depending on their use case. – Fruitage

it would be awesome if you can code examples for your suggested solutions! – Riverhead

Answer 2

I also had random issues with ChunkLoadError on production. It was caused by public cache-control directive. For some reason IIS returned empty files. More details here https://mcmap.net/q/244654/-webpack-code-splitting-chunkloaderror-loading-chunk-x-failed-but-the-chunk-exists

Quetzalcoatl