How to set up webpack-hot-middleware in an express app?
Asked Answered
E

3

8

I am trying to enable webpack HMR in my express app. It's NOT an SPA app. For the view side, I am using EJS and Vue. I don't have the advantage of vue-cli here so I have to manually configure the vue-loader for the SFCs(.vue files) in the webpack. Also worth mentioning, my workflow is very typical: I have my main client-side resources (scss, js, vue etc) in resources dir. and I wish to bundle them inside my public dir.

My webpack.config.js:

const path = require('path');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const VueLoaderPlugin = require('vue-loader/lib/plugin');
const webpack = require('webpack');

module.exports = {
    mode: 'development',
    entry: [
        './resources/css/app.scss',
        './resources/js/app.js',
        'webpack-hot-middleware/client'
    ],
    output: {
        path: path.resolve(__dirname, 'public/js'),
        publicPath: '/',
        filename: 'app.js',
        hotUpdateChunkFilename: "../.hot/[id].[hash].hot-update.js",
        hotUpdateMainFilename: "../.hot/[hash].hot-update.json"
    },
    module: {
        rules: [
            {
                test: /\.(sa|sc|c)ss$/,
                use: [
                    {
                        loader: MiniCssExtractPlugin.loader,
                        options: {
                            hmr: process.env.NODE_ENV === 'development'
                        }
                    },
                    'css-loader',
                    'sass-loader'
                ],
            },
            {
                test: /\.vue$/,
                loader: 'vue-loader'
            }
        ]
    },
    plugins: [
        new VueLoaderPlugin(),
        new MiniCssExtractPlugin({
            filename: '../css/app.css'
        }),
        new webpack.HotModuleReplacementPlugin(),
        new webpack.NoEmitOnErrorsPlugin()
    ]
};

My app/index.js file:

import express from 'express';
import routes from './routes';
import path from 'path';
import webpack from 'webpack';
import devMiddleware from 'webpack-dev-middleware';
import hotMiddleware from 'webpack-hot-middleware';
const config = require('../webpack.config');
const compiler = webpack(config);

const app = express();

app.use(express.static('public'));
app.use(devMiddleware(compiler, {
    noInfo: true,
    publicPath: config.output.publicPath
}));
app.use(hotMiddleware(compiler));

app.set('view engine', 'ejs');
app.set('views', path.join(__dirname, '../resources/views'))

routes(app);

app.listen(4000);

export default app;

The scripts section of my package.json file:

"scripts": {
    "start": "nodemon app --exec babel-node -e js",
    "watch": "./node_modules/.bin/webpack --mode=development --watch",
    "build": "./node_modules/.bin/webpack --mode=production"
}

I am using nodemon to restart server to pick up the changes of server-side code. In one tab I keep npm run start open and in other tab npm run watch.

In my console, I see that HMR connected:

enter image description here

It only picks up the change first time only, and throws some warning like this:

Ignored an update to unaccepted module ./resources/css/app.scss -> 0

And doesn't pick up the subsequent changes. How can I fix this?

Reproduction Repo: https://bitbucket.org/tanmayd/express-test

Eakins answered 30/1, 2020 at 10:55 Comment(10)
I researched this so many times, none of those were acceptable for me. Then I used nodemon for this, I don't know but hope this helps...Stepparent
Thanks for your response. Is nodemon able to replace the static assets with compiled ones? In other words, do you have to reload the browser to see the new changes? I am also using nodemon but only to detect file change and restart the server only.Eakins
Yes, you shouldStepparent
Sorry I don't understand, I should what?Eakins
You should reload browser to see changesStepparent
But I want to see the changes instantly, just like a vue-cli appEakins
I can help you but you should put a GitHub or GitLab link to show me a reproduction of your project, definitely, I can help you with your requirement. leave a link to show your reproduction. I wait.Interpreter
@Interpreter Thread updated!Eakins
Have you taken a look at medium.com/@johnstew/webpack-hmr-with-express-app-76ef42dbac17 ? What about dev.to/riversun/how-to-run-webpack-dev-server-on-express-5ei9?Ioved
Dear bro @Tanmay, I spend the whole evening for you project and leave an answer. I hope it helps your project. for more questions leave a comment under my answer post.Interpreter
Q
5

Since it's not a SPA and you want to use EJS that will require server side rendering. It's not that easy in your case, first you will need to overwrite the render method and after that you need to add those files generated by webpack.

Based on your repo from your description, https://bitbucket.org/tanmayd/express-test, you were on the right track, but you combined development and production settings in your webpack config.

Since I can't push on your repo, I will list below the files that suffered changes or those that are new.

1. Scripts & Packages

"scripts": {
    "start": "cross-env NODE_ENV=development nodemon app --exec babel-node -e js",
    "watch": "./node_modules/.bin/webpack --mode=development --watch",
    "build": "cross-env NODE_ENV=production ./node_modules/.bin/webpack --mode=production",
    "dev": "concurrently --kill-others \"npm run watch\" \"npm run start\"",
    "production": "cross-env NODE_ENV=production babel-node ./app/server.js"
  },

I installed cross-env(because i'm on windows), cheerio(a nodejs jquery kind of version --- it's not that bad), style-loader (which is a must in development while using webpack).

The scripts:

  • start - start the development server
  • build - generate production files
  • production - start the server using the files generated from "build"

2. webpack.config.js - changed

style-loader was added in the mix so webpack will deliver your css from the bundle (see ./resources/js/app.js - line 1). MiniCssExtractPlugin is meant to be used when you want to extract the styles to a separate file, that is in production.

const path = require('path');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const VueLoaderPlugin = require('vue-loader/lib/plugin');
const webpack = require('webpack');

// Plugins
let webpackPlugins = [
    new VueLoaderPlugin(),
    new webpack.HotModuleReplacementPlugin(),
    new webpack.NoEmitOnErrorsPlugin(),
];
// Entry points
let webpackEntryPoints = [
    './resources/js/app.js',
];

if (process.env.NODE_ENV === 'production') {

    webpackPlugins = [
        new VueLoaderPlugin()
    ];
    // MiniCssExtractPlugin should be used in production
    webpackPlugins.push(
        new MiniCssExtractPlugin({
            filename: '../css/app.css',
            allChunks: true
        })
    )

}else{

    // Development
    webpackEntryPoints.push('./resources/css/app.scss');
    webpackEntryPoints.push('webpack-hot-middleware/client');
}


module.exports = {
    mode: process.env.NODE_ENV === 'development' ? 'development' : 'production',
    entry: webpackEntryPoints,
    devServer: {
        hot: true
    },
    output: {
        path: path.resolve(__dirname, 'public/js'),
        filename: 'app.js',
        publicPath: '/'
    },
    module: {
        rules: [
            {
                test: /\.(sa|sc|c)ss$/,
                use: [
                    // use style-loader in development
                    (process.env.NODE_ENV === 'development' ? 'style-loader' : MiniCssExtractPlugin.loader),
                    'css-loader',
                    'sass-loader',
                ],
            },
            {
                test: /\.vue$/,
                loader: 'vue-loader'
            }
        ]
    },
    plugins: webpackPlugins
};

3. ./resources/js/app.js - changed

The styles are now added on the first line import "../css/app.scss";

4. ./app/middlewares.js - new

Here you will find 2 middlewares, overwriteRenderer and webpackAssets.

overwriteRenderer, must be the first middleware before your routes, it's used in both development and production, in development it will suppress the ending of the request after render and will populate the response(res.body) with the rendered string of your file. In production your views will act as layouts, therefore the files that were generated will be added in head(link) and body(script).

webpackAssets will be used only in development, must be the last middleware, this will add to the res.body the files generated in memory by webpack(app.css & app.js). It's a custom version of the example found here webpack-dev-server-ssr

const cheerio = require('cheerio');
let startupID = new Date().getTime();

exports.overwriteRenderer = function (req, res, next) {
    var originalRender = res.render;
    res.render = function (view, options, fn) {
        originalRender.call(this, view, options, function (err, str) {
            if (err) return fn(err, null); // Return the original callback passed on error

            if (process.env.NODE_ENV === 'development') {

                // Force webpack in insert scripts/styles only on text/html
                // Prevent webpack injection on XHR requests
                // You can tweak this as you see fit
                if (!req.xhr) {
                    // We need to set this header now because we don't use the original "fn" from above which was setting the headers for us.
                    res.setHeader('Content-Type', 'text/html');
                }

                res.body = str; // save the rendered string into res.body, this will be used later to inject the scripts/styles from webpack
                next();

            } else {

                const $ = cheerio.load(str.toString());
                if (!req.xhr) {

                    const baseUrl = req.protocol + '://' + req.headers['host'] + "/";
                    // We need to set this header now because we don't use the original "fn" from above which was setting the headers for us.
                    res.setHeader('Content-Type', 'text/html');

                    $("head").append(`<link rel="stylesheet" href="${baseUrl}css/app.css?${startupID}" />`)
                    $("body").append(`<script type="text/javascript" src="${baseUrl}js/app.js?${startupID}"></script>`)

                }

                res.send($.html());

            }

        });
    };
    next();
};
exports.webpackAssets = function (req, res) {

    let body = (res.body || '').toString();

    let h = res.getHeaders();

    /**
     * Inject scripts only when Content-Type is text/html
     */
    if (
        body.trim().length &&
        h['content-type'] === 'text/html'
    ) {

        const webpackJson = typeof res.locals.webpackStats.toJson().assetsByChunkName === "undefined" ?
            res.locals.webpackStats.toJson().children :
            [res.locals.webpackStats.toJson()];

        webpackJson.forEach(item => {

            const assetsByChunkName = item.assetsByChunkName;
            const baseUrl = req.protocol + '://' + req.headers['host'] + "/";
            const $ = require('cheerio').load(body.toString());

            Object.values(assetsByChunkName).forEach(chunk => {

                if (typeof chunk === 'string') {
                    chunk = [chunk];
                }
                if (typeof chunk === 'object' && chunk.length) {

                    chunk.forEach(item => {

                        console.log('File generated by webpack ->', item);

                        if (item.endsWith('js')) {

                            $("body").append(`<script type="text/javascript" src="${baseUrl}${item}"></script>`)

                        }

                    });

                }

                body = $.html();

            });

        });

    }

    res.end(body.toString());

}

5. ./app/index.js - changed

This file is meant for development. Here i added the middlewares from 4 and added the serverSideRender: true option to devMiddleware so webpack will serve us those assets that are used in 4

import express from 'express';
import routes from './routes';
import path from 'path';
import devMiddleware from 'webpack-dev-middleware';
import hotMiddleware from 'webpack-hot-middleware';
import webpack from 'webpack';
const {webpackAssets, overwriteRenderer} = require('./middlewares');
const config = require('../webpack.config');
const compiler = webpack(config);
const app = express();

app.use(express.static('public'));
app.use(devMiddleware(compiler, {
    publicPath: config.output.publicPath,
    serverSideRender: true // enable serverSideRender, https://github.com/webpack/webpack-dev-middleware
}));
app.use(hotMiddleware(compiler));

app.set('view engine', 'ejs');
app.set('views', path.join(__dirname, '../resources/views'));

// This new renderer must be loaded before your routes.
app.use(overwriteRenderer); // Local render

routes(app);

// This is a custom version for server-side rendering from here https://github.com/webpack/webpack-dev-middleware
app.use(webpackAssets);

app.listen(4000, '0.0.0.0', function () {
    console.log(`Server up on port ${this.address().port}`)
    console.log(`Environment: ${process.env.NODE_ENV}`);
});

export default app;

6. ./app/server.js - new

This is the production version. It's mostly a cleanup version of 5, all development tools were removed, and only overwriteRenderer remained.

import express from 'express';
import routes from './routes';
import path from 'path';

const {overwriteRenderer} = require('./middlewares');
const app = express();

app.use(express.static('public'));
app.use(overwriteRenderer); // Live render

app.set('view engine', 'ejs');
app.set('views', path.join(__dirname, '../resources/views'));

routes(app);

app.listen(5000, '0.0.0.0', function() {
    if( process.env.NODE_ENV === 'development'){
        console.error(`Incorrect environment, "production" expected`);
    }
    console.log(`Server up on port ${this.address().port}`);
    console.log(`Environment: ${process.env.NODE_ENV}`);
});
Quark answered 6/2, 2020 at 14:29 Comment(2)
Hi, I'm away from all devices for some time. I will verify the change as soon as I can. Although looking at your code from the phone, things look good so far. Only thing I am unable to comprehend is the purpose of cheerio + overwriteRenderer. I guess I have to run the code to see it in action. I will get back to you, thanksEakins
cheerio acts as a virtual dom, it can load your html string then you can easily select/alter those html elements, which is easy if you know some basic jquery. In this case i'm using it only to add the scripts from webpack. However without cheerio i would have to use some kind of replace to add those scripts which would be a husle. overwriteRenderer is used to prevent the default res.render to end(and add headers to) the request, we need to do that ourselves after we have included the webpack scripts with cheerio.Quark
P
1

I had faced similar issue some time ago and was able to solve by combination of xdotool and exec in node. It might help you as well.

Here is the summary:

  • Have a bash script to reload the browser. The script uses xdotool to get the Chrome window and reload (Script can be used for firefox and other browser also).
    Relevant SO question: How to reload Google Chrome tab from terminal?
  • In the main file (app/index.js), using exec, run the script (inside app.listen callback). When any changes are made, nodemon will reload causing the script to execute and reloading the browser.

Bash script: reload.sh

BID=$(xdotool search --onlyvisible --class Chrome)
xdotool windowfocus $BID key ctrl+r


app/index.js

...
const exec = require('child_process').exec;

app.listen(4000, () => {
    exec('sh script/reload.sh',
        (error, stdout, stderr) => {
            console.log(stdout);
            console.log(stderr);
            if (error !== null) {
                console.log(`exec error: ${error}`);
            }
        }
    );
});

export default app;

Hope it helps. Revert for any doubts.

Panay answered 4/2, 2020 at 18:4 Comment(0)
I
1

Actually, your reproduction has some issues on the declaration which they're not related to your current issue but please observe them:

  1. Do not push the build files to the git server, just send source files.
  2. Set a cleaner on the webpack to clean the public folder on the production build.
  3. Rename the folders and files to a name that just exactly they do.
  4. Install nodemon on your project in the dev dependencies.

And your problem, I changed many things on your reproduction structure and if you have no time to read this answer post just see this repo and get what you want.

  1. Change the app/index.js to the below:
import express from 'express';
import routes from './routes';
import hotServerMiddleware from 'webpack-hot-server-middleware';
import devMiddleware from 'webpack-dev-middleware';
import hotMiddleware from 'webpack-hot-middleware';
import webpack from 'webpack';
const config = require('../webpack.config');
const compiler = webpack(config);

const app = express();

app.use(devMiddleware(compiler, {
    watchOptions: {
        poll: 100,
        ignored: /node_modules/,
    },
    headers: { 'Access-Control-Allow-Origin': '*' },
    hot: true,
    quiet: true,
    noInfo: true,
    writeToDisk: true,
    stats: 'minimal',
    serverSideRender: true,
    publicPath: '/public/'
}));
app.use(hotMiddleware(compiler.compilers.find(compiler => compiler.name === 'client')));
app.use(hotServerMiddleware(compiler));

const PORT = process.env.PORT || 4000;

routes(app);

app.listen(PORT, error => {
    if (error) {
        return console.error(error);
    } else {
        console.log(`Development Express server running at http://localhost:${PORT}`);
    }
});

export default app;
  1. Install webpack-hot-server-middleware, nodemon and vue-server-renderer in the project and change the start script to have package.json like below:
{
  "name": "express-test",
  "version": "1.0.0",
  "main": "index.js",
  "author": "Tanmay Mishu ([email protected])",
  "license": "MIT",
  "scripts": {
    "start": "NODE_ENV=development nodemon app --exec babel-node -e ./app/index.js",
    "watch": "./node_modules/.bin/webpack --mode=development --watch",
    "build": "./node_modules/.bin/webpack --mode=production",
    "dev": "concurrently --kill-others \"npm run watch\" \"npm run start\""
  },
  "dependencies": {
    "body-parser": "^1.19.0",
    "csurf": "^1.11.0",
    "dotenv": "^8.2.0",
    "ejs": "^3.0.1",
    "errorhandler": "^1.5.1",
    "express": "^4.17.1",
    "express-validator": "^6.3.1",
    "global": "^4.4.0",
    "mongodb": "^3.5.2",
    "mongoose": "^5.8.10",
    "multer": "^1.4.2",
    "node-sass-middleware": "^0.11.0",
    "nodemon": "^2.0.2",
    "vue": "^2.6.11",
    "vue-server-renderer": "^2.6.11"
  },
  "devDependencies": {
    "babel-cli": "^6.26.0",
    "babel-preset-env": "^1.7.0",
    "babel-preset-stage-0": "^6.24.1",
    "concurrently": "^5.1.0",
    "css-loader": "^3.4.2",
    "mini-css-extract-plugin": "^0.9.0",
    "node-sass": "^4.13.1",
    "nodemon": "^2.0.2",
    "sass-loader": "^8.0.2",
    "vue-loader": "^15.8.3",
    "vue-style-loader": "^4.1.2",
    "vue-template-compiler": "^2.6.11",
    "webpack": "^4.41.5",
    "webpack-cli": "^3.3.10",
    "webpack-dev-middleware": "^3.7.2",
    "webpack-hot-middleware": "^2.25.0",
    "webpack-hot-server-middleware": "^0.6.0"
  }
}
  1. Change the entire your webpack configuration file to below:
const path = require('path');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const VueLoaderPlugin = require('vue-loader/lib/plugin');
const webpack = require('webpack');

module.exports = [
    {
        name: 'client',
        target: 'web',
        mode: 'development',
        entry: [
            'webpack-hot-middleware/client?reload=true',
            './resources/js/app.js',
        ],
        devServer: {
            hot: true
        },
        output: {
            path: path.resolve(__dirname, 'public'),
            filename: 'client.js',
            publicPath: '/',
        },
        module: {
            rules: [
                {
                    test: /\.(sa|sc|c)ss$/,
                    use: [
                        {
                            loader: MiniCssExtractPlugin.loader,
                            options: {
                                hmr: process.env.NODE_ENV === 'development'
                            }
                        },
                        'css-loader',
                        'sass-loader'
                    ],
                },
                {
                    test: /\.vue$/,
                    loader: 'vue-loader'
                }
            ]
        },
        plugins: [
            new VueLoaderPlugin(),
            new MiniCssExtractPlugin({
                filename: 'app.css'
            }),
            new webpack.HotModuleReplacementPlugin(),
            new webpack.NoEmitOnErrorsPlugin(),
        ]
    },
    {
        name: 'server',
        target: 'node',
        mode: 'development',
        entry: [
            './resources/js/appServer.js',
        ],
        devServer: {
            hot: true
        },
        output: {
            path: path.resolve(__dirname, 'public'),
            filename: 'server.js',
            publicPath: '/',
            libraryTarget: 'commonjs2',
        },
        module: {
            rules: [
                {
                    test: /\.(sa|sc|c)ss$/,
                    use: [
                        {
                            loader: MiniCssExtractPlugin.loader,
                            options: {
                                hmr: process.env.NODE_ENV === 'development'
                            }
                        },
                        'css-loader',
                        'sass-loader'
                    ],
                },
                {
                    test: /\.vue$/,
                    loader: 'vue-loader'
                }
            ]
        },
        plugins: [
            new VueLoaderPlugin(),
            new MiniCssExtractPlugin({
                filename: 'app.css'
            }),
            new webpack.HotModuleReplacementPlugin(),
            new webpack.NoEmitOnErrorsPlugin(),
        ]
    }
];
  1. Add a file that name is htmlRenderer.js inside the resources folder:
export default html => `
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Tanmay Mishu</title>
    <link rel="stylesheet" href="/app.css">
</head>
<body>
    <div id="app">${html}</div>
    <script src="/client.js"></script>
</body>
</html>`;
  1. Add a new file that name is appServer.js and its codes should be like following:
import Vue from 'vue';
import App from './components/App.vue';
import htmlRenderer from "../htmlRenderer";

const renderer = require('vue-server-renderer').createRenderer()

export default function serverRenderer({clientStats, serverStats}) {
    Vue.config.devtools = true;

    return (req, res, next) => {
        const app = new Vue({
            render: h => h(App),
        });

        renderer.renderToString(app, (err, html) => {
            if (err) {
                res.status(500).end('Internal Server Error')
                return
            }
            res.end(htmlRenderer(html))
        })
    };
}

Now, just run yarn start and enjoy server-side rendering alongside hot reload.

Interpreter answered 7/2, 2020 at 15:5 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.