What is Node.js' Connect, Express and "middleware"?

Question

Despite knowing JavaScript quite well, I'm confused what exactly these three projects in Node.js ecosystem do. Is it something like Rails' Rack? Can someone please explain?

Gasolier

I haven't used connect, but this page sure makes it sound analogous to Rails' Rack. Do you understand what middleware is, outside of the context of Node? – Expertism

Honestly, not as much as i would like to. As far as I know it's the layer that does all the pre-app stuff like routing, gzipping, headers, cookies..? Am I right? So does it work in a way that routing to the proper MVC controller/action not inside of the MVC framework (like Rails), but in the middleware? – Gasolier

See also here project70.com/nodejs/understanding-connect-and-middleware – Sum

THIS WILL CLEAR ALL YOUR DOUBTS AND ANSWER MANY MORE QUERIES THAT YOU HAVE I understand that it's too late (hopefully someone scrolls down...), but reading the following blog article will clear all the questions that you have about Connect, Express and Middleware. It also teaches you a bit about Node.js too. http://evanhahn.com/understanding-express/ – Maecenas

@DiegoCaxito Your link is broken. – Filibertofilibuster

Answer 1

[Update: As of its 4.0 release, Express no longer uses Connect. However, Express is still compatible with middleware written for Connect. My original answer is below.]

I'm glad you asked about this, because it's definitely a common point of confusion for folks looking at Node.js. Here's my best shot at explaining it:

• Node.js itself offers an http module, whose createServer method returns an object that you can use to respond to HTTP requests. That object inherits the http.Server prototype.

• Connect also offers a createServer method, which returns an object that inherits an extended version of http.Server. Connect's extensions are mainly there to make it easy to plug in middleware. That's why Connect describes itself as a "middleware framework," and is often analogized to Ruby's Rack.

• Express does to Connect what Connect does to the http module: It offers a createServer method that extends Connect's Server prototype. So all of the functionality of Connect is there, plus view rendering and a handy DSL for describing routes. Ruby's Sinatra is a good analogy.

• Then there are other frameworks that go even further and extend Express! Zappa, for instance, which integrates support for CoffeeScript, server-side jQuery, and testing.

Here's a concrete example of what's meant by "middleware": Out of the box, none of the above serves static files for you. But just throw in connect.static (a middleware that comes with Connect), configured to point to a directory, and your server will provide access to the files in that directory. Note that Express provides Connect's middlewares also; express.static is the same as connect.static. (Both were known as staticProvider until recently.)

My impression is that most "real" Node.js apps are being developed with Express these days; the features it adds are extremely useful, and all of the lower-level functionality is still there if you want it.

Birgit

One thing that upsets me about Connect is that its documentation doesn't seem to acknowledge that Node is more than a HTTP server. "Connect is a middleware framework for Node.js" -- no, "Connect is a middleware framework for Node.js's HTTP server" – Infanticide

@Infanticide I think you're reading into that too much. The makers of Connect are preeminent Node developers; they're well aware that Node is more than an HTTP server. But it does have an HTTP server built in, and Connect is a middleware framework that you can use in your Node.js app. – Birgit

Oh I'm sure the makers of Connect are fully aware of that. They couldn't have achieved what they have without a thorough understanding of Node. But the choice of words is confusing for newcomers to Node; and to newcomers to Connect. – Infanticide

crystal clear, what all answers should strive for. Excellent work Trevor. – Wivina

Great explanation. Answers like this help bring new people into the Node.js ecosystem. For people getting familiar with developing web apps in Node.js, Express is the place to start. To continue the Ruby analogy, Express is comparable to Sinatra. It's particularly great for creating JSON APIs for Ajax client-side apps. One thing I've found is that once an application hits a certain level of complexity, another layer is needed that is more Rails like. I'm working on Locomotive for this purpose, which further layers on top of Express. – Proem

Zappa has not been updated in about a year and no longer works with current versions of node; for those looking to use it now, ZappaJS is a fork that is under active development and works. – Door

I would like to say that sails.js is better to start than express.js, especially if you're coming from ruby or Php with MVC framework experience. – Metallist

The relationships in this answer are wrong now. I posted about the updated relationship : https://mcmap.net/q/63809/-what-is-node-js-39-connect-express-and-quot-middleware-quot – Oby

Well I am one of those "Node newbies" and the explanation is great so far but I still didn't quite get, what middleware exactly is. Is it just some kind of Interface, that's implemented, so that the middleware listens for some events and acts on them? I googled arround for a little but couldn't find anything about middleware specificly. Thanks in advance :) – Colver

"Connect also offers" link is gone -> github.com/senchalabs/connect – Tybie

@Colver from what I've heard, middlewares are functions that execute lines, then call the next middleware function, and after it returns, they execute a few more lines, then they return. So they're pretty much functions that wrap around other functions. A logging function that simply does console.log('Request received') and ('Request completed') seems like a nice example – Timmons

Answer 2

The accepted answer is really old (and now wrong). Here's the information (with source) based on the current version of Connect (3.0) / Express (4.0).

What Node.js comes with

http / https createServer which simply takes a callback(req,res) e.g.

var server = http.createServer(function (request, response) {

    // respond
    response.write('hello client!');
    response.end();

});

server.listen(3000);

What connect adds

Middleware is basically any software that sits between your application code and some low level API. Connect extends the built-in HTTP server functionality and adds a plugin framework. The plugins act as middleware and hence connect is a middleware framework

The way it does that is pretty simple (and in fact the code is really short!). As soon as you call var connect = require('connect'); var app = connect(); you get a function app that can:

1. Can handle a request and return a response. This is because you basically get this function
2. Has a member function .use (source) to manage plugins (that comes from here because of this simple line of code).

Because of 1.) you can do the following :

var app = connect();

// Register with http
http.createServer(app)
    .listen(3000);

Combine with 2.) and you get:

var connect = require('connect');

// Create a connect dispatcher
var app = connect()
      // register a middleware
      .use(function (req, res, next) { next(); });

// Register with http
http.createServer(app)
    .listen(3000);

Connect provides a utility function to register itself with http so that you don't need to make the call to http.createServer(app). Its called listen and the code simply creates a new http server, register's connect as the callback and forwards the arguments to http.listen. From source

app.listen = function(){
  var server = http.createServer(this);
  return server.listen.apply(server, arguments);
};

So, you can do:

var connect = require('connect');

// Create a connect dispatcher and register with http
var app = connect()
          .listen(3000);
console.log('server running on port 3000');

It's still your good old http.createServer with a plugin framework on top.

What ExpressJS adds

ExpressJS and connect are parallel projects. Connect is just a middleware framework, with a nice use function. Express does not depend on Connect (see package.json). However it does the everything that connect does i.e:

1. Can be registered with createServer like connect since it too is just a function that can take a req/res pair (source).
2. A use function to register middleware.
3. A utility listen function to register itself with http

In addition to what connect provides (which express duplicates), it has a bunch of more features. e.g.

1. Has view engine support.
2. Has top level verbs (get/post etc.) for its router.
3. Has application settings support.

The middleware is shared

The use function of ExpressJS and connect is compatible and therefore the middleware is shared. Both are middleware frameworks, express just has more than a simple middleware framework.

Which one should you use?

My opinion: you are informed enough ^based on above^ to make your own choice.

• Use http.createServer if you are creating something like connect / expressjs from scratch.
• Use connect if you are authoring middleware, testing protocols etc. since it is a nice abstraction on top of http.createServer
• Use ExpressJS if you are authoring websites.

Most people should just use ExpressJS.

What's wrong about the accepted answer

These might have been true as some point in time, but wrong now:

that inherits an extended version of http.Server

Wrong. It doesn't extend it and as you have seen ... uses it

Express does to Connect what Connect does to the http module

Express 4.0 doesn't even depend on connect. see the current package.json dependencies section

Oby

you say gives you the ability to handle a request and return a response but people say that Express is really the web server...I'm confused. Wouldn't sending a response back need web server capability (like Express)? – Kesterson

good stuff, thanks! very helpful...especially not knowing that connect really is what provides the routing, and express just inherits that, it isn't the sole/source provider of routing. And the use cases at the end are helpful because I assumed I'd have to use connect AND express but really all you need to use is express for web apps so this cleared a huge thing up for me. You don't install both, you install one or the other! – Kesterson

Your answer should be on top. when I read the accepted answer, I upvoted it. But after reading your answer...naahhh – Lorislorita

Answer 3

node.js

Node.js is a javascript motor for the server side.
In addition to all the js capabilities, it includes networking capabilities (like HTTP), and access to the file system.
This is different from client-side js where the networking tasks are monopolized by the browser, and access to the file system is forbidden for security reasons.

node.js as a web server: express

Something that runs in the server, understands HTTP and can access files sounds like a web server. But it isn't one.
To make node.js behave like a web server one has to program it: handle the incoming HTTP requests and provide the appropriate responses.
This is what Express does: it's the implementation of a web server in js.
Thus, implementing a web site is like configuring Express routes, and programming the site's specific features.

Middleware and Connect

Serving pages involves a number of tasks. Many of those tasks are well known and very common, so node's Connect module (one of the many modules available to run under node) implements those tasks.
See the current impressing offering:

• logger request logger with custom format support
• csrf Cross-site request forgery protection
• compress Gzip compression middleware
• basicAuth basic http authentication
• bodyParser extensible request body parser
• json application/json parser
• urlencoded application/x-www-form-urlencoded parser
• multipart multipart/form-data parser
• timeout request timeouts
• cookieParser cookie parser
• session session management support with bundled MemoryStore
• cookieSession cookie-based session support
• methodOverride faux HTTP method support
• responseTime calculates response-time and exposes via X-Response-Time
• staticCache memory cache layer for the static() middleware
• static streaming static file server supporting Range and more
• directory directory listing middleware
• vhost virtual host sub-domain mapping middleware
• favicon efficient favicon server (with default icon)
• limit limit the bytesize of request bodies
• query automatic querystring parser, populating req.query
• errorHandler flexible error handler

Connect is the framework and through it you can pick the (sub)modules you need.
The Contrib Middleware page enumerates a long list of additional middlewares.
Express itself comes with the most common Connect middlewares.

What to do?

Install node.js.
Node comes with npm, the node package manager.
The command npm install -g express will download and install express globally (check the express guide).
Running express foo in a command line (not in node) will create a ready-to-run application named foo. Change to its (newly created) directory and run it with node with the command node <appname>, then open http://localhost:3000 and see. Now you are in.

Gwendagwendolen

great reply thanks. This is the kind of simple crap every blog post misses, the simple setup which can be ??? if you've never done it before. Yea it's simple when you have already done it but you have no clue how to start for the FIRST time! I hate it when devs overlook that in blog posts, it's essential. I don't want to have to FIND another blog post just to find setup. Just provide a link to another blog post in your other posts, that's extremely helpful so I don't have to hunt around for one. Save me the hunting trip! – Kesterson

Express 4.0.0 need to do sudo npm install -g express-generator – Futurism

@getsetbro you just mean 'npm install' to install the dependencies. – Cork

Answer 4

Connect offers a "higher level" APIs for common HTTP server functionality like session management, authentication, logging and more. Express is built on top of Connect with advanced (Sinatra like) functionality.

Forsake

Answer 5

Node.js itself offers an HTTP module, whose createServer method returns an object that you can use to respond to HTTP requests. That object inherits the http.Server prototype.

Jochbed

Answer 6

Related information, especially if you are using NTVS for working with the Visual Studio IDE. The NTVS adds both NodeJS and Express tools, scaffolding, project templates to Visual Studio 2012, 2013.

Also, the verbiage that calls ExpressJS or Connect as a "WebServer" is incorrect. You can create a basic WebServer with or without them. A basic NodeJS program can also use the http module to handle http requests, Thus becoming a rudimentary web server.

Holinshed

Answer 7

middleware as the name suggests actually middleware is sit between middle.. middle of what? middle of request and response..how request,response,express server sit in express app in this picture you can see requests are coming from client then the express server server serves those requests.. then lets dig deeper.. actually we can divide this whole express server's whole task in to small seperate tasks like in this way. how middleware sit between request and response small chunk of server parts doing some particular task and passed request to next one.. finally doing all the tasks response has been made.. all middle ware can access request object,response object and next function of request response cycle..

this is good example for explaining middleware in express youtube video for middleware

Scott

Answer 8

Middleware are special functions that run between or in the middle of request comming in & response going out from our api. For example we use app.use((req, res, next)=>{ }

here next work as a middleware between req and res in express js

Inefficacy

Answer 9

The stupid simple answer

Connect and Express are web servers for nodejs. Unlike Apache and IIS, they can both use the same modules, referred to as "middleware".

Logy