To understand the difference you need to understand why a Long Term Support (LTS) version of Node exists.
Node LTS is primarily aimed at enterprise use where there may be more resistance to frequent updates, extensive procurement procedures and lengthy test and quality requirements.
From Rod Vagg a member of the Node LTS working group:
The point of establishing an LTS plan for Node is to build on top of an existing stable release cycle by delivering new new versions on a predictable schedule that have a clearly defined extended support lifecycle. While this may seem at odds with the open source tradition of “release early, release often” it is an essential requirement for enterprise application development and operations teams. It also affects companies ... that provide professional support for Node.js.
https://medium.com/@nodesource/essential-steps-long-term-support-for-node-js-8ecf7514dbd#.za353bn08
The official Node post mentioned in one of the comments also does a fine job of explaining this:
https://nodejs.org/en/blog/community/node-v5/
It is sometimes misunderstood that odd version number releases (v5, v7) are a "beta" of the next LTS release. This is a convention used in other project but not in Node and is somewhat misleading. In fact, the next LTS release is chosen from a specific point release of the current version, and this will usually be an even version release (v6, v8). It's not that the latest versions are betas - it's that the LTS releases fulfil a specific need some organisations have.
For a point of reference look at Firefox's Extended Support Releases (https://www.mozilla.org/en-US/firefox/organizations/faq/) or Ubuntu's LTS releases (https://wiki.ubuntu.com/LTS). I don't think anyone would suggested the latest version of Firefox was a beta and that most users should stick with the older ERS versions.
Generally if you are able to keep up with the latest stable and future Node releases you should do so. These are stable and production ready releases with excellent community support. Unstable and experimental functionality is kept behind build and runtime flags and should not affect your day to day operations.
Generally I always lean towards the latest version for features and performance (as I now do allot of ES6 / ES7)
This would suggest to me that stable versions and beyond will suit your needs well. This will give you access to the latest and greatest language features provide by the underlying JavaScript engine (V8 or Chakra)
Of note: A significant difference between v4 (and earlier) and v5 is that v4 ships with npm v2 whereas v5 ships with npm v3. v3 of npm has some potentially breaking changes for your project in how it handles peerDependancies. The way npm v3 now tries to install a dependancies own dependancies as flat as possible and avoid duplicates may affect your projects but has been a big boon for Windows users so this may also impact your decision making.