web3 websocket connection prevents node process from exiting

Question

I have a node js process that creates a web3 websocket connection, like so:

web3 = new Web3('ws://localhost:7545')

When the process completes (I send it a SIGTERM), it does not exit, but rather hangs forever with no console output.

I registered a listener on SIGINT and SIGTERM to observe at what handles the process has outstanding with process._getActiveRequests() and process._getActiveHandles(), I see this:

 Socket {
    connecting: false,
    _hadError: false,
    _handle: 
     TCP {
       reading: true,
       owner: [Circular],
       onread: [Function: onread],
       onconnection: null,
       writeQueueSize: 0 },
    <snip>
    _peername: { address: '127.0.0.1', family: 'IPv4', port: 7545 },
    <snip>
}

For completeness, here is the code that's listening for the signals:

async function stop() {
  console.log('Shutting down...')

  if (process.env.DEBUG) console.log(process._getActiveHandles())

  process.exit(0)
}

process.on('SIGTERM', async () => {
  console.log('Received SIGTERM')
  await stop()
})

process.on('SIGINT', async () => {
  console.log('Received SIGINT')
  await stop()
})

Looks like web3 is holding a socket open, which makes sense since I never told it to close the connection. Looking through the documentation and googling, it doesn't look like there's a close or end method for the web3 object.

Manually closing the socket in stop above allows the process to successfully exit:

web3.currentProvider.connection.close()

Anyone have a more elegant or officially sanctioned solution? It feels funny to me that you have to manually do this rather than have the object destroy itself on process end. Other clients seem to do this automatically without explicitly telling them to close their connections. Perhaps it is cleaner to tell all the clients created by your node process to close their handles/connections on shutdown anyway, but to me, this was unexpected.

Carvalho

What do you mean by When the process completes? Your process is an HTTP server listening to a port. How could it know it should stop listening? – Irishirishism

Sorry, I was unclear about how the process completes. I am sending the node process a signal to terminate. express and mongo seem to automatically close handles on process exit, but web3 does not. – Carvalho

On what system is it running? Does your process have children processes? – Irishirishism

debian jessie. no children – Carvalho

Debian is running inside a Docker container, though I don't think that should matter. – Carvalho

Try pkill -TERM -P <pid> instead of kill <pid>. I can't reproduce because it seems it requires setting up a provider. Your issue may be tracked by: github.com/ethereum/web3.js/issues/1446. In which case it is a bug of web3. – Irishirishism

wrong code: 1.) why is stop() async when uses only sync code? 2.) never user process.exit() - node will stop itself on sigterm when event loop is empty. In sigterm callback just do what you must do - when your app do not exit itself on sigterm without calling process.exit() you have bug in your code - uncleared some setTimeout() or setInterval() or watches or unclosed servers/connections etc. – Articulate

Answer 1

It feels funny to me that you have to manually do this rather than have the object destroy itself on process end

It feels funny because you have probably been exposed to more synchronous programming compared to asynchronous. Consider the below code

fs = require('fs')
data = fs.readFileSync('file.txt', 'utf-8');
console.log("Read data", data)

When you run above you get the output

$ node sync.js
Read data Hello World

This is a synchronous code. Now consider the asynchronous version of the same

fs = require('fs')
data = fs.readFile('file.txt', 'utf-8', function(err, data) {
    console.log("Got data back from file", data)
});
console.log("Read data", data);

When you run you get the below output

$ node async.js
Read data undefined
Got data back from file Hello World

Now if you think as a synchronous programmer, the program should have ended at the last console.log("Read data", data);, but what you get is another statement printed afterwards. Now this feels funny? Let's add a exit statement to the process

fs = require('fs')
data = fs.readFile('file.txt', 'utf-8', function(err, data) {
    console.log("Got data back from file", data)
});
console.log("Read data", data);
process.exit(0)

Now when you run the program, it ends at the last statement.

$ node async.js
Read data undefined

But the file is not actually read. Why? because you never gave time for JavaScript engine to execute the pending callbacks. Ideally a process automatically finishes when there is no work left for it to do (no pending callbacks, function calls etc...). This is the way asynchronous world works. There are some good SO threads and articles you should look into

https://medium.freecodecamp.org/walking-inside-nodejs-event-loop-85caeca391a9

https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/

How to exit in Node.js

Why doesn't my Node.js process terminate once all listeners have been removed?

How does a node.js process know when to stop?

So in the async world you need to either tell the process to exit or it will automatically exit when there are no pending tasks (which you know how to check - process._getActiveRequests() and process._getActiveHandles())

Purdum

I think I was being unclear about my expectations. I updated the question and hopefully it is more understandable. What feels funny to me is not how the event loop works, but the fact that the web3 client was not listening for process exit and cleaning up. – Carvalho

@AndyPang, that the thing, your expectation is a catch 22. Your process will only exit when there is nothing pending. The websocket is a continuous open channel, which is suppose to remain open. In NodeJS things don't shutdown on its own, if you need process to exit, then you need to use process.exit(0). If it was a http request then it would have finished on its own, but its a socket and it should remain open until its been told to close – Purdum

I don't think you understand the question. web3 is a client library that opens a websocket connection to do its work. I am a consumer of that library. My expectation is not for the websocket to close itself. It is for the web3 library, which creates and manages the websocket connection, to clean up after itself. – Carvalho

@AndyPang, can shown the rest of the web3 code you have used, I would like to understand that better – Purdum

Sure, you can find it here: github.com/ethereum/web3.js. I have not been able to find any connection cleanup code in there, but I haven't spent much time digging. – Carvalho

I know about the lib source code, I mean how you have used it after opening the connection? Because your question only has the new Web3 code – Purdum

Answer 2

The provider API for the JavaScript web3 module has gone through some substantial change recently due to the implementation of EIP-1193 and the impending release of Web3 1.0.0.

Per the code, it looks like web3.currentProvider.disconnect() should work. This method also accepts optional code and reason arguments, as described in the MDN reference docs for WebSocket.close(...).

Important: you'll notice that I referenced the source code above and not the documentation. That's because at present the disconnect method is not considered part of the public API. If you use it in your code, you should be sure to add a test case for it, as it could break at any time! From what I can see, WebSocketProvider.disconnect was introduced in [email protected] and is still present in the latest release as of today, which is [email protected]. Given that the stable 1.0.0 release is due to drop very soon, I don't think it's likely that this will change much between now and [email protected], but there's no holds barred when it comes to the structure of internal APIs.

I've discussed making the internal providers public at length with the current maintainer, Samuel Furter, aka nividia on GitHub. I don't fully agree with his decision to keep it internal here, but in his defense he's the only maintainer at present and he's had his hands very full with stabilizing the long-standing work in progress on the 1.0 branch.

As a result of these discussions, my opinion at the moment is that those who need a stable API for their WebSocket provider should write an EIP-1193 compatible provider of their own, and publish it on NPM for others to use. Please follow semver for this, and include a similar disconnect method in your own public API. Bonus points if you write it in TypeScript, as this gives you the ability to explicitly declare class members as public, protected, or private.

If you do this, be aware that EIP-1193 is still in draft status, so you'll need to keep an eye on the EIP-1193 discussions on EthereumMagicians and in the Provider Ring Discord to stay on top of any changes that might occur.

Shilohshim

appreciate the note about this not being documented. Also as a note, if you using the IpcProvider web3@^1.2.2 there's no disconnect() method – Sexdecillion

Answer 3

At the end of your node js process, simply call:

web3.currentProvider.connection.close()

Whithersoever