Are IPython engines independent processes?

Question

From the IPython Architecture Overview documentation we know that ...

The IPython engine is a Python instance that takes Python commands over a network connection.

Given that it is a Python instance does that imply that these engines are stand alone processes? I can manually load a set of engines via a command like ipcluster start -n 4. Doing thus is the creation of engines considered the creation of child processes of some parent process or just a means to kick off a set of independent processes that rely on IPC communication to get their work done? I can also invoke an engine via the ipengine command, which is surely standalone as its entered directly to the OS command line with no relation to anything really.

As background I'm trying to drill into how the many IPython engines manipulated through a Client from a python script will interact with another process kicked off in that script.

Distressful

Answer 1

Here's a simple way to find out the processes involved, print the list of current processes before I fire off the controller and engines and then print the list after they're fired off. There's a wmic command to get the job done...

C:\>wmic process get description,executablepath

Interestingly enough the controller gets 5 python processes going, and each engine creates one additional python process. So from this investigation I also learned that an engine is its own process, as well as the controller...

C:\>wmic process get description,executablepath | findstr ipengine
ipengine.exe                   C:\Python34\Scripts\ipengine.exe
ipengine.exe                   C:\Python34\Scripts\ipengine.exe

C:\>wmic process get description,executablepath  | findstr ipcontroller
ipcontroller.exe               C:\Python34\Scripts\ipcontroller.exe

From the looks of it they all seem standalone, though I don't think the OS's running process list carries any information about how the processes are related as far as the parent/child relationship is concerned. That may be a developer only formalism that has no representation that's tracked in the OS, but I don't know about these sort of internals to know either way.

Here's a definitive quote from MinRK that addresses this question directly:

"Every engine is its own isolated process...Each kernel is a separate process and can be on any machine... It's like you started a terminal IPython session, and every engine is a separate IPython session. If you do a=5 in this one, a=10 in that one, this guy has 10 this guy has 5."

Here's further definitive validation, inspired by a great SE Hot Network Question on ServerFault that mentioned use of ProcessExplorer which actually tracks parent child processes...

Process Explorer is a Sysinternals tool maintained by Microsoft. It can display the command line of the process in the process's properties dialog as well as the parent that launched it, though the name of that process may no longer be available. --Corrodias

If I fire off more engines in another command window that section of ProcessExplorer just duplicates exactly as you see in the screenshot.

And just for the sake of completeness, here' what the command ipcluster start --n=5 looks like...

Distressful