Understanding Kernel-FrontEnd communication -- Why does my Front End freeze?

Question

EDIT: Just a confirmation whether you can reproduce this or not would be useful. Only a single computer is needed to try this (no remote connection necessary).

Update It seems other can't reproduce this on Mac or Win7, so it's either WinXP-specific or specific to my machine. At this point I'm giving up.

---

It would be good to have a tutorial on how the Front End and the Kernel communicate, so we can debug remote kernel issues. Any such general answers (or links to tutorials elsewhere) are most welcome. I am already aware of Remote Kernel Strategies by Sascha Kratky.

Now the practical question: When I connect to a remote machine in any way (e.g. using remote kernel strategies), the front end will sooner or later freeze up. How can I prevent this?

Here's how to reproduce the problem on a single machine (no remote connection needed):

1. Create a new kernel configuration (Evaluation menu -> Kernel Configuration Options...)

   Select Advanced Options.

   Arguments to MLOpen:

   -LinkName "8000@YOUR-IP-HERE,8001@YOUR-IP-HERE" -LinkMode Listen 
   -LinkProtocol TCPIP -LinkHost "YOUR-IP-HERE" -LinkOptions MLDontInteract
   

   (Replace YOUR-IP-HERE with your computer's IP address. On Windows you can get this by typing ipconfig in a command window.)

   Shell command to launch kernel: leave empty (we will do it manually)

2. Open a new notebook, set the notebook's kernel to the one you just set up, and evaluate something (1+1).

3. Now we need to launch the kernel manually, before the connection times out. So start a new kernel (math) in command line mode, and evaluate the following:

   link=LinkConnect["8000,8001",LinkProtocol->"TCPIP"] (* connect to front end link *)
   
   $ParentLink = link; (* set the front end link as parent link *)
   

   Now the connection is established, and everything appears to work correctly (save for the one message in the front end's pink message window)

4. Evaluate Manipulate[n, {n, 0, 100, 1}]. First, this appears to work too. Now play with the slider. Sooner or later the front end will freeze and needs to be killed. For me, dragging the slider up and down for ~10 seconds without releasing it always suffices.

Why does the Front End freeze? How can I create a TCPIP-mode connection between the front end and the kernel in a way that everything will be working correctly?

---

Some notes:

• I'm on Windows XP.
• The problem never appears if I use LinkProtocol -> "SharedMemory".
• Aborting calculations using Evaluation -> Abort Evaluation does work correctly.
• I have verified using Links[] that MathLink`$PreemptiveLink and MathLink`$ServiceLink are created and LinkConnectedQ[] returns True on them.

Again, note that the Front End will freeze only after dragging the Manipualate slider continuously for a few seconds without releasing it.

Link to same question on MathGroup.

_{Related: Firewall settings for remote kernel to work (Mathematica)}

Smollett

To the closers: this is a valid programming question. The connection from the kernel can be established programmatically (to either the front end, another kernel, or other software), as I'm doing here. The question is also about understanding and using Mathematica's communication protocol. – Smollett

Just my two cents, when you drag a graphical element that windows message loop is usually modified until the mouse up message, so that the element or window being dragged around does so smoothly. Possibly, this is causing sync messages between your computers to be interrupted resulting in a timeout condition. – Redeploy

I tried this on Windows 7 and could not reproduce the hang, which is surprising because I would have expected this method to fail because the frontend and kernel maintain several links (besides $ParentLink): A service link and a pre-emptive link. I believe if the pre-emptive link is not set up correctly, Dynamic will fail in a bad way. – Havoc

@Arnoud Thanks for testing this! I also got a private reply yesterday, someone saying he can't reproduce on Mac. At this point I think this happens only on my machine. I did verify that the preemptive and service links are correctly created (please see the 4th note in the question, I added it yesterday), at least it seems they are. I've been told these are launched on Front End connect by sending SystemFiles/FrontEnd/TextResources/GetFEKernelInit.tr to the kernel. – Smollett

Answer 1

(not an answer, but hard to write this as comment)

fyi, there is a little bit on information about how the frontend talks to the kernel here http://reference.wolfram.com/mathematica/tutorial/AdvancedDynamicFunctionality.html

about half way down, it says

The front end and kernel communicate with each other through several 
MathLink connections, known as the main link, the preemptive link, 
and the service link. The main and preemptive links are pathways by 
which the front end can send evaluation requests to the kernel, and 
the kernel can respond with results. The service link works in reverse, 
with the kernel sending requests to the front end.

and more there. May be that will help. Also, same information is in the PDF file on page 31:

http://www.wolfram.com/learningcenter/tutorialcollection/DynamicInteractivity/

The actual messaging protocol between the kernel and front end must be very advanced.

Atli