How can I send a ZeroMQ message from a ROUTER socket to a specific DEALER socket using cppzmq?
Asked Answered
P

1

7

I've put together this minimal example in order to send a message from a Router socket to a specific DEALER socker (That has it's identity set). When running these two programs it appears to hang on the ROUTER waiting from the reply from the DEALER, and the DEALER hangs waiting for the request from the ROUTER. So it appears that the message that the ROUTER is sending is never making it to the DEALER.

Router.cpp

#include <iostream>
#include <zmq.hpp>
#include <string>
#include <thread>
#include <chrono>

int main() {
    zmq::context_t context;
    zmq::socket_t socket (context, zmq::socket_type::router);
    // Enforce sending routable messages only
    socket.setsockopt(ZMQ_ROUTER_MANDATORY, 1);
    socket.bind("tcp://*:5555");

    try {
        std::string jobRequest = "ExampleJobRequest";

        std::cout << "Router: Sending msg: " << jobRequest << std::endl;

        // Set the address, then the empty delimiter and then the request itself
        socket.send("PEER2", ZMQ_SNDMORE);
        //socket.send(zmq::message_t(), ZMQ_SNDMORE);
        socket.send(zmq::str_buffer("ExampleJobRequest")) ;

        // Set the address, then the empty delimiter and then the request itself
        socket.send("PEER2", ZMQ_SNDMORE);
        //socket.send(zmq::message_t(), ZMQ_SNDMORE);
        socket.send(zmq::str_buffer("ExampleJobRequest")) ;

        // Receive the reply from the camera
        std::cout << "Router: Waiting for reply from camera " << std::endl;
        zmq::message_t reply;
        socket.recv(&reply);

        std::cout << "Router: Received " <<  std::string(static_cast<char*>(reply.data()), reply.size()) << std::endl;
    } catch (std::exception e) {
        std::cout << "Router Error: " << e.what();
    }

    std::this_thread::sleep_for(std::chrono::seconds(1));
    socket.close();
    context.close();
}

Dealer.cpp

#include <zmq.hpp>
#include <string>
#include <iostream>
#include <thread>

int main (void)
{
    //  Prepare our context and socket
    zmq::context_t context;
    zmq::socket_t socket (context, zmq::socket_type::dealer);

    std::cout << "Dealer: Connecting to RunJob server… \n";
    socket.setsockopt(ZMQ_IDENTITY, "PEER2", 5);
    socket.connect ("tcp://localhost:5555");

    while(true) {
        try {
            // Wait for next request from client
            std::cout << "Dealer: Waiting for request" << std::endl;
            zmq::message_t request;
            zmq::message_t empty;

            // Receive request
            socket.recv(&request);

            std::string requestString = std::string(static_cast<char*>(request.data()), request.size());

            std::cout << "Dealer: Received request" << std::endl;
            std::cout << requestString << std::endl;

            // ZMQ_SNDMORE - "Specifies that the message being sent is a multi-part message, and that further message parts are to follow"
            socket.send(zmq::str_buffer("Job completed"), zmq::send_flags::dontwait);
        }catch (std::exception e) {
            std::cout << "Router Error: " << e.what();
        }
    }

    // Used to set various 0MQ Socket Settings
    // ZMQ_Linger - Set linger period for socket shutdown
    socket.setsockopt(ZMQ_LINGER, 0);
    socket.close();
    context.close();

    return 0;
}

I had originally considered that I should be prepending the message with an empty delimiter, socket.send(zmq::message_t(), ZMQ_SNDMORE);, but this caused an error. Also using the following also caused an error to be thrown in the try/catch block. The error simply prints 'Unknown error':

zmq::message_t delimiter(0);
socket.send(delimiter, ZMQ_SNDMORE);

Using the following to create the delimiter also causes the same error:

socket.send(zmq::message_t(), ZMQ_SNDMORE);

To my knowledge, when using cppzmq you don't need to add the empty delimiter (I could be wrong about this, but after reading around and looking at other peoples example and testing my own code, this is what i determined).

Here's a very basic diagram with the end goal of this design :

ROUTER to DEALER messaging design

In my research, i haven't found a good example of this code. The Cppzmq github has very little documentation and few examples.

Here are some other sources i've looked at:

Promiscuity answered 24/2, 2020 at 17:6 Comment(0)
B
6

The main idea about ROUTER/DEALER pattern is that it is an asynchronous generalisation of REPLY/REQUEST. Yet you are trying to reverse the sockets in your pattern, discovering it doesn't fit and contorting the code to try and make it fit. Don't do that.

What you need to do is "go with the flow". In the simple method, for which examples exist, the DEALER should send the first message. The ROUTER then responds to that.

The next level is for the DEALER to identify itself in its startup message. The ROUTER can then give a specific response to that DEALER.

At the next level you can go truly asynchronous. The ROUTER can take a copy of each DEALER's identification message, and use the message copies to send asynchronous messages to any DEALER at any time. One copy of the identification message would have the "PEER2" frame appended to it and sent to the DEALER. This works because the copies of the messages include the routing frames. Ideally, you would also strip the 'message' frames, to leave only the routing frames in the copy.

Caveat - I don't use cppzmq, I use CZMQ. I can say that using CZMQ this sort of frame manipulation is very easy.

Burns answered 24/2, 2020 at 23:35 Comment(11)
This is indeed the solution. I just tried it by having the dealer send an empty message, but the router actually received the dealer's id, and then subsequent sends from the router began to succeed.Traceetracer
Hi John, Thanks so much for your response. So what your saying is that i'm attacking this problem incorrectly and i need to take it one step at a time. I first need to set up my example so that the DEALER sends the first message and the ROUTER responds. Then I set it up to have the DEALER send an identification message to the ROUTER. Then i can set it up to be asynchronous as you described.Promiscuity
You also mentioned "Yet you are trying to reverse the sockets in your pattern, discovering it doesn't fit and contorting the code to try and make it fit". When you say this, is this because I'm having the ROUTER send the first message with the DEALER should be. Or are you saying this because I should have a Single DEALER and multiple ROUTERS that complete the job requests. Thanks again for your thoughtful response.Promiscuity
I guess i'm confused if i'm able to achieve my design with the ROUTER DEALER pattern. My basic idea is 1 "client" that sends "Job requests" to several "servers" it's connected to. The "client" should be able to send a "job request" to a specific server, the server processes that request and responds to the "client" the result of the request. Is this achievable with the ROUTER/DEALER? Or do i need to return to the drawing board?Promiscuity
Vpaladino. Yes, the DEALER must send the first message.Burns
Vpaladino. It is possible to use one dealer and many routers. However, with that configuration the dealer cannot choose which router a message goes to. Instead, zmq chooses for you with a round robin algorithm. For your problem, you need one router and many dealers because that way round the router can select the dealer to send any particular message. However, as I said, the dealer must send the first (identification) message to allow the router to reply to it. If the router stores a copy of that first message, it can 'reply' to the dealer as many times as it likes.Burns
@JohnJefferies Ahah, i believe i understand now! So if i send the ID message from the dealer and receive it on the router, i can store a reference to that message_t object and use it to "reply" back to the dealer at any time. Thank you so much for taking the time to help me out, it really means a lot to me and I really do appreciate it.Promiscuity
@JohnJefferies I do have one final question if you wouldn't mind answering, and then i'll stop bothering you. With your description, the router.cpp holds several message_t objects and can send a reply back to specific one at any time, but after the router.cpp send that spontaneous message to a specific dealer, can the dealer send a reply back to the router with the "result"?Promiscuity
@Vpaladino. The dealer can send messages at any time to the router just using the normal send functions because zmq automatically adds the routing frames. One more thing; you have to store a copy of the message on the router, not just a reference to it. That's because a message is destroyed when it has been sent.Burns
@JohnJefferies Hey John. I was able to get this code all squared away and set up similar to how you suggested so I wanted to say thank you so much for your help! Currently the Dealer sockets must know the Router's IP address in order to connect to it and send messages. I was wondering if you knew a way I could send messages with the Dealers not knowing the routers address and the router have a list of IPs for the dealer.Promiscuity
@Promiscuity The dealer must know the address of the router in order to send a message to it. That's fundamental to TCP/IP. If that's not the case you can arrange for a broker to route messages. Something like, the router registers with the broker using some arbitrary ID, then the dealer sends a message which includes the ID to the broker expecting it to be forwarded to the right place. Alternatively, the broker could simply tell the dealer the address of the router. The protocols involved are you to design. Of course, everyone needs to know the address of the broker.Burns

© 2022 - 2024 — McMap. All rights reserved.