Bloomberg API request timing out

Question

Having set up a ReferenceDataRequest I send it along to an EventQueue

Service refdata = _session.GetService("//blp/refdata");
Request request = refdata.CreateRequest("ReferenceDataRequest");
// append the appropriate symbol and field data to the request
EventQueue eventQueue = new EventQueue();
Guid guid = Guid.NewGuid();
CorrelationID id = new CorrelationID(guid);
_session.SendRequest(request, eventQueue, id);
long _eventWaitTimeout = 60000;
myEvent = eventQueue.NextEvent(_eventWaitTimeout);

Normally I can grab the message from the queue, but I'm hitting the situation now that if I'm making a number of requests in the same run of the app (normally around the tenth), I see a TIMEOUT EventType

if (myEvent.Type == Event.EventType.TIMEOUT)
    throw new Exception("Timed Out - need to rethink this strategy");
else
    msg = myEvent.GetMessages().First();

These are being made on the same thread, but I'm assuming that there's something somewhere along the line that I'm consuming and not releasing.

Anyone have any clues or advice?

There aren't many references on SO to BLP's API, but hopefully we can start to rectify that situation.

Kiefer

Answer 1

I didn't really ever get around to solving this question, but we did find a workaround.

Based on a small, apparently throwaway, comment in the Server API documentation, we opted to create a second session. One session is responsible for static requests, the other for real-time. e.g.

_marketDataSession.OpenService("//blp/mktdata"); 
_staticSession.OpenService("//blp/refdata");

The means one session operates in subscription mode, the other more synchronously - I think it was this duality which was at the root of our problems.

Since making that change, we've not had any problems.

Kiefer

I think the issue is that you're now explicitly opening the service before you get the reference. We use if (session.OpenService(BlpConstants.BLP_REF_DATA)) return session.GetService(BlpConstants.BLP_REF_DATA); else return null; whereas your initial question did not open the service first – Dudeen

Answer 2

I just wanted to share something, thanks to the code you included in your initial post.

If you make a request for historical intraday data for a long duration (which results in many events generated by Bloomberg API), do not use the pattern specified in the API documentation, as it may end up making your application very slow to retrieve all events. Basically, do not call NextEvent() on a Session object! Use a dedicated EventQueue instead.

Instead of doing this:

var cID = new CorrelationID(1);
session.SendRequest(request, cID);
do {
   Event eventObj = session.NextEvent();
   ...
}

Do this:

var cID = new CorrelationID(1);
var eventQueue = new EventQueue();
session.SendRequest(request, eventQueue, cID);
do {
   Event eventObj = eventQueue.NextEvent();
   ...
}

This can result in some performance improvement, though the API is known to not be particularly deterministic...

Simone

Answer 3

My reading of the docs agrees that you need separate sessions for the "//blp/mktdata" and "//blp/refdata" services.

Snoopy

Answer 4

A client appeared to have a similar problem. I solved it by making hundreds of sessions rather than passing in hundreds of requests in one session. Bloomberg may not be to happy with this BFI (brute force and ignorance) approach as we are sending the field requests for each session but it works.

Stronski

Answer 5

Nice to see another person on stackoverflow enjoying the pain of bloomberg API :-)

I'm ashamed to say I use the following pattern (I suspect copied from the example code). It seems to work reasonably robustly, but probably ignores some important messages. But I don't get your time-out problem. It's Java, but all the languages work basically the same.

  cid = session.sendRequest(request, null);
  while (true) {
    Event event = session.nextEvent();
    MessageIterator msgIter = event.messageIterator();
    while (msgIter.hasNext()) {
      Message msg = msgIter.next();
      if (msg.correlationID() == cid) {
        processMessage(msg, fieldStrings, result);
      }
    }
    if (event.eventType() == Event.EventType.RESPONSE) {
      break;
    }
  }

This may work because it consumes all messages off each event.

Rotter

Thanks. That's a very similar construction (and does like like the example code) - there are no unconsumed messages that I can see in previous calls through this function. If I call it just once, it's fine, but it's barfing after multiple calls. I think it might be related to a concurrent subscription to some RealTime fields, but it's quite hard to nail down ... – Kiefer

are you sharing the same queue between ref-data and market-data? That might be a problem. – Rotter

Answer 6

It sounds like you are making too many requests at once. BB will only process a certain number of requests per connection at any given time. Note that opening more and more connections will not help because there are limits per subscription as well. If you make a large number of time consuming requests simultaneously, some may timeout. Also, you should process the request completely(until you receive RESPONSE message), or cancel them. A partial request that is outstanding is wasting a slot. Since splitting into two sessions, seems to have helped you, it sounds like you are also making a lot of subscription requests at the same time. Are you using subscriptions as a way to take snapshots? That is subscribe to an instrument, get initial values, and de-subscribe. If so, you should try to find a different design. This is not the way the subscriptions are intended to be used. An outstanding subscription request also uses a request slot. That is why it is best to batch as many subscriptions as possible in a single subscription list instead of making many individual requests. Hope this helps with your use of the api.

Bikini

Answer 7

By the way, I can't tell from your sample code, but while you are blocked on messages from the event queue, are you also reading from the main event queue while(in a seperate event queue)? You must process all the messages out of the queue, especially if you have outstanding subscriptions. Responses can queue up really fast. If you are not processing messages, the session may hit some queue limits which may be why you are getting timeouts. Also, if you don't read messages, you may be marked a slow consumer and not receive more data until you start consuming the pending messages. The api is async. Event queues are just a way to block on specific requests without having to process all messages from the main queue in a context where blocking is ok, and it would otherwise be be difficult to interrupt the logic flow to process parts asynchronously.

Bikini