I am working on a web-API access package in Julia, following fairly standard REST. It has lots of methods like:
post_foo(cred::ServiceCred, x, y)
get_foo(cred::ServiceCred, x, y)
Sometimes I would like to post multiple things to the API at once (or get multiple things off it).
Let's say I have multiple ys I want to post_foo
Since I have done Base.broadcastable(cred::ServiceCred) = Ref(cred),
I can do:
post_foo.(cred, "id123", ["a", "b", "c"])
And that works and is quite nice from an interface perspective. I can even do:
post_foo.(cred, ["id1", "id2", "id3"], ["a", "b", "c"])
to make it do post_foo(cred, "id1", "a") etc in turn.
But because these are web-requests, I spend a bunch of time just waiting for the response to be returned.
If I use asynchronous processing, it can be much faster, as it will sent them all off without and not block until it has nothing left to send.
And then the remote server will process them in parallel, and all the responses back.
Fairly easy to do with asyncmap.
but that is less nice:
ansyncmap(["a", "b", "c"]) do y
post_foo(cred, "id123", y)
end
So I got thinking.
What if I set up a custom BroadcastStyle for my ServiceCred such that it resulted in all the processing being resolved asynchronous, and then not blocking to til the end.
I think it might even be able to subvert the fusing machinery for more complicated calls so all the requests are fired off as soon as their inputs arrive if things chain together nicely (but that might asking a bit much).
- Is this actually possible?
- If not then why not,
- If so how exactly would I do this?