How do I link multiple target blocks with a source block in TPL Dataflow?

Question

I expected the following to produce output from both publishers, but it only produces output from the first one:

var broadcastBlock = new BroadcastBlock<int>(null);
var transformBlock = new TransformBlock<int, int>(i => i*10);
var publish1 = new ActionBlock<int>(i => Console.WriteLine("Publisher 1:" + i));
var publish2 = new ActionBlock<int>(i => Console.WriteLine("Publisher 2:" + i));

broadcastBlock.LinkTo(transformBlock, new DataflowLinkOptions() { PropagateCompletion = true });
transformBlock.LinkTo(publish1, new DataflowLinkOptions() { PropagateCompletion = true });
transformBlock.LinkTo(publish2, new DataflowLinkOptions() { PropagateCompletion = true });

foreach (var i in Enumerable.Range(0, 5))
{
    broadcastBlock.Post(i);
}
broadcastBlock.Complete();
Task.WhenAll(publish1.Completion, publish2.Completion).Wait();

I'm obviously missing something fundamental here, any ideas?

Trypanosomiasis

Answer 1

You are linking 2 ActionBlocks to a single TransformBlock. You should be linking the 2 ActionBlocks to the BrodcastBlock and link the BroadcastBlock to the TransformBlock.

What you have:

BroadCast => Transfrom => ActionBlock
                       => ActionBlock

What you need:

Transfrom => BroadCast => ActionBlock
                       => ActionBlock

Ames

Thanks for the reply. So is the issue that a Transform block cannot be linked to multiple targets directly, and we must use the Broadcast block as an intermediary? I might have missed it but didn't see the documentation alluding to this. It does work when I introduce a broadcast block in between – Trypanosomiasis

@AmitG Some blocks can be linked to multiple targets but they don't "copy" the item to all the targets. For every item they only post to a single target. BroadcastBlock offers the item to all targets until its item changes. – Ames

It is my understanding that a BroadcastBlock only offers the latest value to all target blocks. This means that if the TransformBlock in this example has new values, the ActionBlocks might not receive all values - especially if they take longer to consume than the TransformBlock produces. – Indefatigable

@Indefatigable It only offers the latest to be received, but it will deliver to every linked target block that can accept the message. See documentation. At the very bottom: "BroadcastBlock<T> ensures that the current element is broadcast to any linked targets before allowing the element to be overwritten." – Kulturkampf

@MarcL. It still does not guarantee delivery to all targets. That is okay if you only require one of the targets to process the message - but if you need delivery to all targets, e.g. a multiplexer block, then you'll have to write your own block. – Indefatigable

@Indefatigable Ouch. On my reading of the above documentation, it does seem to guarantee delivery to all accepting, linked target buffers (e.g., buffer isn't full). (Of course, that "guarantee" is constrained by the durability of RAM, runtime stability, etc. I usually regale consumers with the Tommy Boy quote of choice,) Do you have a counterexample? If so, probably you or I should open another question. – Kulturkampf

@MarcL. I read it like you did. A WriteOnceBlock only accepts one element ever, dropping all following messages it receives. It cannot be overwritten. A BroadcastBlock's element can be overwritten, but it "ensures that the current element is broadcast to any linked targets before allowing the element to be overwritten." But if it's receiving messages faster than the targets process them, so that messages 3, 4, 5 are received before message 2 was delivered to anyone because all targets are still busy processing message 1, will message 2, 3, 4 be buffered or overwritten by 5? – Gumboil

@LWChris, my experience is that they will be buffered. – Kulturkampf

"You are linking 2 ActionBlocks to a single TransformBlock. You should be linking the 2 ActionBlocks to the BrodcastBlock and link the BroadcastBlock to the TransformBlock." -- The LinkTo method points from the source to the target, not vice versa. So I think that the wording of the answer is confusing. The drawing is correct though. – Easter