tpl-dataflow Questions
2
Solved
I run a pretty typical producer/consumer model on different tasks.
Task1: Reads batches of byte[] from binary files and kicks off a new task for each collection of byte arrays. (the operation is ...
Unwinking asked 15/6, 2012 at 14:46
1
Solved
I am using TPL pipeline design together with Stephen Cleary's Try library In short it wraps value/exception and floats it down the pipeline. So even items that have thrown exceptions inside their p...
Diarmit asked 2/6, 2020 at 15:1
2
Intro:
I am building a single-node web crawler to simply validate URLs are 200 OK in a .NET Core console application. I have a collection of URLs at different hosts to which I am sending requests ...
Sibling asked 13/7, 2019 at 20:53
3
Solved
I need to introduce a retry policy to the workflow. Let's say there are 3 blocks that are connected in such a way:
var executionOptions = new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism...
Subtractive asked 4/7, 2013 at 11:57
5
Solved
How can I re-write the code that the code completes when BOTH transformblocks completed? I thought completion means that it is marked complete AND the " out queue" is empty?
public Test()
{
bro...
Shulamite asked 22/11, 2012 at 10:0
2
I am trying to create a dataflow using tpl with the following form:
-> LoadDataBlock1 -> ProcessDataBlock1 ->
GetInputPathsBlock -> LoadDataBlock2 -> ProcessDataBlock2 -> Merge...
Factual asked 5/11, 2019 at 15:6
2
Solved
I have a method that accepts an IAsyncEnumerable as argument, and returns also an IAsyncEnumerable. It calls a web method for each item in the input stream, and propagates the result to the output ...
Teplica asked 3/10, 2019 at 9:40
1
Solved
Consider this example:
class Program
{
private static readonly ITargetBlock<string> Mesh = CreateMesh();
private static readonly AsyncLocal<string> AsyncLocalContext
= new AsyncLoc...
Tetrabranchiate asked 1/10, 2019 at 6:44
5
Solved
I created this to test out a parallel extract:
public static async Task ExtractToDirectoryAsync(this FileInfo file, DirectoryInfo folder)
{
ActionBlock<ZipArchiveEntry> block = new Actio...
Creation asked 24/4, 2013 at 18:29
1
I've created something similar to a web crawler to create a report of the 1000+ Webservices I need to manage. I therefore created a TPL Dataflow Pipeline to manage getting and processing the data.
...
Synchronize asked 24/3, 2019 at 13:19
2
Solved
I have worked with TPL dataflow. Really liked it. I had heard the term Akka many times from my java/scala friends so I tried to read about it and found out that akka has a .net port too. Great. Whe...
Bohi asked 1/9, 2016 at 5:24
3
I have implemented a producer..consumer pattern using TPL Dataflow. The use case is that code reads messages from the Kafka bus. For efficiency, we need to process messages in batches when going to...
Alsatia asked 3/10, 2018 at 18:15
1
Solved
I'm working on a TPL Dataflow pipeline and noticed some strange behaviour related to ordering/parallelism in TransformManyBlocks (might apply to other blocks as well).
Here is my code to reproduce...
Mong asked 11/7, 2018 at 2:36
2
Solved
TPL Dataflow provides a very useful function:
public static IPropagatorBlock<TInput, TOutput> Encapsulate<TInput, TOutput>(
ITargetBlock<TInput> target,
ISourceBlock<TOutpu...
Blenheim asked 8/9, 2015 at 14:2
3
Solved
With the TPL Dataflow library, I would like to do something like this:
myActionBlock.Post(newValue, cancelAllPreviousPosts: true);
It appears that the cancellation token on ActionBlock cancels t...
Digitigrade asked 19/2, 2014 at 18:7
1
Solved
I have the following TPL Dataflow that never completes when using a predicate to filter the items passed from the TransformBlock to the ActionBlock.
If the predicate returns false for any of the ...
Kloof asked 23/11, 2017 at 12:5
0
I am new to TPL Dataflow.
I am trying to build a throttled async update to a fairly fast flowing input stream. BufferBlock seemed a nice match for this with the idea that I could call the Receive...
Stopover asked 16/10, 2017 at 10:39
1
I have the following pseudo code:
var queue = new BufferBlock<int>(new DataflowBlockOptions { BoundedCapacity = 5 });
var a = new ActionBlock<int>(async item =>
{
await Task.Delay...
Girandole asked 7/9, 2017 at 20:12
1
Solved
I'm building an application using TPL dataflow. Actually I've the following problem. I've one transformblock var tfb1 = new TranformBlock<InMsg, IReadOnlyCollection<OutMsg>>. So tfb1 re...
Espinosa asked 19/7, 2017 at 12:9
3
Solved
I was wondering if there are benefits associated with using a BufferBlock linked to one or many ActionBlocks, other than throttling (using BoundedCapacity), instead of just posting directly to Acti...
Kelda asked 8/10, 2012 at 11:52
2
Solved
My scenario is that I have a BufferBlock<Stream> receiving Stream's from an external source, let's say the file system or some FTP server. These file Streams will pass into another block and ...
Menswear asked 1/12, 2016 at 8:39
3
Solved
I created a TPL Dataflow pipeline which consist of 3 TransformBlock's and an ActionBlock at the end.
var loadXml = new TransformBlock<Job, Job>(job => { ... }); // I/O
var validateData = ...
Indign asked 13/1, 2017 at 11:35
1
Is there any way to limit performance degradation with TPL Dataflow throttling?
I have a complicated pipeline of components and trying to limit a memory requirements needed. I read in parallel fro...
Habitude asked 22/12, 2016 at 20:20
3
Solved
TPL Dataflow provides a TransformBlock for transforming input, e.g.:
var tb = new TransformBlock<int, int>(i => i * 2);
Is it possible to not output some of the input, e.g. if the input...
Inconsistent asked 31/10, 2012 at 17:56
2
I am currently working on a research project which involves indexing a large number of files (240k); they are mostly html, xml, doc, xls, zip, rar, pdf, and text with filesizes ranging from a few K...
Parboil asked 5/5, 2012 at 14:15
© 2022 - 2024 — McMap. All rights reserved.