ConcurrentBag - Add Multiple Items?

Question

Is there a way to add multiple items to ConcurrentBag all at once, instead of one at a time? I don't see an AddRange() method on ConcurrentBag, but there is a Concat(). However, that's not working for me:

ConcurrentBag<T> objectList = new ConcurrentBag<T>();

timeChunks.ForEach(timeChunk =>
{
    List<T> newList = Foo.SomeMethod<T>(x => x.SomeReadTime > timeChunk.StartTime);
    objectList.Concat<T>(newList);
});

This code used to be in a Parallel.ForEach(), but I changed it to the above so I could troubleshoot it. The variable newList indeed has objects, but after the objectList.Concat<> line, objectList always has 0 objects in it. Does Concat<> not work that way? Do I need to add items to ConcurrentBag one at a time, with the Add() method?

Suellensuelo

Answer 1

Concat is an extension method provided by LINQ. It is an immutable operation that returns another IEnumerable that can enumerate the source collection followed immediately by the specified collection. It does not, in any way, change the source collection.

You will need to add your items to the ConcurrentBag one at a time.

Robber

Answer 2

(I know this is an old post, thought I'd add a little something).

Like others have said: yes, you need to add them one by one. In my case, I added a small extension method to make things a bit cleaner, but under the hood it does the same thing:

public static void AddRange<T>(this ConcurrentBag<T> @this, IEnumerable<T> toAdd)
{
    foreach (var element in toAdd)
    {
        @this.Add(element);
    }
}

And then:

ConcurrentBag<int> ccBag = new ConcurrentBag<int>();
var listOfThings = new List<int>() { 1, 2, 4, 5, 6, 7, 8, 9 };
ccBag.AddRange(listOfThings);

I also looked at using AsParallel to add within the extension method, but after running some tests on adding a list of strings of various sizes, it was consistantly slower to use AsParallel (as shown here) as opposed to the traditional for loop.

public static void AddRange<T>(this ConcurrentBag<T> @this, IEnumerable<T> toAdd)
{
    toAdd.AsParallel().ForAll(t => @this.Add(t));
}

Georgeannageorgeanne

This should be the answer to the question. – Colton

This is the correct answer. Also run tests trying to use Parallel (AsParallel().ForAll()) and it's slower. – Joiejoin

If somebody is also interested (just like me) in: "is there any sense to add items in parallel, as the receiver - is a single collection?", probably useful info about ConcurrentBag implementation could be seen in couple of answers: https://mcmap.net/q/89342/-how-might-a-class-like-net-39-s-concurrentbag-lt-t-gt-be-implemented https://mcmap.net/q/89342/-how-might-a-class-like-net-39-s-concurrentbag-lt-t-gt-be-implemented – Lattermost

Answer 3

I faced a similar issue, trying to process smaller chunks of data in parallel, because one large chunk was timing out the web service I was using to access my data on the sending side but I did not want things to run slower by processing each chunk serially. Processing the data record by record was even slower - since the service I was calling could handle bulk requests, it would be better to submit as many as possible without timing out.

Like Vlad said, concatting a concurrent bag to a list of an object type doesn't return a concurrent bag, so concat won't work! (It took me a while to realize I couldn't do that.)

Try this instead - create a List<T>, and then create a ConcurrentBag<List<T>>. On each parallel iteration, it will be adding a new list to the concurrent bag. When the parallel loop is done, loop through the ConcurrentBag and concat (or union if you want to eliminate possible duplicates) to the first List<T> that you created to "flatten" everything into one list.

Savannahsavant

To flatten use SelectMany in the end. – Hydroplane

IMHO this should be the accepted answer. It describes the OP's ultimate need, why concat won't work, and a solid solution. – Anticholinergic

Answer 4

The Concat method is an approach contained at public Enumerable static class that support System.Linq library (inner .NET System.Core assembly).

BUT, the Concat contains limits to provide the "add range" requirement to ConcurrentBag<T> object, with a behavior as image bellow (at line 47 in Visual Studio):

To Concat method matches the "add range" requirement, it's necessary to renew the current ConcurrentBag<T> object instance; if the program needs to add multiples ranges, it's necessary to makes auto-reference instances from current ConcurrentBag<T> (recursively) for each range.

Then, I do not use Concat approach and if I may do a recommendation, I DO NOT RECOMMEND. I follow a similar example from Eric's answer, where I developed a derived class from ConcurrentBag<T> that provides me the AddRange method using the ConcurrentBag<T> base method to add IEnumerable<T> items to the derived instance, as bellow:

public class ConcurrentBagCompleted<T> : ConcurrentBag<T>
{
    public ConcurrentBagCompleted() : base() { }

    public ConcurrentBagCompleted(IEnumerable<T> collection):base(collection)
    {
    }

    public void AddRange(IEnumerable<T> collection)
    {
        Parallel.ForEach(collection, item =>
        {
            base.Add(item);
        });
    }
}

Lowerclassman

Answer 5

There is a second constructor for ConcurrentBag<T> that takes IEnumerable as parameter and construct the bag from it. It is the fastest, safest method I found for such cases. Please, refer to ConcurrentBag<T>(IEnumerable<T>) So, in the case in the question the solution would be:

            ConcurrentBag<T> objectList = new ConcurrentBag<T>();

        timeChunks.ForEach(timeChunk =>
        {
            List<T> newList = Foo.SomeMethod<T>(x => x.SomeReadTime > timeChunk.StartTime);
            newList.AddRange(objectList.ToList());
            objectList=new ConcurrentBag<T>(newList);
        });

Greaves

Answer 6

ConcurrentBag<T> bag = new ConcurrentBag<T>();

Parallel.ForEach (items, item => {
    bag.Add(item);
});

Coinsurance

Thanks for answering. Code-only answers are usually frowned upon. Are you able to add some additional details as to why this resolves the OP's problem? – Thug

Answer 7

Yes :)

Concat is perhaps one of the Enumerable extensions. It doesn't add anything to the ConcurrentBag, it just returns some funky object containing the original bag and whatever you tried to add there.

Beware that the result of Concat is not a ConcurrentBag anymore, so you would not want to use it. It's a part of general LINQ framework, making possible to combine immutable sequences. This framework, of course, doesn't try to extend the concurrent properties of the operands to the result, so the resulting object will not be so well suited for multithreaded access.

(Basically, Concat applies to ConcurrentBag because it exposes IEnumerable<T> interface.)

Senescent

This "answer" clearly does not answer the question. Not sure why it was marked as the answer. – Colton

"Yes" to which of the OP questions? – We