What .NET 4.0 System.Collections.Concurrent collection added in functionality to .NET 3.0 SynchronizedCollection?

Question

.NET 4.0 introduced the System.Collections.Concurrent namespace:

"The System.Collections.Concurrent namespace provides several thread-safe collection classes that should be used in place of the corresponding types in the System.Collections and System.Collections.Generic namespaces whenever multiple threads are accessing the collection concurrently"

• BlockingCollection<T> class
• ConcurrentBag<T> class
• ConcurrentQueue<T> class
• ConcurrentDictionary<TKey, TValue> class
• OrderablePartitioner<TSource> class
• Partitioner class
• IProducerConsumerCollection<T> interface

The SynchronizedCollection<T> class (available since .NET 3.0):

"Provides a thread-safe collection that contains objects of a type specified by the generic parameter as elements"

...is in the System.Collections.Generic namespace.

So, why is the SynchronizedCollection<T> class thread-safe but not concurrent?

What specifically makes the SynchronizedCollection<T> generic class different and incompatible with collections from those in System.Collections.Concurrent?

Update: Let me rephrase the question: What is the common denominator and distinguishing new feature in all generic collections that belong to the System.Collections.Concurrent namespace, which is absent in (and impossible while using) the SynchronizedCollection<T> generic class?

I changed the title to "What .NET 4.0 System.Collections.Concurrent collection added in functionality to .NET 3.0 SynchronizedCollection?". But mostly I am interested to know what it is that made it impossible to do on the basis of .NET 3.0

Update2: Concerning the note:

"This question may already have an answer here:

What is the difference between SynchronizedCollection and the other concurrent collections?"

The answer is confusing in context of my question - are the new features evolutionary (using pre-.NET 4.0 features) or revolutionary (unavailable in pre-.NET 4.0)?

Amphibiotic

What you mean saying "thread-safe but not concurrent"? – Goneness

It is not me. It is MSDN tells that all collections are thread-safe, but SynchronizedCollection is not in System.Collections.Concurrent – Amphibiotic

Perhaps it is just nomenclature. Systems.Collections.Concurrent wasn't, if I remember correctly, around in .NET 3.0. In order not to break any existing code the SynchronizedCollection remained where it was. – Competitive

I'm not sure I understand what you are asking. "What specifically makes SynchronizedCollection Generic Class different and incompatible with collections from those in System.Collections.Concurrent ? " - it's a different class, that's what makes it different. Do you understand, say, the relationship between .NET 1.0's ArrayList and .NET 2.0's List<> ? – Goldia

I understand that .NET 2.0's generic List<> adds type-safety in comparison to .NET 1.0 ArrayList. But I do not understand how this extends to or helps me in understanding the thread-safety relationship (differences in) between .NET 3.0 SynchronizedCollection<T> and .NET 4.0 System.Collections.Concurrent collections, all of which are type-safe (generic). – Amphibiotic

In the same way that 2.0 added new ('different', 'incompatible') classes with similar functionality to existing classes in 1.0, so 4.0 added new classes with similar functionality to existing classes in 3.5. If you are asking "what are the differences in functionality between X and Y", you should edit your question to say just that. At the moment it's not clear if that's what you're asking. – Goldia

I changed the title to "What .NET 4.0 System.Collections.Concurrent collection added in functionality to .NET 3.0 SynchronizedCollection?". Has it made the question more clear? I am asking because I'm learning and could not find ready hints (on differences), if all is clear to experienced programmer, just answer the question. But mostly I am interested to know why were it impossible to do on the basis of .NET 3.0 – Amphibiotic

Possible duplicate: #4655650 – Bilodeau

@Jakob Christensen, it is not duplicate question, it might be duplicate answer what is not the same from the p.o.v. of both searching and providing answers – Amphibiotic

Answer 1

The major focus of .NET 4.0 was concurrency. Microsoft introduced the System.Threading.Tasks namespace in .NET 4.0, so it made sense to add the System.Collections.Concurrent namespace as well. As mentioned elsewhere the collections in System.Collections.Concurrent are highly performant as they are implemented in a lock-free manner. This is not an easy task and I think this is why Microsoft felt that it belonged in a new major .NET release instead of in a refresh of .NET 3.x. AFAIK the lock-free behavior is among other things implemented using the generic version of Interlocked.CompareExchange which is also new to .NET 4.0.

If you are interested in how to implement the classes of System.Collections.Concurrent, I recommend reading Joe Duffy's book "Concurrent Programming on Windows" which he wrote prior to the release of .NET 4.0. The book shows how some of the concurrent collections are implemented.

Bilodeau

Thanks. That's Joe Duffy's book "Concurrent Programming on Windows" is very funny. No .NET (C#) in title, content, overviews and even on the first 2-3 perusing of sample chapter it seems to be about C++ (or Assembler). I would never have found it even I was specifically searching for such book related to C#-programming – Amphibiotic

@Fulproof: Duffy's book discusses both the .NET API and the Windows API (C++). – Bilodeau

Answer 2

The previous thread-safe collection classes have a rather major flaw, they cannot be iterated in a thread-safe way. Iterating is in general very difficult to make thread-safe, there is no clean way to make code that uses an iterator aware of items that are added to or removed from a collection while the code is iterating the collection. The only truly safe way is to lock access to the collection while the iteration is taking place. That's very undesirable, such a lock will typically be held for a very long time. Next best way is to make a copy of the collection, a copy that won't be modified so can always be safely iterated. Nobody likes the O(n) storage requirement of that solution.

The Synchronized() method in the .NET 1.0 collection classes was particularly troublesome. Microsoft did not make it a secret that iterating isn't thread-safe. They even added code to detect the mishap, it throws an exception when a thread modified the collection. This exception however doesn't occur often enough and bewildered .NET programmers that failed to read the warning label and just flat-out assume that surely a thread-safe collection is safe no matter what you do. Well, it wasn't and calling a collection "thread-safe" when the most basic operation on a collection isn't thread-safe is of course a problem. What Microsoft meant was "it is thread-safe because using threads won't corrupt the collection". What programmers read was "it is thread-safe". Microsoft didn't repeat the same mistake in the .NET 2.0 generic collection classes.

This was addressed in .NET 4, iterating the new collections won't throw an exception. They will not otherwise do anything to prevent your code from misbehaving because it is reading stale data. That's an unsolvable problem that you have to solve yourself.

Staghound

1) I always believed that thread-unsafety arises from CRUD modifications (adding, removing, changing/updating)? Or your " terated", "iterating" imply something different? 2) I really became confused after reading your answer what is thread safe. Pre-.NET 4.0 collections was not thread-safe as well as .NET 4 collections are not thread-safe – Amphibiotic

Well, sure, collections that are not being modified are always thread-safe. A class like List<> is fundamentally thread-unsafe, it can become corrupted when you don't use a lock. The Count property can mismatch the actual number of items in the list. The collections returned by Synchronized() are practically thread-unsafe. They won't become corrupted like List<> but they'll throw exceptions that make your program fail. – Staghound

Well, it is already my head is not safe. "It isn't definite that reading is ok -- you never know what is happening under the hoods -- for example, a getter might need to initialize data on first usage (therefore writing to local fields)===For Strings, you are in luck -- they are immutable, so all you can do is read them. With other types, you will have to take precautions against them changing in other threads while you are reading them." – Amphibiotic

I'm just wondering , besides your vast knowledge , how do you know the historic story ? where it started, when it was fixed and what was the problems.... I admire your knowledge. – Limestone

Answer 3

The linked duplicate answers the 'new functionality' part of your question. I will just speak here to this note:

what it is that that made it impossible to do on the basis of .NET 3.0

My guess is nothing made it impossible. I suspect that if you get a decompile of the new classes in .NET 4.0 and (with syntax alterations as required) create such classes yourself against .NET 3.5, they would work perfectly well.

This is the same as how, say, Enumerable.Zip can be manually implemented in .NET 3.5, if .NET 4.0 is not available. Indeed, I remember seeing somewhere an implementation of LINQ-like libraries against .NET 2.0 (without the C# language features of course!).

In general with .NET, unless libraries depend on new CLR features, there's nothing stopping the library being reimplemented against an earlier version. One notable example of the situation is generics: prior to .NET 2.0, there was no way of getting typesafe collection support without handcranking each different class, perhaps using codegen tools such as CodeSmith.

So if nothing prevented the existence of the Concurrent classes in .NET 3.5, why didn't Microsoft create them then? Simply time and budget - shipping is a feature, and some features are less important than that, so they have to wait till a later version.

Goldia

Would not it be more logical to include the classes based on .NET 3.0 in .NET3.8 (3.x) like it was with .NET 3.5 which is just qualitative extension of .NET 3.0? – Amphibiotic