Difference linq and plinq

Question

What is the difference between these two?

What is the best way to compare ?

It is always better plinq ?

When we use plinq ?

Inextinguishable

some good info here scip.be/index.php?Page=ArticlesNET08&Lang=EN – Conidiophore

I think it's fairly safe to assume that if it were always better to use PLINQ, LINQ wouldn't exist. Consequence: It's not always better to use PLINQ. – Rhinoplasty

Answer 1

Linq is a collection of technologies that work together to solve a similar family of problems - in all of them you have a source of data (xml file or files, database contents, collection of objects in memory) and you want to retrieve some or all of this data and act on it in some way. Linq works on the commonality of that set of problems such that:

var brithdays = from user in users where
  user.dob.Date == DateTime.Today && user.ReceiveMails
  select new{user.Firstname, user.Lastname, user.Email};
foreach(bdUser in birthdays)
  SendBirthdayMail(bdUser.Firstname, bdUser.Lastname, bdUser.Email);

And the equivalent (explicit use of Linq-related classes and methods with a traditional C# syntax):

var birthdays = users
  .Where(user => user.dob.Date == DateTime.Today)
  .Select(user => new{user.Firstname, user.Lastname, user.Email});
foreach(bdUser in birthdays)
  SendBirthdayMail(bdUser.Firstname, bdUser.Lastname, bdUser.Email);

Are both examples of code that could work regardless of whether it's going to be turned into database calls, parsing of xml documents, or a search through an array of objects.

The only difference is what sort of object users is. If it was a list, array, or other enumerable collection, it would be linq-to-objects, if it was a System.Data.Linq.Table it would be linq to sql. The former would result in in-memory operations, the latter in a SQL query that would then be deserialised to in-memory objects as late as possible.

If it was a ParallelQuery - produced by calling .AsParallel on an in-memory enumerable collection - then the query will be performed in-memroy, parallelised (most of the time) so as to performed by multiple threads - ideally keeping each core busy moving the work forward.

Obviously the idea here is to be faster. When it works well, it does.

There are some downsides though.

First, there's always some overhead to getting the parallelisation going, even in cases where it ends up not being possible to parallelise. If there isn't enough work being done on data, this overhead will out-weigh any potential gains.

Second, the benefits of parallel processing depends on the cores available. With a query that doesn't end up blocking on resources on a 4-core machine, you theoretically get a 4-times speed up (4 hyper-threaded might give you more or even less, but probably not 8-times since hyper-threading's doubling of some parts of the CPU doesn't give a clear two-times increase). With the same query on a single-core, or with processor affinity meaning only one core is available (e.g. a webserver in "web-garden" mode), then there's no speed-up. There could still be a gain if there's blocking on resources, but the benefit depends on the machine then.

Third, if there's any shared resource (maybe an collection results are being output to) is used in a non-threadsafe way, it can go pretty badly wrong with incorrect results, crashes, etc.

Fourth, if there's a shared resource being used in a threadsafe way, and that threadsafety comes from locking, there could be enough contention to become a bottleneck that undoes all the benefits from the parallelisation.

Fifth, if you've a four-core machine working on more or less the same algorithm on four different threads (perhaps in a client-server situation due to four clients, or on a desktop situation from a set of similar tasks higher in the process), then they're alreay making the best use of those cores. Splitting the work in the algorithm up so as to be handled across all four cores means you've moved from four threads using one core each to 16 threads fighting over four cores. At best it'll be the same, and likely overheads will make it slightly worse.

It can still be a major win in a lot of cases, but the above should make it clear that it won't always.

Stablish

Answer 2

I also wanted to know when to use PLINQ instead of LINQ so I ran some tests.

Summary: There are two questions to answer when deciding whether to use LINQ or PLINQ to run a query.

1. How many iterations are involved in running the query (how many objects are in the collection)?

2. How much work is involved in an iteration?

Use LINQ unless PLINQ is more performant. PLINQ can be more performant than LINQ if querying the collection involves too many iterations AND/OR each iteration involves too much work.

But then two difficult questions arise:

1. How many iterations are too many iterations?
2. How much work is too much work?

My advice is to test your query. Test once using LINQ and once using PLINQ and then compare the two results.

Test 1: Increasing the number of iterations in the query by increasing the number of objects in the collection.

The overhead of initialising PLINQ takes around 20ms. If PLINQ's strengths aren't utilised, this is wasted time because LINQ has 0ms overhead.

The work involved in each iteration is always the same for each test. The work is kept minimal.

Definition of work: Multiplying the int (object in the collection) by 10.

When iterating 1 million objects where each iteration involves minimal work, PLINQ is faster than LINQ. Although in a professional environment, I've never queried or even initialised a collection of 10 million objects in memory so this might be an unlikely scenario where PLINQ happens to be superior to LINQ.

╔═══════════╦═══════════╦════════════╗
║ # Objects ║ LINQ (ms) ║ PLINQ (ms) ║
╠═══════════╬═══════════╬════════════╣
║ 1         ║         1 ║         20 ║
║ 10        ║         0 ║         18 ║
║ 100       ║         0 ║         20 ║
║ 1k        ║         0 ║         23 ║
║ 10k       ║         1 ║         17 ║
║ 100k      ║         4 ║         37 ║
║ 1m        ║        36 ║         76 ║
║ 10m       ║       392 ║        285 ║
║ 100m      ║      3834 ║       2596 ║
╚═══════════╩═══════════╩════════════╝

Test 2: Increasing the work involved in a iteration

I set the number of objects in the collection to always be 10 so the query involves a low number of iterations. For each test, I increased the work involved to process each iteration.

Definition of work: Multiplying the int (object in the collection) by 10.

Definition of increasing the work: Increasing the number of iterations to multiply the int by 10.

PLINQ was faster at querying the collection as the work was significantly increased when the number of iterations inside a work iteration was increased to 10 million and I concluded that PLINQ is superior to LINQ when a single iteration involves this amount of work.

"# Iterations" in this table means the number of iterations inside a work iteration. See Test 2 code below.

╔══════════════╦═══════════╦════════════╗
║ # Iterations ║ LINQ (ms) ║ PLINQ (ms) ║
╠══════════════╬═══════════╬════════════╣
║ 1            ║         1 ║         22 ║
║ 10           ║         1 ║         32 ║
║ 100          ║         0 ║         25 ║
║ 1k           ║         1 ║         18 ║
║ 10k          ║         0 ║         21 ║
║ 100k         ║         3 ║         30 ║
║ 1m           ║        27 ║         52 ║
║ 10m          ║       263 ║        107 ║
║ 100m         ║      2624 ║        728 ║
║ 1b           ║     26300 ║       6774 ║
╚══════════════╩═══════════╩════════════╝

Test 1 code:

class Program
{
    private static IEnumerable<int> _numbers;

    static void Main(string[] args)
    {
        const int numberOfObjectsInCollection = 1000000000;

        _numbers = Enumerable.Range(0, numberOfObjectsInCollection);

        var watch = new Stopwatch();

        watch.Start();

        var parallelTask = Task.Run(() => ParallelTask());

        parallelTask.Wait();

        watch.Stop();

        Console.WriteLine($"Parallel: {watch.ElapsedMilliseconds}ms");

        watch.Reset();

        watch.Start();

        var sequentialTask = Task.Run(() => SequentialTask());

        sequentialTask.Wait();

        watch.Stop();

        Console.WriteLine($"Sequential: {watch.ElapsedMilliseconds}ms");

        Console.ReadKey();
    }

    private static void ParallelTask()
    {
        _numbers
            .AsParallel()
            .Select(x => DoWork(x))
            .ToArray();
    }

    private static void SequentialTask()
    {
        _numbers
            .Select(x => DoWork(x))
            .ToArray();
    }

    private static int DoWork(int @int)
    {
        return @int * 10;
    }
}

Test 2 code:

class Program
{
    private static IEnumerable<int> _numbers;

    static void Main(string[] args)
    {
        _numbers = Enumerable.Range(0, 10);

        var watch = new Stopwatch();

        watch.Start();

        var parallelTask = Task.Run(() => ParallelTask());

        parallelTask.Wait();

        watch.Stop();

        Console.WriteLine($"Parallel: {watch.ElapsedMilliseconds}ms");

        watch.Reset();

        watch.Start();

        var sequentialTask = Task.Run(() => SequentialTask());

        sequentialTask.Wait();

        watch.Stop();

        Console.WriteLine($"Sequential: {watch.ElapsedMilliseconds}ms");

        Console.ReadKey();
    }

    private static void ParallelTask()
    {
        _numbers
            .AsParallel()
            .Select(x => DoWork(x))
            .ToArray();
    }

    private static void SequentialTask()
    {
        _numbers
            .Select(x => DoWork(x))
            .ToArray();
    }

    private static int DoWork(int @int)
    {
        const int numberOfIterations = 1000000000;

        for (int i = 0; i < numberOfIterations; i++)
        {
            @int = @int * 10;
        }

        return @int;
    }
}

Phene

Answer 3

PLinq is the parallel version of Linq. Some queries can be executed on multiple threads and then PLinq gives a performance increase.

However other queries can´t be executed in parallel or will give wrong results if done so. So when to use PLinq is something you should decide on for each query and make sure the performance actually increases.

MSDN has a lot of documentation on it.

Richie

Answer 4

Consider avoiding anonymous types while working with PLINQ because according to Threading in C#, by Joe Albahari:

anonymous types (being classes and therefore reference types) incur the cost of heap-based allocation and subsequent garbage collection.

(...)

stack-based allocation is highly parallelizable (as each thread has its own stack), whereas all threads must compete for the same heap — managed by a single memory manager and garbage collector.

Impressure

Answer 5

PLINQ can significantly increase the speed of LINQ to Objects queries by using all available cores on the host computer more efficiently. This increased performance brings high performance computing power onto the desktop.

Dunbar