.NET Multithreading and Synchronization

Sometimes we write codes that perform a certain set of functions for many different independent values in identical way.  These set of operations may take remarkably amount of time.

For example, suppose that the program needs to calculate the salaries of all employees and write the results to database. Here is the simplest implementation:

Code:

 void RunSingleThread(List<long> employeeIdList)
        {
            foreach (var employeeId in employeeIdList)
            {
                CalculateSalaryAndWriteToDb(employeeId);
            }
        }

Since every employee is independent from each other, such a task can be performed for many employees in parallel way. Here we can utilize multithreading to shorten the time. We need to implement some steps to achieve it:

n  Decide how many items will run in parallel. As a best practice, this number should be the processor count of the machine.

n Divide the list of items into separated lists. Each of these lists need to have number of parallel items at maximum.

n  In a loop, run all items of executing list in parallel and synchronize them.

n  When all items of executing list completes, skip to next list in the loop.

Here is the C# Code:

        void CalculateSalaryAndWriteToDb(long employeeId, ManualResetEvent mre)
        {
            // Make calculations that takes time.
            // Write To Database.

            mre.Set();
        }

        void RunMultiThread(List<long> employeeIdList)
        {
            int parallelItemCount = Environment.ProcessorCount;
            List<List<long>> groupedItems = GroupItems<long>(employeeIdList, parallelItemCount);
            foreach (List<long> group in groupedItems)
            {
                List<ManualResetEvent> mreList = new List<ManualResetEvent>();
                foreach (var employeeId in group)
                {
                    ManualResetEvent mre = new ManualResetEvent(false);
                    mreList.Add(mre);
                    ThreadPool.QueueUserWorkItem(o => CalculateSalaryAndWriteToDb(employeeId , mre));
                }
                WaitHandle.WaitAll(mreList.ToArray());
            }
        }

        List<List<T>> GroupItems<T>(List<T> allItems, int groupMaxCount)
        {
            List<List<T>> res = new List<List<T>>();
            int count = 0;
            List<T> temp = new List<T>();
            foreach (var item in allItems)
            {
                count++;
                temp.Add(item);
                if (count % groupMaxCount == 0)
                {
                    res.Add(temp.ToList());
                    temp.Clear();
                }
            }
            if (temp.Any())
            {
                res.Add(temp);
            }
            return res;
        }

Note that, WaitHandle.WaitAll function cannot synchronize more than 64 synchronization objects. System.NotSupportedException is thrown by .NET Framework.