What's the most CPU-efficient way to "waste time" in a thread?

Question

I have a number of threads (100's) that each execute for a few seconds at a time. When they are executing, they spend a significant amount of that time waiting for a response from another system (a serial device). I am mindful that having 100 threads executing at once could be a resource hog so I actually limit the number of threads that can start at any one time.

It occurs to me though that there must be good and bad ways of waiting for an external event inside a thread. Is this approach CPU-intensive?:

send command ;
repeat
until response arrived ;
process response ;    

and does this approach make it more efficient?:

send command ;
repeat
    Sleep (20) ;
until response arrived ;
process response ;  

* ADDITIONAL INFO *

The environment is x86 Windows XP. The thread code is a long and involved series of interactions with a serial device but in general, it consists of writing characters to a COM port (using the AsyncFree serial library) and waiting for characters to be returned by camping on the incoming characters buffer and processing them when they arrive. I imagine the serial library makes device reads and writes. The time in the thread can be as long as a minute , or as short as a couple of seconds, but most of that time is spent waiting for characters to leave the port, or waiting for the response characters (baud rate is slow), hence my question about the best way for the thread to behave while it is waiting. Currently I am calling Sleep in a loop waiting for CharactersInBuffer to become non-zero, processing each character when it arrives, and exiting the thread when I have the complete response. So the code looks more like (ignoring handling of timeouts, etc):

send command ;
Packet = '' ;
repeat

    repeat
        Sleep (20) ;
    until response character arrived ;
    build Packet

until complete packet arrived
process response ;  

Korwun

Please post your OS, that might be relevant. – Spidery

It depends on what the loop is doing. Can you provide code? – Cocteau

@MarcusAdams: Any loop that's not yielding it's timeslice to the OS or waiting on an event/sync object is going to use all the CPU time that the OS will make available to it. – Scopula

If you don't see your CPU utilisation at 100% then it sounds like you have no problem. Without seeing real code it's hard to say more. until response arrived is the key. What does that actually do? Is it a blocking call? – Acaleph

@rossmcm: There should be a way for the thread to wait for a notification from the OS that a response is available rather than polling for it. – Scopula

@afrazier, tell the OP that. That's why I'm asking. – Cocteau

@MarcusAdams, afrazier. Thanks for your responses. More info added to question. – Korwun

Sleep(0) tells the scheduler what calling thread does not need any leftovers of current timeslice and asks for next available schedule. So, you will get faster response opposed to Sleep(20) and will avoid unneccessary CPU burning with generic spinlock. – Quake

But you should really look into WaitCommEvent and/or overlapped operations for this purpose. – Quake

@Premature I suspect the underlying COM port library uses WaitForEvent (that's where the threads seem to be stuck at when I look at MadExcept dumps) – Korwun

@Korwun If your threads are already blocking you've probably got nothing to do. – Acaleph

TBH, your question could do with some further explanation. OK, you have 100 threads that are waiting for chars on some FIFO buffer queue scheme. When the Apro rxData event is fired by the Apro receive thread, how do you know which of the 100 queues to push the rx. chars on? Is there some other lower-level serial protocol going on that blocks the serial stream up into messages/packets, or is only one of the 100 threads allowed to take control of the port at any time, perform a protocol exchange and then 'release' the port? Sorry, I don't have an overall grasp of how your system works. – Poised

@Martin. There are up to 256 channels, each with their own COM port /FIFO's etc. It's possible for all 256 channels to fire off a thread together (but at present I throttle it to 5-10 or so). All channels are independent and don't ever have to wait for each other, though their OnTerminate events might all arrive together back in the main thread and will be processed "serially", though this processing load is not great. – Korwun

Answer 1

If the thread is truly waiting with something like a WaitForSingleObject, which uses no processor time, then times out, there is no reason to put a delay in the thread with sleep.

Your user isn't waiting on the thread to be responsive, it's not using processor time, and other threads won't be blocked, so there would be no reason to put the thread to sleep.

As David Heffernan indicated in his comment, if it's not using 100% of your CPU now, then there's no problem.

You might use sleep() if you were single threaded and you had to occasionally respond to the user in between waiting on the serial port to respond.

Also, having a thread sleep would not make it more efficient. It would simply yield processor cycles to other threads.

Take a look at sleep(0) as a CPU efficient way of "wasting time" in a thread.

Cocteau

Also, having a thread sleep would not make it more efficient. It would simply yield processor cycles to other threads. Isn't that the point? If I put my thread to sleep to 20 mS and yield processing to other threads, doesn't that make more CPU available to other threads/other processes? If I have 10000 threads and they have all just entered a Sleep (10000) state, what is their combined CPU usage? – Korwun

@rossmom, If I'm in a convoy, stopping my truck to let others pass doesn't increase efficiency. The same amount of work needs to be done. It's simply more polite. If I stop my truck when there's no one to let pass, I'm wasting time. If 10000 threads are sleeping, technically, they are using 0 CPU usage, though the operating system (scheduler) thread will have a lot of work to do, and if you run out of RAM, you'll surely slow down. Using sleep(0) will get everything done as soon as possible while still being polite. Sleep(0) is more efficient than sleep(200), since time is also a resource. – Cocteau

OK, so I now understand that Sleep (0) effectively says "move on, nothing to do here", so if I stick to my current architecture of limiting the number of threads that can start to ten, and replace my Sleep (20) by Sleep (0), I'll have a situation that has low latency and low CPU usage? – Korwun

@Korwun Sleep(0) will only yield if there is a ready to run thread waiting. Otherwise it's a nop. – Acaleph

@rossmcm, with Sleep(0) instead of Sleep(20), you'll have low latency, but you're guaranteed higher CPU usage while there's work to be done. CPU usage isn't bad. If nobody else is using the CPU, why not you? Sleep(0) will keep the computer responsive to other processes, which helps keep the UI responsive, and allows the user to keep watching their movie or surf the web while your program runs. If you want to intentionally make the user wait longer with Sleep(20) just so they don't see the CPU spike, that's your prerogative. I prefer they think their computer's slow instead of my program. – Cocteau

Sleep(0) does not yield to other apps, it yields to other ready threads of the same priority, if there are any. If you have 100 threads with a sleep(0) loop and no other blocking calls, there will be 100 ready threads. If another single-thread app is running and trying to make progress, then its ready thread will get the same CPU as the 100 threads in your app, ie. 1/101 of the available CPU. As others have posted, it all depends on whether the check for available characters blocks or not. If it does, there is no point in sleep(anything) - just comment it out. If it doesnt, then it should! – Poised

Answer 2

The most efficient way to prevent a thread from using CPU time is to put it in a "wait mode."

I don't use delphi at all, but it seems that the fundamentals for that are there. See "Chapter 11. Synchronizers and Events" and more specifically "Event simulation using semaphores".

If you want to wait without using CPU, then use WaitForEvent:

The signal state of the event is examined. If it indicates that the event is signalled, then the internal semaphore is signalled, and the count of threads blocked on the semaphore is decremented. The count of blocked threads is then incremented, and a wait is performed on the internal semaphore.

If this is I/O related, then things work a little different. If it's a socket, then it might already be blocking, if it's asynchronous I/O, then you can use a semaphore and WaitForEvent and so on.

In .NET there is the Monitor.Wait, Monitor.Signal, ManualResetEvent, CountDownLatch, etc., but I don't know what are the equivalent things in delphi.

Gregoriogregorius

Answer 3

I cannot speak for AsyncFree's capabilities, but in general COM port programming in Windows supports Overlapped I/O, so you can efficiently wait for a notification when data arrives by using the WaitCommEvent() function with one of the WaitFor...() family of functions, such as WaitForSingleObject(). The thread can be put into a sleep state until the notify is issues, at which time it "wakes up" to read from the port until there is nothing further to read, then it can go back to sleep until the next notify.

Beberg