Why disabling interrupts disables kernel preemption and how spin lock disables preemption

Question

I am reading Linux Kernel Development recently, and I have a few questions related to disabling preemption.

1. In the "Interrupt Control" section of chapter 7, it says:

   Moreover, disabling interrupts also disables kernel preemption.

   I also read from the book that kernel preemption can occur in the follow cases:

   When an interrupt handler exits, before returning to kernel-space.
   When kernel code becomes preemptible again.
   If a task in the kernel explicitly calls schedule()
   If a task in ther kernel blocks (which results in a call to schedule())

   But I can't relate disabling interrupts with these cases.

2. As far as I know, a spinlock would disable preemption with the preempt_disable() function.

   The post What exactly are "spin-locks"? says:

   On a single core machine a spinlock is simply a "disable interrupts" or "raise IRQL" which prevents thread scheduling completely.

   Does preempt_disable() disable preemption by disabling interrupts?

Ashil

Answer 1

I am not a scheduler guru, but I would like to explain how I see it. Here are several things.

1. preempt_disable() doesn't disable IRQ. It just increases a thread_info->preempt_count variable.
2. Disabling interrupts also disables preemption because scheduler isn't working after that - but only on a single-CPU machine. On the SMP it isn't enough because when you close the interrupts on one CPU the other / others still does / do something asynchronously.
3. The Big Lock (means - closing all interrupts on all CPUs) is slowing the system down dramatically - so it is why it not anymore in use. This is also the reason why preempt_disable() doesn't close the IRQ.

You can see what is preempt_disable(). Try this: 1. Get a spinlock. 2. Call schedule()

In the dmesg you will see something like "BUG: scheduling while atomic". This happens when scheduler detects that your process in atomic (not preemptive) context but it schedules itself.

Good luck.

Rieth

Thank you for your detailed answer. But I still don't understand why scheduler isn't working after disabling interrupts on a single-CPU machine. – Ashil

No interrupts - no clock. No clock - no timers. – Rieth

What will be the effect if i only disable the preemption(not disabling interrupts) and at the same time interrupt occur in system and what will happen after disabling interrupt also? and in which case "preempt_count" variable will be more effectively. (studying interrupt handling in arm) – Manicotti

@Sebastian Mountaniol : "The Big Lock (means - closing all interrupts on all CPUs)" Can you explain this? Big lock is actually a lock although a global one for all LWPs. But it does not amounts to disabling of interrupts. – Infusible

Answer 2

In a test kernel module I wrote to monitor/profile a task, I've tried disabling interrupts by:

1 - Using local_irq_save()

2 - Using spin_lock_irqsave()

3 - Manually disable_irq() to all IRQs in /proc/interrupts

In all 3 cases I could still use the hrtimer to measure time even though IRQs were disabled (and a task I was monitoring got preempted as well).

I find this veeeeerrrryyyy strange... I personally was anticipating what Sebastian Mountaniol pointed out -> No interrupts - no clock. No clock - no timers...

Linux kernel 2.6.32 on a single core, single CPU... Can anyone have a better explanation ?

Lochia

I thought some of the Linux kernel timers use a trick of last timer interrupt plus the CPU cycle timer. So you'd get accurate time until the cycle counter overflow. – Semibreve

Answer 3

1. preempt_disable() doesn't disable the interrupts. It however increments the count of preempt counter. Let's say you call preempt_disable() n times in your code path, preemption will only enable at the nth preempt_enable().
2. disabling interrupts to prevent preemption : not a safe way. This will undoubtedly disable normal kernel preemption because scheduler_tick() won't be called on system tick (no interrupt handler invoked). However, if the program triggers the schedule function, preemption will occur if preempt_disable() was not invoked.
3. In linux, raw_spin_lock() doesn't disable local interrupts which may lead to deadlock. For instance, if an interrupt handler is invoked which tries to lock already held spin lock, it won't be able to unless the process itself releases it which is not possible as interrupt return wouldn't occur. So, it's better to use raw_spin_lock_irq(), which disables interrupts.

Edacity

Answer 4

Interrupt disabling disables some forms of kernel preemption, but there are other ways kernel preemption can happen. For this reason, disabling interrupts is not considered a safe way to prevent kernel preemption.

For instance, with interrupts disabled, cond_resched() would still cause preemption, but wouldn't if preemption was explicitly disabled.

This is why, in regards to your second question, spin locks don't use interrupt disabling to disable preemption. They explicitly call preempt_disable() which increments preempt_count, and disables all ways that preemption can happen except for explicit calls to schedule().

Dutch