In short, normal driver writers have two choices. First, do you need a schedulable entity to perform your deferred work—fundamentally, do you need to sleep for any rea-son? Then work queues are your only option. Otherwise, tasklets are preferred. Only if scalability becomes a concern do you investigate softirqs.
Locking Between the Bottom Halves
We have not discussed locking yet, which is such a fun and expansive topic that we devote the next two chapters to it. Nonetheless, you need to understand that it is crucial to protect shared data from concurrent access while using bottom halves, even on a single processor machine. Remember, a bottom half can run at virtually any moment.You might want to come back to this section after reading the next two chapters if the concept of locking is foreign to you.
One of the benefits of tasklets is that they are serialized with respect to themselves:
The same tasklet will not run concurrently, even on two different processors.This means you do not have to worry about intra-tasklet concurrency issues. Inter-tasklet concur-rency (that is, when two different tasklets share the same data) requires proper locking.
Because softirqs provide no serialization, (even two instances of the same softirq might run simultaneously), all shared data needs an appropriate lock.
If process context code and a bottom half share data, you need to disable bottom-half processing and obtain a lock before accessing the data. Doing both ensures local and SMP protection and prevents a deadlock.
If interrupt context code and a bottom half share data, you need to disable interrupts and obtain a lock before accessing the data.This also ensures both local and SMP protec-tion and prevents a deadlock.
Any shared data in a work queue requires locking, too.The locking issues are no dif-ferent from normal kernel code because work queues run in process context.
Chapter 9,“An Introduction to Kernel Synchronization,” provides a background on the issues surrounding concurrency, and Chapter 10 covers the kernel locking primitives.
These chapters cover how to protect data that bottom halves use.
Disabling Bottom Halves
Normally, it is not sufficient to only disable bottom halves. More often, to safely protect shared data, you need to obtain a lock and disable bottom halves. Such methods, which you might use in a driver, are covered in Chapter 10. If you are writing core kernel code, however, you might need to disable just the bottom halves.
To disable all bottom-half processing (specifically, all softirqs and thus all tasklets), call
local_bh_disable().To enable bottom-half processing, call local_bh_enable().Yes, the function is misnamed; no one bothered to change the name when the BH interface gave way to softirqs.Table 8.4 is a summary of these functions.
ptg The calls can be nested—only the final call to local_bh_enable() actually enables
bottom halves. For example, the first time local_bh_disable() is called, local softirq processing is disabled. If local_bh_disable() is called three more times, local processing remains disabled. Processing is not reenabled until the fourth call to local_bh_enable().
The functions accomplish this by maintaining a per-task counter via the
preempt_count (interestingly, the same counter used by kernel preemption).8 When the counter reaches zero, bottom-half processing is possible. Because bottom halves were dis-abled, local_bh_enable() also checks for any pending bottom halves and executes them.
The functions are unique to each supported architecture and are usually written as complicated macros in <asm/softirq.h>.The following are close C representations for the curious:
/*
* disable local bottom halves by incrementing the preempt_count
*/
void local_bh_disable(void) {
struct thread_info *t = current_thread_info();
t->preempt_count += SOFTIRQ_OFFSET;
} /*
* decrement the preempt_count - this will ‘automatically’ enable
* bottom halves if the count returns to zero
*
* optionally run any bottom halves that are pending
*/
void local_bh_enable(void) {
8 This counter is used both by the interrupt and bottom-half subsystems. Thus, in Linux, a single per-task counter represents the atomicity of a per-task. This has proven useful for work such as debugging sleeping-while-atomic bugs.
Table 8.4 Bottom Half Control Methods
Method Description
void local_bh_disable() Disables softirq and tasklet processing on the local processor
void local_bh_enable() Enables softirq and tasklet processing on the local processor
ptg Conclusion 159
struct thread_info *t = current_thread_info();
t->preempt_count -= SOFTIRQ_OFFSET;
/*
* is preempt_count zero and are any bottom halves pending?
* if so, run them
*/
if (unlikely(!t->preempt_count && softirq_pending(smp_processor_id()))) do_softirq();
}
These calls do not disable the execution of work queues. Because work queues run in process context, there are no issues with asynchronous execution, and thus, there is no need to disable them. Because softirqs and tasklets can occur asynchronously (say, on return from handling an interrupt), however, kernel code may need to disable them.With work queues, on the other hand, protecting shared data is the same as in any process con-text. Chapters 8 and 9 give the details.
Conclusion
In this chapter, we covered the three mechanisms used to defer work in the Linux kernel:
softirqs, tasklets, and work queues.We went over their design and implementation.We dis-cussed how to use them in your own code and we insulted their poorly conceived names.
For historical completeness, we also looked at the bottom-half mechanisms that existed in previous versions of the Linux kernel: BH’s and task queues.
We talked a lot in this chapter about synchronization and concurrency because such topics apply quite a bit to bottom halves.We even wrapped up the chapter with a discus-sion on disabling bottom halves for reasons of concurrency protection. It is now time to dive head first into these topics. Chapter 9 discusses kernel synchronization and concur-rency in the abstract, providing a foundation for understanding the issues at the heart of the problem. Chapter 10 discusses the specific interfaces provided by our beloved kernel to solve these problems. Armed with the next two chapters, the world is your oyster.
ptg
ptg