Topics

• interrupt handling
• circular buffers
• atomic hardware operations
• going to sleep safely

Elements of Interrupt Handling

• registration of handler
• interaction with hardware
• what is safe to do inside a handler
• deregistration of handler
• probing for interrupts
• tasklets and BHs
• interrupt sharing

Interrupt Masking/Disabling/Blocking

Independent mechanisms exist at several levels:

• CPU
  • can be set to ignore all interrupts
  • on some architectures, may specify a priority level
  • interrupts stay "pending" until unmasked/unblocked
  • e.g., see local_irq_disable
• Interrupt Controller
  • sits between CPU and devices that generate interrupts
  • can be instructed by CPU not to pass interrupts through
  • e.g., see disable_8259A_irq
• I/O Device
  • generates interrupts
  • may be instructed whether it is OK to generate an interrupt
  • generally waits for interrupt to be "acknowledged" by CPU
  • e.g., see enabling of parallel port interrupt in short.c
• Linux Software IRQ "Wrapper" Layer
  • interrupt actually happens and is handled by common handler code
  • driver-installed handler is not called, if "disabled"
  • driver-installed handler will be called later, when IRQ is "enbled"
  • e.g., see disable_irq_nosync

Terminology varies across different operating systems, machine architectures, and textbooks. The terms "enabled/disable", "block/unblock", "mask/unmask" are used for all the levels. You need to be aware of the specific context to intepret what is being said.

Be carefull when programming that you know the level at which you are working, and use the right mechanism.

Enabling Interrupt Reporting

• most devices don't generate interrupts until told to do so
• for PC parallel port this is done by setting "bit 4" (the fifth bit!) of port 2
• the parallel interface generates an interrupt whenever pin 10 (so-called ACK) changes from low to high ("edge triggered")
• example in short.c
• other interfaces have other protocols for enabling and generating interrupts

How to Mask/Block All Interrupts

local_irq_save(flags)
local_irq_restore(flags);

• these are macros, so no "&" is needed
• avoid doing this if at all possible
• especially, never use local_irq_disable() (why?)
• nonpremptible sections of any kind must always be very short
• they adversely affect system performance (why?)
• the effect of this macro only disables interrupts on one processor
• in many situations this is the wrong semantics; you really need the combination of a spinlock with irq masking (why?)

You may still find calls to cli, which is deprecated. If you must disable interrupts, use local_irq_save.

Installing a Handler in Linux

int request_irq (unsigned int irq,
      irqreturn_t (*handler)(int, void *, struct pt_regs ();
      unsigned long flags,
      const char *dev_name,
      void *dev_id);

• another case of allocating a finite resource
• irq - the IRQ number
• handler - function to execute
• flags -
  • SA_INTERRUPT - handler runs with all interrupts disabled on the current processor
    used for "fast" interrupt handlers
  • SA_SHIRQ - IRQ is shareble
  • SA_SAMPLE_RANDOM - may be used to generate randomness
• dev_name - string to identify new owner
• dev_id - points to device structure that now owns IRQ

Uninstalling a Handler in Linux

void free_irq (unsigned int irq, void *dev_id);

• See example of call to free_irq in short_cleanup<
  • Handler should bormally be removed on last release of device, instead of device driver finalization (why?)

You should be aware that Linux provides quite a few layers between the device driver and the actual hardware interrupt handling machinery. That is good, for the writer of a device driver. It is not o good from the point of view of really understanding what is going on in the hardware.

To get at lower levels, you might look at the following files:

• include/asm-i386/hw_irq.h
• include/asm-i386/irq.h
• arch/kernel/i8259.c
• arch/i386/kernel/irq.c
• arch/i386/kernel/entry.S

Observe that Linux provides a standard wrapper called common_interrupt in entry.S, which is the handler for several interrupts (the ones that don't require special treatment).

The macro SAVE_ALL is used to save state.

The restoration of state and the return from the interrupt ("iret") instruction are done via the RESTORE_ALL macro.

The jump to common_interrupt comes from the actual list of interrupt handler pointers, which start at irq_entries_start.

This common handler saves the hardware state and then calls do_irq (in arch/i386/irq.c), which in turn (indirectly) calls handle_IRQ_event. The latter finally loops through all the device-driver supplied interrupt handlers, calling each of the handlers that has been registered for the IRQ.

While you are looking at entry.S is a good time to look at sys_call_table, the list of kernel entry points for system calls.

The /proc Interfaces for Interrupts

/proc/interrupts (not architecture-dependent)

           CPU0       CPU1       CPU2       CPU3       
  0: 1586341347 1585528157 1777830968 1777774150    IO-APIC-edge  timer
  1:          8        206        226        122    IO-APIC-edge  i8042
  8:          1          0          0          0    IO-APIC-edge  rtc
  9:          0          0          0          0   IO-APIC-level  acpi
 14:    5884978   15312126   16861986   11370803    IO-APIC-edge  ide0
 15:   11570235    9788762   11114601   16580399    IO-APIC-edge  ide1
169:          0          0          0          0   IO-APIC-level  uhci_hcd
177:          0          0          0          0   IO-APIC-level  uhci_hcd
185:          0          0          0          0   IO-APIC-level  uhci_hcd
201: 1141842659          0          0          0   IO-APIC-level  eth0
NMI:          0          0          0          0 
LOC: 2433069657 2433069656 2433069655 2433069654 
ERR:          0
MIS:          0

The above gives you the number of interrupts handled so far, the kind of interrupt controller, and the current binding, just of IRQs that are currently allocated.

/proc/stat (architecture-dependent)

cpu  1233025445 63381 248114678 1196507856 11289489 541966 1281297
cpu0 254094243 19779 52695947 358138543 6237229 338999 1181299
cpu1 309512428 17953 56288118 304523505 2266333 60294 37391
cpu2 291520890 13109 71285374 308131585 1653202 69933 31928
cpu3 377897883 12538 67845237 225714222 1132723 72739 30678
intr 7967851846 2432553280 562 0 2 2 0 0 0 1 0 0 0 0 0 49430146 49054250 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1141846307 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ctxt 2796936863
btime 1110394705
processes 3104956
procs_running 2
procs_blocked 0

Writing a Handler

See example short_interrupt and others in short.c.

• handlers are usually (intentionally) short and simple
• if there is more work, it should be done in a tasklet, or thread
• handler may use parameters irq, dev_id, and regs to figure out what caused the interrupt
  This allows sharing handlers

Enabling and Disabling Interrupts at Software Wrapper Level

• disable_irq - disables just one IRQ (simulated in software)
• The function disable_irq_nosync - does not wait for a currently executing handler (if any) to complete
• enable_irq

disable_irq_nosync calls the disable method of a specific IRQ's descriptor:

desc->handler->disable(irq)

These descriptors are in the array irq_desc[].

The disable method depends on the specific interrupt controller device. For example, for the i8259A it is set in make_8259A_irq to a values stored in i8259A_irq_type, which is a reference to the function disable_8259A_irq. This actually disables the IRQ at the interrupt controller (PIC) if that is possible.

The execution of the driver-level handler is also disabled in software. See the code of disable_irq_nosync via the link above, which ORs IRQ_DISABLED into the IRQ descriptor's status, and the corresponding test of IRQ_DISABLED in _do_irq.

Probing/Autodetecting IRQ Number

• trial-and-error
• install handler; generate interrupt; see if it was handled
• See an example of the standard way of doing this using prove_irq_on/off in short_kernelprobe.
• probe_irq_on
  • returns a bit mask of all the interrupts that are unassigned
  • has the side-effect of enabling them all, at the PIC level
• probe_irq_off
  • returns the IRQ number of the one interrupt that was received, if there was only one
  • returns zero if no interrupts were received
  • returns a negative value if interrupts were received for more than one IRQ number
  • has the side-effect of disabling the interrupts specified by the mask (which should be the unassigned interrupts)
• See also the do-it-yourself probing example short_selfprobe.

Probing is necessary for some drivers. A driver needs some way to find out whether the device supports interrupts, and which IRQ(s) it is using. If the device is not capable of telling the driver what IRQ it wants to use, or letting the driver tell it what IRQ to use, in some reliably way, the driver needs to probe.

Bottom Halves

• handler has two parts
• "top half" is called via the hardware interrupt
• "bottom half" is scheduled by the top half, but executes later, after return from the interrupt
• bottom half may be a tasklet, or an old-style "BH"

Note: There was an old "bottom half" mechanism that became deprecated in favor of tasklets by kernel 2.4, and in kernel 2.6 no longer exists. The term remains as an abstraction, meant to cover tasklets and work queues.

Tasklets

• are used to schedule execution of all but the short initial part of the interrupt handler
• are executed at some later time
• are guaranteed to run on the same CPU as the function that schedules them (the first time, if scheduled more than once)
• if tasklet is scheduled more than once before it has started executing (because the interrupt arrives more than once), the tasklet only executes once
• another interrupt may be received while the tasklet is running, so locking between tasklet and handler may still be required

See example of tasklet scheduled from a handler in short_tl_interrupt.

• declaration of tasklet object
• declaration of tasklet function
• interrupt handler that schedules tasklet
• installation of this handler

To see a part of how tasklet scheduling is done, take a look at the definitions of tasklet_schedule (in include/linux/interrupt.h) and __tasklet_schedule (in kernel/softirq.c).

To see one place where tasklets are scheduled, look at the call to __do_softirq (declared in kernel/softirq.c) in do_softirq (declared in kernel/irq.c).

Interrupt Sharing

• does not work well with edge-triggered interrupts
  interrupts from other devices may be lost while one device is holding the line active
• well behaved drivers can share a single IRQ
• all must use the SA_SHIRQ option when requesting the IRQ
• the dev_id argument must be unique
• the kernel function do_irq calls __do_IRQ, which call each of the registered handlers
• only one of them actually handles the interrupt
• each handler must be able to tell whether it is the one that needs to handle the interrupt this time

See example of a handler for a shared IRQ in short.c.

The parallel printer protocol does not allow for sharing, but the toy parallel device (LEDs, plus jumper between pins 9 and 10) does.