Topics

• kmalloc and "friends"
• get_free_page and "friends"
• vmalloc and "friends"
• memory usage pitfalls

kmalloc

• allocates contiguous (virtually and physically) memory
• may block
• does not clear the memory it obtains
• allocates memory only in certain predefined sizes
  e.g., see the Buddy System
• maximum size is 128 KB
• smallest size is 32 or 64 bytes
• will round up, and may double size of request
• sizes that are exact powers of two are likely to be bad
  e.g. 1000 will probably occupy 1024 bytes, but 1024 may occupy 2048 bytes

See kmalloc in mm/slab.c.

Flags Argument of kmalloc

basic:

• GFP_KERNEL - (get_free_pages)
  done for a process running in kernel space
  process may block while a page is freed up
  (implies calling function must be reentrant)
• GFP_ATOMIC -
  can be called outside a process context
  e.g., by an interrupt handler, task queue function, tasklet, or kernel timer function
  may fail of there is no free memory immediately available
• GFP_BUFFER - may sleep, used for buffer cache
  fewer attempts to flush dirty pages to disk
• GFP_USER - for user-space pages; may sleep, low priority
• GFP_HIGHUSER - like GFP_USER, but for high memory
• GFP_NOIO - like GFP_KERNEL, but no I/O permitted to satisfy request
• GFP_NOFS - like GFP_KERNEL, but no filesystem calls to satisfy request

Besides one of the above, additional flags maybe "or"ed in to select additional attributes of the allocation process.

• __GFP_DMA - include only DMA-compatible memory in search
  (can be OR'd to GFP_KERNEL or GFP_ATOMIC)
• __GFP_HIGHMEM - include high memory in search
• __GFP_COLD - don't care about whether memory is in-cache
• __GFP_NOWARN - suppress console warnings on failure
• __GFP_REPEAT - retry once
• __GFP_NOFAIL - keep retrying forever
• __GFP_NORETRY - don't retry at all

Memory Zones

• DMA-compatible memory
• Normal memory
• High memory - requires special handling to be accessed

Memory Allocation "Lookaside" (Slab) Caches

• hold recently deallocated equal-sized freed objects
• a cache is of type kmem-cache_t
• kemem_cache_create - returns a pointer to a newly created cache
  • name - useful for debugging
  • size - size of individual memory areas in this cache
  • offset - specifies alignment of objects (normally 0)
  • flags
    • SLAB_NO_HEAP - protects against "raiding" when memory is low
    • SLAB_HWCACHE_ALIGN
    • SLAB_CACHE_DMA
  • pointer to constructor function
  • pointer to destructor function (must be idempotent)
• kmem_cache_alloc - allocates an object from a cache
• kmem_cache_free - frees an object to a cache
• kmem_cache_destroy - unlinks a cache
  failure indicates some objects have not yet been freed (memory leak)

The use of "cache" here has no direct relation to any hardware memory caching, and the word "lookaside" has nothing to with the TLB.

Constructor/Destructors

• destructor must be idempotent
• both have flags argument
  • SLAB_CTOR_ATOMIC - constructor/desctructor is not allowed to block
  • SLAB_CTOR_CONSTRUCTOR - this is a constructor call
    allows overloading one function to serve as both constructor and destructor

Examples

For example, see scullc.

• scullc_cache pointer declaration
• kmem_cache_create call
• kmem_cache_alloc call
• kmem_cache_free call
• kmem_cache_destroy call

get_free_page and "friends"

• allocation:
  • get_zeroed_page
  • __get_free_page - does not zero the page
  • __get_free_pages - allocates several pages
  • __get_dma_pages
  • order - specifies number of pages as 2^order
  • flags - similar to kmalloc allocation may block or fail, accordingly
• deallocation:
  • free_page
  • free_pages

Examples

For example, see scullp.

• PAGE_SIZE used to compute quantum in scullp_read
• __get_free_pages call in scullp_write
• free_pages call in scullp_trim

vmalloc and "friends"

• vmalloc - allocates virtual memory (may fail)
• vfree - releases memory allocated by vmalloc
• ioremap - used to map I/O memory into physical memory (more on this in a later chapter)
• iounmap - undoes effect of ioremap

Comparisons

• both kmalloc and vmalloc return addresses that are mapped by the MMU
• kmalloc and get_free_page(s)
  • allocate contigous range of both real memory and virtual address space
  • one-to-one mapping of virtual address to physical address
    possibly offset by PAGE_OFFSET constant
  • requires no modification to page tables
    (they are already set up for kernel memory)
• vmalloc
  • allocates a range of virtual address space
  • virtual address has no fixed mapping to physical memory
    (this is only virtual memory)
  • memory is not seen as physically contiguous by DMA I/O devices
  • requires setting up page table entries
  • may block (can't use in interrupt handler)
  • good use: large sequential buffer only accessed by software
  • do not use for I/O buffers accessed by DMA hardware
• ioremap
  • does not allocate any memory
  • builds new page table entries
  • allows access of I/O memory through virtual addresses
  • do not use ioremaped addresses like regular pointers
    (for portability)
  • only use the macros readb, writeb, etc.

These all work on the kernel virtual address space. The operations to map/remap ranges of the user virtual address space (of a process) will be covered later. The main operation for mapping physical addresses into user space is remap_page_range.

Beware: vmalloc and ioremap work in entire pages, only.

Examples

For example, see scullp.

• PAGE_SIZE used to compute quantum in scullv_read
• vmalloc call in scullv_write
• vfree call in scullv_trim

Pitfalls

• Failure to provide code to handle failures of calls,
  including every memory allocation
• Using too much memory
• Designing a driver that has no built-in limit on memory usage

Beware: You must include recovery code for the case of failure on every call to any memory allocator. If you do not you will be contributing to the number of kernel/driver bugs.

Do you know what happens when the kernel runs out of physical memory?

Per-CPU Variables

• separate copy kept for each CPU
  essentially, an array of similar objects, indexed by CPU number
• when you access it, you (magically) get the copy for the current CPU
• can reduce need for locking, in some applications
• only need to lock the current CPU (no preemption):

  int cpu;
  ...
  cpu = get_cpu();
  ptr = per_cpu_ptr(per_cpu_var, cpu);
  ... work with ptr ...
  put_cpu();

• based on percpu.h
• example in linux/percpu_counter.h

On some architectures (which provide direct support for per-CPU data) this may be a small bounded resource, so take care to keep size and number of per-CPU variables small.