Processes and Daemons

• Fundamentally, kernels provide a few logical constructs that mediate access to either real or virtual resources. The two most important in Unix are processes and filesystems.
• You can view the characteristics of processes on a Unix machine with a variety of programs, including ps, top, lsof, and ls.
• The Wikipedia page (at least the version linked here) is quite a good description of the fundamental ideas about daemons.

Processes and Daemons : fork and clone

• A new process is created by fork; or, alternatively, in Linux with clone since processes and threads are both just a task_struct in Linux (the linked sched.h is from 3.16, and is in "include/linux/sched.h".)
• With clone, memory, file descriptors and signal handlers are still shared between parent and child.
• With fork, these are copied, not shared.

Starting a Unix/Linux process

• exec instantiates a new executable:
• Usually, when doing an exec the named file is loaded into the current process's memory space
• Unless the first two characters of the file are #! and the following characters name a valid pathname to an executable file, in which case that file is instead loaded
• If the executable is dynamically linked, then the dynamic loader maps in the necessary bits (not done if the binary is statically linked.)
• Then code in the initial .text section is then executed. (There are three main types of sections:
  • .text sections for executable code
  • .data sections (including read-only .rodata sections)
  • .bss sections (Blocks Started by Symbol) which contains ``uninitialized'' data)

Daemon processes

When we refer to a daemon process, we are referring to a process with these characteristics:

• Generally persistent (though it may spawn temporary helper processes like xinetd does)
• No controlling terminal (and the controlling tty process group tpgid) is shown as -1 in ps)
• Parent process is generally init (process 1)
• Generally has its own process group id and session id;
• Generally a daemon provides a service. So why not put such services in the kernel?
• Another level of modularity that is easy to control
• Let's keep from growing the already largish kernel
• Ease (and safety) of killing and restarting processes
• Logically, daemons generally share the characteristics one expects of ordinary user processes (except for the lack of controlling terminal.)

BSD-ish: Kernel and user daemons: swapper

• An increasing number of Unix/Linux daemons execute in kernel mode; pagedaemon and swapper are two early examples from the BSD world), but their numbers have been growing in recent years.

Kernel and user daemons: init

init (pid 1) daemon: The first ``user'' process started by the kernel; its userid is 0. All other ``normal'' processes are descendants of init. Depending on the boot configuration, you might see something along these lines:

• Spawn a single-user shell at the console.
• Begin multi-user start-up scripts
• Perhaps start up daemontools "svscanboot" or runit's "runsvdir"
• Or start systemd, initng, upstart, or !?!?

There is a lot of flux in this area. It's hard to characterize the successes and the failures of each approach since each has an environment that it is most appropriate for, and all have downsides in other environments. Articles like this and this demonstrate some of the vehemence that can be attached to these discussions.

Kernel and user daemons: update (aka bdflush/kupdate and fsflush)

• update daemons: An update daemon executes the sync() system call every 30 seconds or so. The sync() system call flushes the system buffer cache; it is desirable because UNIX uses delayed write when buffering file I/O to and from disk.
• The update daemon goes by many names (also see pdflush, bdflush(2), and bdflush(8) in Linux and fsflush in Solaris).

Comments in the code: what kernel comments say about dirty buffers and pages

/*
 * The relationship between dirty buffers and dirty pages:
 *
 * Whenever a page has any dirty buffers, the page's dirty bit is set, and
 * the page is tagged dirty in its radix tree.
 *
 * At all times, the dirtiness of the buffers represents the dirtiness of
 * subsections of the page.  If the page has buffers, the page dirty bit is
 * merely a hint about the true dirty state.
 *
 * When a page is set dirty in its entirety, all its buffers are marked dirty
 * (if the page has buffers).
 *
 * When a buffer is marked dirty, its page is dirtied, but the page's other
 * buffers are not.
 *
 * Also.  When blockdev buffers are explicitly read with bread(), they
 * individually become uptodate.  But their backing page remains not
 * uptodate - even if all of its buffers are uptodate.  A subsequent
 * block_read_full_page() against that page will discover all the uptodate
 * buffers, will set the page uptodate and will perform no I/O.
 */

(from fs/buffer.c in kernel 3.16)

Kernel and user daemons: inetd and xinetd

• Even though well-written daemons generally consume little CPU time, they do take up virtual memory and process table entries.
• Years ago, as people created new services, the idea of a super-daemon inetd was created to manage the class of network daemons.
• Many network servers were mediated by the inetd daemon at connect time, though some, such as sendmail, postfix, qmail, and sshd were not typically under inetd.
• The original inetd listened for requests for connections on behalf of the various network services and then started the appropriate daemon, handing off the network connection pointers to the daemon.
• Some examples are pserver, rlogin, telnet, ftp, talk, and finger.
• This mediation allowed ideas like tcpwrappers (a predecessor to general firewall solutions) to evolve.

Amusingly enough, this very same line of reasoning was revived by systemd; see this blog posting by its author. (note that daemontools also has used a related idea since 2001, but more for monitoring purposes.)

Kernel and user daemons: inetd and xinetd

When installing new software packages you may have to modify /etc/inetd.conf, /etc/xinetd.d/ files, and/or /etc/services. A hangup signal (kill -HUP SOMEPID) will get the inetd/xinetd to re-read its config file. Or you might be able to use a startup script, such as ``/etc/init.d/inetd restart'') or ``service inetd restart''.

Kernel and user daemons: portmap and rpcbind

portmap/rpcbind : portmap (rpcbind on OpenSolaris and BSD) maps Sun Remote Procedure Call (RPC) services to ports (/etc/rpc). Typically, /etc/rpc looks something like:

[root@vm5 etc]# more /etc/rpc
#ident  ``@(#)rpc        1.11    95/07/14 SMI''   /* SVr4.0 
#
#       rpc
#
portmapper      100000  portmap sunrpc rpcbind
rstatd          100001  rstat rup perfmeter rstat_svc
rusersd         100002  rusers
nfs             100003  nfsprog
ypserv          100004  ypprog
mountd          100005  mount showmount
ypbind          100007
walld           100008  rwall shutdown
yppasswdd       100009  yppasswd

Kernel and user daemons: portmap/rpcbind

• Sun RPC is used by other services, such as NFS and NIS. RPC servers register with this daemon and RPC clients get the port number for a service from the daemon. You can find operational information using rpcinfo. For example, rpcinfo -p will list the RPC services on the local machine.
• Some daemons may fail if portmap isn't running. Most UNIXes these days automatically start up portmap after installation, so it's usually not a problem. Also, there are subtle points that have oddly creeped in from the old tcpwrappers package that can affect the portmapper. See for example /etc/hosts.deny.

Kernel and user daemons: syslogd

syslogd : syslogd is a daemon whose function is to handle logging requests from

• the kernel
• other user processes, primarily daemon processes
• processes on other machines, since syslogd can listen for logging requests across a network

Note that syslog is generally being replace rsyslog.

Kernel and user daemons: syslogd

A process can make a logging request to the syslogd by using the function syslog(3). syslogd determines what to do with logging requests according to the configuration file /etc/syslog.conf

/etc/syslog.conf generally looks something like:

*.info;mail.none;news.none;authpriv.none;cron.none  /var/log/messages
authpriv.*                                          /var/log/secure
mail.*                                              /var/log/maillog
cron.*                                              /var/log/cron
*.emerg                                             *
uucp,news.crit                                      /var/log/spooler
local7.*                                            /var/log/boot.log

Kernel and user daemons: syslogd

• For a single UNIX machine, the default /etc/[r]syslog.conf will suffice. Also, you should note that Linux distributions have been moving to rsyslogd, which provides expanded capabilities (such as logging directly to a database) and still tries to preserve the capabilities of the original syslogd.
• You should read the file and figure out where the most common error messages end up (/var/adm/messages or /var/log/messages are typical default locations).
• If you are going to manage a number of UNIX machines, consider modifying /etc/[r]syslog.conf on the machines so all the syslog messages are routed to a single ``LOGHOST''.

Whew!

Thus ends our initial summary of daemons in the Unix world!