Introduction: Why study secure coding?

The operating system is the layer of software closest to the hardware, and therefore has the most privilege and the greatest potential for damage if it fails or is intentionally subverted by an adversary. If the operating system is vulnerable, the whole system is vulnerable. No amount of effort in other software layers can compensate. Moreover, even if the operating system itself is secure, the security can easily be compromised through user errors, ranging from weak passwords through mis-configuration and unsafe use of operating system services. Therefore trustworthiness, include both security against intentional attack and robustness against failures whatever the cause, must be a cross-cutting concern of not only every operating system developer but also application developers, system administrators, and users.

The C language (and other languages intended to be "upward-compatible" with it, or whose implementation is based on an underlying C-language runtime system) is inherently insecure, and can be said to encourage programmers to write insecure code. While some developers have chosen to use more secure languages for new development, there are many legacy systems written in C that need to be maintained, and there remains a very strong attachment to C among certain communities. The C language and its upward-compatible variants (including e.g., C++, Objective C) continue to dominate in operating systems and embedded systems. Even most Java implementations are based on runtime libraries written in C. Therefore, we are using the C language in this course.

Many of the lessons that can be learned from study of C-language vulnerabilities extend to programming in other languages, even if those languages don't have as many built-in pitfalls as C. Any programming language that is fully expressive in the sense of Turing Completness (i.e., allowing a programmer to achieve any desired effect by intent) is also expressive enough to allow a programmer to unintentionally wreak havoc. Moreover, the same programming errors that produce vulnerabilities that can be exploited by a malicious agent can also cause failures with serious consequences even in the absence of intentional attack. Ultimately, the only way to produce and maintain trustworthy software sysems is through careful, hazard-aware programming.

A full treatment of secure C coding is outside the scope of an operating systems survey course, but one objective of this operating systems course is that every student achieve a level of awareness of vulnerabilities, begin the process of learning securing coding, and be able to find resources to continue that learning throughout her/her life as a programmer.

Since the textbook for this course does not treat this subject adequately, I have made links to some supporting materials accessible by this link.