OS Security

Topics

• Security Requirements
• Kinds of Threats and Vulnerabilities
• OS Security Principles & Issues
• Protection Mechanisms
• Unix Security Mechanisms
• Windows 2000 Security
• Taxonomy of Malicious Software
• Trusted Systems

The study of OS security provides a context for review of the OS. Throughout this course we have pointed out security issues as they arose in the context of various other aspects of the OS. Security issues we have discussed before include:

• buffer overflows, stack crashing
• isolation of process address spaces
• vulnerabilities through dynamically linked and shared libraries
• vulnerabilities through the PATH environment variable effects on exec and shells
• page and segment level memory protection (rwx), in a VM system
• file permissions
• set-user-id and set-group-id programs (good and bad aspects)
• trust in mounting filesystems

Computer and Network Security Requirements

• Confidentiality (includes secrecy)
  Requires information in a computer system only be accessible for reading by authorized parties
  Threat: unauthorized release of information
• Integrity
  Assets can be modified by authorized parties only
  Threat: unauthorized modification of information
• Availability
  Assets be available to authorized parties
  Threat: unauthorized denial of service
• Authenticity
  Requires that a computer system be able to verify the identity of a user
  Threat: unauthorized release of information, or modification of information

The key factor here is authorization. We cannot lock up system in a sealed vault. Authorized users must still be able to use it.

Aspects of Security

• external = access to computer system
• internal = allowed actions within the computer system
  • policies - how we decide what to do
  • mechanisms - means by which it can be done
    including protection mechanisms
• authentication = verification of user's identity

Types of Threats

• Interruption
  • An asset of the system is destroyed or becomes unavailable or unusable
  • Attack on availability
  • Destruction of hardware
  • Disabling the file management system
  • Cutting of a communication line
  
• Interception
  • An unauthorized party gains access to an asset
  • Attack on confidentiality
  • Illicit copying of files or programs
  • Wiretapping to capture data in a network
  
• Modification
  • An unauthorized party not only gains access but tampers with an asset
  • Attack on integrity
  • Changing values in a data file
  • Altering a program so that it performs differently
  • Modifying the content of messages being transmitted in a network
  
• Fabrication (a special case of modification)
  • An unauthorized party inserts counterfeit objects into the system
  • Attack on authenticity
  • Addition of records to a file
  • Insertion of spurious messages in a network
  

Threats to Computer System Assets

• To Hardware
  • Threats: loss of availability, through accidental and deliberate damage, theft
  • Measures: physical security, administrative security, backup power, redundant hardeware
• To Software
  • Threats: loss of availability, through deletion, alteration, damage; loss of integrity; theft
  • Modifications to software (e.g., by viruses) may not be visible
  • Measures: detection of modifications using checksums, backups
• Data
  • Involves files
  • Threats: loss of availability, secrecy, and integrity
  • Not always obvious: statistical analysis can lead to determination of individual information which threatens privacy
• Communication Lines and Networks
  • Threats: release, loss, modification, falsification of messages; unauthorized remote access

Communication Lines and Networks: Active Attacks

• Traffic analysis can reveal information even if the message contents are encrypted
• Masquerade takes place when one entity pretends to be a different entity
• Replay involves the passive capture of a data unit and its subsequent retransmission to produce an unauthorized effect
• Modification of messages means that some portion of a legitimate message is altered, or that messages are delayed or reordered, to produce an unauthorized effect
• Denial of service prevents or inhibits the normal use or management of communications facilities, e.g., by disabling the network or overloading it with messages

The data communicated might be a telephone conversion, an electronic mail message, or a transferred file. We want to prevent the release and prevent the undetected deletion or modification of the content.

OS Security Fundamentals

• Security is based on trust
  • "trustworthiness = reliability + honesty + benificence"
  • How much do you trust the OS?
  • How much do you trust users?
  • How much do you trust application software vendors & developers?
• Mechanisms depend on lower layers.
  • File protection depends on OS
• Policies depend on human compliance.
  • Lock on 103 Love classroom depends on humans to close/lock door
  • Passwords depend on people to not share them
• Security is at odds with convenience
  • Humans will neglect or undermine security policies, if they are onerous.

When one studies security, it is easy to lose perspective and adopt a "Maginot Line" attitude. One can pay a lot of attention to the design of security mechanisms and policies, but ignore other weaknesses. This is like devoting all one's attention to designing better locks, but ignoring other problems. People who buy expensive locks but forget to lock their doors, or live in houses that have easily broken windows, don't have any better security for the expensive locks. Similarly, one can pay attention to designing better alarm systems, but ignore other weaknesses, like people who turn off the alarm system, or ignore the alarms (maybe because there are too many false alarms). One can design an impregnable safe, lock up all one's valuables in it, and then forget the combination. One can buy security tools, but hire a system administrator who is untrustworthy. etc.

Security Issues for Operating Systems: What can the OS do to help?

• Provide security features, e.g., for protection of user data
  • File protection mechanisms
  • Encryption/decryption mechanisms
• Be trustworthy
  • other security depends on the OS to be trustworthy
  • if OS can be taken over, all is lost

Features are easier to provide than trustworthiness, but are useless if the system is not trustworthy.

Trustworthiness starts with reliability.

Buffer Overflow: Examples of Security Exploits

Please see the separate notes on buffer overflow exploits.

One moral of this story is that programming languages used for security-critical software should always prevent or detect array overflows. For example, the Ada language has strict runtime tests for array overflows and undefined/null pointers, which would prevent a buffer overflow from going undetected.

The deeper moral is that system security depends on software quality assurance. Very mundane, but hard to find, software defects can be exploited to break through security.

The Fundamental Problem with Security: Two Kinds of Properties

• There exists an x such that P(x)
  e.g., "There is a leak in the roof."
  e.g., "There is an unchecked buffer overflow in this program, right here."
• For all x, P(x),
  or There does not exist an x such that P(x)
  e.g., "There is no leak in the roof."
  e.g., "There is no defect in system software that allows unauthorized transfer privileges."
  

Which can be guaranteed by adding code to a system?

Which can be verified by just looking at part of the code in a system?

Which form would a security assertion take?

Which form would an insecurity assertion take?

The Problem with Making Real Operating Systems Secure

• Writing defect-free software is hard
• OS's are very large and involve many authors, compounding the problem
• Partitioning and limiting scope of trust, can help
• Most widely used existing OS's were written without security in mind
  • Unix - small trusted group of researchers, locked in lab, no network
  • DOS - individual user, on desk, no network
• Security became an important issue when networking was added
• Now, we are faced with making an existing system secure, or building a new secure operating system

Why Making Big Improvements in OS Technology is Hard, in General

• Retrofitting existing OS' is hard (Why?)
• Writing a new OS from scratch costs a very large amoung of money (Why?)
• Persuading people to use a new OS is difficult (Why?)
• New OS technology is adopted if it can be added onto an existing OS
• Otherwise, it is not adopted
• What about security?

Protection versus Security

• protection is offered by mechanisms
  e.g., putting locks on doors
• security depends on policies,
  which specify how protection mechanisms are to be used
  e.g., who gets keys? when are doors locked?
• protection domain of a process
  • resources process can access
  • operations it can perform on those resources
  • good policy is to restrict domain to what is needed

Design Principles for Protection Mechanisms

• economy - overhead is tolerable
• complete mediation - all requests are checked
• open design - does not depend on attacker's ignorance
• separation of privileges
• least privilege
• least common mechanism
• acceptability
• fail-safe defaults

How do any of the above relate to the phrase ``security through obscurity''?

Degrees of Protection

• No protection
  OK if sensitive procedures are run at separate times
• Isolation
  Each process operates separately from other processes with no sharing or communication
• Share all or share nothing
  Owner of an object declares it public or private
• Share via access limitation
  • Operating system checks the permissibility of each access by a specific user to a specific object
  • Operating system acts as the guard
• Share via dynamic capabilities
  Dynamic creation of sharing rights for objects
• Limit use of an object
  • Limit not only access to an object but also the use to which that object may be put
  • Example: a user may be able to derive statistical summaries but not to determine specific data values

Protection of Memory

• Separation of processes provided by bounds registers
• Separation with controlled sharing provided by paged and segmented virtual memory systems

User-Oriented Access Control

• Control of who can access the system: authentication and identification of authorized users
• System Log on
• Requires both a user identifier (ID) and a password
• System only allows users to log on if the ID is known to the system and password associated with the ID is correct
• Threat: users can reveal their password to others either intentionally or accidentally
• Threat: password may be intercepted
• Threat: guessing passwords
• Threat: ID/password file may be obtained, and cracked
• Authentication by tokens
  • physical (smart) card
  • private key, used in authentication protocol
• Authentication by trusted intermediary system

Data-Oriented Access Control

• Control of what users are authorized to do
• Associated with each user, there can be a user profile that specifies permissible operations and file accesses
• Operating system enforces these rules
• Database management system may further control access to specific records or portions of records

Access Matrix

• Subject
  An entity capable of accessing objects
• Object
  Anything to which access is controlled
• Access rights
  The way in which an object is accessed by a subject

This is useful as an academic abstraction, but nobody actually implements it. Why?

Access Control List

• Matrix decomposed by columns

  Note: The use of the term "matrix" does not mean this is implemented as an array.

• list contains pairs (subject, Permissions(subject,object))
• For each object, an access control list gives users and their permitted access rights
• when subject s wants to access a to object o
  • search ACL of o for entry corresponding to s
  • check whether this entry contains a

ACL Example

ACL Pros & Cons

• slow access time
• difficult review of accessibility, by subject
• easy revocation
• easy review of acessibility, by object
• amenable to protection groups, which saves storage
• right to change protections easily controlled

Shell & API Example: Access Control Lists in SunOS

#include <sys/acl.h>
int  acl(char *pathp, int cmd, int nentries, aclent_t *aclbufp);
int  facl(int fildes, int cmd, int nentries, aclent_t *aclbufp);

The SunOS man-pages say:

The acl() and facl() functions get or set the ACL of a file whose name is given by pathp or referenced by the open file descriptor fildes. The nentries argument specifies how many ACL entries fit into buffer aclbufp. The acl() function is used to manipulate ACL on file system objects.

The values for the parameter cmd are:

• SETACL - copies nentries ACL entries from aclbufp to the file's ACL.
• GETACL - copies the file's ACL entries to the buffer aclbufp.
• GETACLCNT - returns the number of entries in the files ACL

Shell commands getfacl, setfacl, aclcheck, and aclsort are also provided. See the Solaris man pages for more detail, if you are interested.

Capability Tickets versus Capability Lists

• Capability list is conceptually similar to a row of an access matrix
  It specifies the authorized object(s) and operations for a user
• Capability, or capability ticket contains information similar to a capability list, but also has the characteristic of conferring on the user a given set of capabilities, without further access control checking.

The textbook is not very clear on this subject. There is a big distinction between a "capability list" (conceptually, a row of an access matrix), and a "capability" (also known as a capability ticket or capablity token).

UNIX Open File Descriptions Viewed as Capability Tickets

How and why?

By now you should know enough about Unix to be able to recognize the ways in which an open file description behaves like a capability token.

1. Can open file descriptions be propagated? If so how? Can this be prevented? If so, How?
2. Can open file descriptions be reviewed? If so, how?
3. Can open file descriptions be revoked? If so, how?
4. Are open file descriptions garbage-collected? If so, how?

Summary of Unix (Solaris) Protection Mechanisms

• passwords
• file access modes (a special form of access control list)
  • read, write, execute
  • owner, group, others
  • set user ID on execution
  • set group ID on execution
  • turn on sticky bit
  • enable "mandatory file/record locking" (Solaris)
• general access control lists (setfacl, getfacl)
• memory protection
• shared memory permissions

We have covered these protection mechanisms in the course.

Windows 2000 Security

• Access Control Scheme
  • Name/password
  • Access token associated with each process object indicating privileges associated with a user

Access Token

• Security ID
  • Identifies a user uniquely across all the machines on the network (logon name)
• Group SIDs
  • List of the groups to which this user belongs
• Privileges
  • List of security-sensitive system services that this user may call
• Default owner
  • If this process crates another object, this field specifies who is the owner
• Default ACL
  • Initial list of protections applied to the objects that the user creates

Security Descriptor

• Flags
  • Defines type and contents of a security descriptor
• Owner
  • Owner of the object can generally perform any action on the security descriptor
• System Access Control List (SACL)
  • Specifies what kinds of operations on the object should generate audit messages
• Discretionary Access Control List (DACL)
  • Determines which users and groups can access this object for which operations

Access Mask

Intruder Types

• Malfeasor - known authorized user, but abuses trust
• Masquerader - pretends to be a known authorized user, e.g., by stealing password
• Clandestine user - invisible to system

Intrusion Techniques

• Objective of intruder is the gain access to the system or to increase the range of privileges accessible on a system
• Protected information that an intruder acquires is a password

Passwords: Techniques for Learning a Password

• Try default password used with standard accounts shipped with computer
• Exhaustively try all short passwords
• Try words in dictionary or a list of likely passwords
• Collect information about users and use these items as passwords
• Try user's phone numbers, social security numbers, and room numbers
• Try all legitimate license plate numbers for this state
• Use a "Trojan horse" to bypass restrictions on access
• Tap the line between a remote user and the host system

ID Provides Security

• Needed before password, first step of security
• Determines whether the user is authorized to gain access to a system
• After password check, serves as identification
• Determines the privileges accorded to the user
  Guest or anonymous accounts have mover limited privileges than others
• ID is used for discretionary access control
  A user may grant permission to files to others by ID

Password Selection Strategies

• Computer generated passwords
• Users have difficulty remembering them
• Need to write it down
• Have history of poor acceptance
• Reactive password checking strategy
• System periodically runs its own password cracker to find guessable passwords
• System cancels passwords that are guessed and notifies user
• Consumes resources to do this
• Hacker can use this on their own machine with a copy of the password file
• Proactive password checker
• The system checks at the time of selection if the password is allowable
• With guidance from the system users can select memorable passwords that are difficult to guess

Intrusion Detection

• Assume the behavior of the intruder differs from the legitimate user
• Statistical anomaly detection
• Collect data related to the behavior of legitimate users over a period of time
• Statistical tests are used to determine if the behavior is not legitimate behavior
• Rule-based detection
• Rules are developed to detect deviation from previous usage pattern
• Expert system searches for suspicious behavior
• Audit record
• Native audit records
  All operating systems include accounting software that collects information on user activity
• Detection-specific audit records
  Collection facility can be implemented that generates audit records containing only that information required by the intrusion detection system

Profiles of Behavior of Intruders and Authorized Users

Classification of Malicious Programs

• Those that need a host program
  • Fragments of programs that cannot exist independently of some application program, utility, or system program
• Independent
  • Self-contained programs that can be scheduled and run by the operating system

Taxonomy of Malicious Software

Trapdoor

• Entry point into a program that allows someone who is aware of trapdoor to gain access
• used by programmers to debug and test programs
  • Avoids necessary setup and authentication
  • Method to activate program if something wrong with authentication procedure

Logic Bomb

• Code embedded in a legitimate program that is set to "explode" when certain conditions are met
  • Presence or absence of certain files
  • Particular day of the week
  • Particular user running application

Trojan Horse

• Useful program that contains hidden code that when invoked performs some unwanted or harmful function
• Can be used to accomplish functions indirectly that an unauthorized user could not accomplish directly
• User may set file permission so everyone has

Worms

• Use network connections to spread form system to system
• Electronic mail facility
  • A worm mails a copy of itself to other systems
• Remote execution capability
  • A worm executes a copy of itself on another system
• Remote log-in capability
  • A worm logs on to a remote system as a user and then uses commands to copy itself from one system to the other

Zombie Program

• Program that secretly takes over another Internet-attached computer
• It uses that computer to launch attacks that are difficult to trace to the zombie's creator

This is not the same thing as a Unix zombie process.

Viruses

• Program that can "infect" other programs by modifying them
  • Modification includes copy of virus program
  • The infected program can infect other programs

Virus Stages

• Dormant phase
  • Virus is idle
• Propagation phase
  • Virus places an identical copy of itself into other programs or into certain system areas on the disk
• Triggering phase
  • Virus is activated to perform the function for which it was intended
  • Caused by a variety of system events
• Execution phase
  • Function is performed

Types of Viruses

• Parasitic
• Attaches itself to executable files and replicates
• When the infected program is executed, it looks for other executables to infect
• Memory-resident
• Lodges in main memory as part of a resident system program
• Once in memory, it infects every program that executes
• Boot sector
• Infects boot record
• Spreads when system is booted from the disk containing the virus
• Stealth
• Designed to hide itself from detection by antivirus software
• May use compression
• Polymorphic
• Mutates with every infection, making detection by the "signature" of the virus impossible
• Mutation engine creates a random encryption key to encrypt the remainder of the virus
  The key is stored with the virus

Macro Viruses

• Platform independent
  frequently target Microsoft Word
• Infect document, not executable portions of code
• Easily spread
• A macro is an executable program embedded in a word processing document or other type of file
• Autoexecuting macros in Word:
  • Autoexecute
    Executes when Word is started
  • Automacro
    Executes when defined event occurs such as opening or closing a document
  • Command macro
    Executed when user invokes a command (e.g., File Save)

E-mail Virus

• Activated when recipient opens the e-mail attachment
• Activated by open an e-mail that contains the virus
• Uses Visual Basic scripting language
• Propagates itself to all of the e-mail addresses known to the infected host

Antivirus Approaches

• Detection
• Identification
• Removal

Generic Decryption

• CPU emulator
  Instructions in an executable file are interpreted by the emulator rather than the processor
• Virus signature scanner
  Scan target code looking for known
• Emulation control module
  Controls the execution of the target code

Trusted Systems

• Multilevel security
  • Information organized into categories
  • No read up
    • Only read objects of a less or equal security level
  • No write down
    • Only write objects of greater or equal security level

Reference Monitor Concepts

The reference monitor controls every access by subjects to objects. It enforces policies that are recorded in the security kernel database, and maintains an audit file, which records attempts at security violations, and may also be useful in analysis of security failures after the fact.

Trojan Horse Attack

User Bob has a private file, that only he should be able to access. User Alice creates Trojan horse program, which can access a "back pocket" file of Alice's. Alice persuades Bob to use her program.

Trojan Horse Attack

When Bob run's the Trojan horse, the program surreptitiously reads the content of Bob's file and copies it to Alices "back pocket" file.

Trojan Horse Defense

Now suppose Bob has a higher security level than Alice, and they are using a trusted sytem, with a reference monitor.

Trojan Horse Defense

When Alice's Trojan horse, executed by Bob, tries to write to Alice's "back pocket" file, it is detected and prevented, because it would be a "write down" in security level. (Note that this also would prevent Bob from intentionally passing information to Alice.)