Unix File Concepts

File Descriptors, Open File Descriptions, Streams

What are the differences between the following, and how do they relate to one another?

• a file name
• a path name
• a directorory
• a file
• an open file description (a Unix term)
• a file descriptor (a Unix term)
• a stream
• a buffer

Diagram of File Descriptors, Open File Descriptions, Streams

This diagram shows the relationships of file descriptors, streams, buffers, open file descriptions, and files.

Pay special attention to the many-one relationships.

Stream

• is an abstraction for a stream (sequence) of bytes
• contains a buffer, and a reference to a file descriptor
• is local to a specific process
• can be created by fdopen() from an open file description
• fopen() combines the effects of open() and fdopen()
• closed by fclose()
• stdin, stdout, and stderr are streams
• datatype FILE
• support operations fprintf(), fscanf(), fflush()

Streams provide buffered I/O support. They are built on top of the lower-level abstractions of open file descriptor and open file description.

The fdopen operation takes a file descriptor and access mode information, creates a stream, and returns a pointer to the stream object. The fclose operation deallocates the stream object.

Stream Example

abstracted from simple_fork.c

FILE *outstream;
...
if (!(outstream = fdopen (outfildes, "w"))) {
  perror ("could not open stream for writing");  exit (-1);
}
  ...
fprintf (outstream, "some text\n");

Read the entire example program.

File Descriptor

• is an integer that refers to an open file description
• the mapping of decriptor to description is local to a specific process
• the open file description is a kernel data structure, not directly accessible by the process
• 0 is the file descriptor for stdin
• 1 is the file descriptor for stdout
• 2 is the file descriptor for stderr
• other file descriptors are assigned as open() is called
• dup() and dup2() can be used to manipulate file descriptors

File Descriptor Example

abstracted from hw1.c

int fd;
fd = open ("my_file", O_RDONLY, 0);
if (fd == -1) {
  perror ("open failed\n"); exit (-1);
}       
close (0);
if (dup2 (fd, 0) < 0) {
  perror ("dup2 failed"); exit(-1);
}
close (fd);

The dup2() call acts like an assignment statement for file descriptors. It make file descriptor 0 (stdin) refer to the same open file description as fd. We can then close fd, so that only 0 refers to the open file.

Read the entire example program.

Open File Description

• refers to a file
• is created by an open() operation
• a system kernel data structure
• not directly accessible by any process
• may be referred to (shared) by file descriptors of several processes
• contains a reference to a file, a location in the file, and access mode information (e.g., read or write)
• the open operation takes a pathname and a set of access modes, creates an open file description, and returns a file descriptor
• the close operation decrements the reference count of the open file description, and possibly deallocates it

The above are just summaries. See the Unix man-pages for complete descriptions of the effects of open, close, fdopen, and fclose.

The reference from the open file description to the actual file data is also indirect, through operating system data structures we will study later.

Directories, Filenames, Pathnames, Files

The diagram shows the relationships of directories, filenames, pathnames, and files.

Again pay attention to which relationships are one-one and which are many-one.

Regular Unix File

• contains data
• may be referred to by links from 0, 1, or more directories
• has permission mode information (e..g, read, write, execute)
• has no intrinsic name
• can be created via the open operation, with open mode O_CREAT
• unlink decrements reference count of the file, and possibly deallocates it

Unix Directory

• is a file
• contains a mapping from filename to file
• mapping may be a direct link to the file, or a symbolic link
• creating a direct link increments the reference count of the file
• symbolic link does not guarantee file exists
• is created via mkdir
• is removed via rmdir

Unix files do not have names. Only links (in directories) have names.

Shell command ln makes direct and indirect links. The form "ln -s" is used to create symbolic links.

Shell command rm removes links.

The corresponcding system calls (C API) are link() and unlink().

See the Unix man pages for more detail on each of these system calls.

Pipe

• an anonymous file-like object
• acts like a stream that can go between two processes
• created by call to pipe()

  int pipe(int filedes[2]);
  

• has two file descriptors, for each end
  • filedes[0] for reading
  • filedes[1] for writing
• can be named, and then is called a "fifo"
• signal SIGPIPE is generated if pipe is "broken" (we will discuss signals more later)

Read the entire example program.

Pipe Example

abstracted from hmwk1.c

int pipefd[2];
...
if (pipe (pipefd) == -1) {perror ("could not create pipe"); exit (-1);}
child = fork();
if (child != 0) {
  close (pipefd[0]);  /* the reading end of the pipe */
  if (write (pipefd[1], &outbuf, sizeof(outbuf)) == -1)
    perror ("write to pipe failed"); exit -1;
  }
  ...
  close (pipefd[1]);
} else {
  close (pipefd[1]);
  if ((nread = read (pipefd[0], &buffer, count)) == -1) {
    perror ("read failed"); exit (-1);
  }
  ...
  close (pipefd[0]);
}

The parent process creates the pipe, and then forks. The parent closes one end of the pipe. The child closes the other. The parent writes to the pipe, and the child reads from it.

This example uses of the unbuffered I/O operations, read() and write(), just for variety. A stream could be associated with either or both of the file descriptors, allowing use of the stream input and output functions. In program hmk1.c, the child process used dup2() to map the input side of the pipe to stdin, before executing a new program.

Read the entire example program.

Unix Special Files

• describe and refer to hardware devices
• block special file
  • is like disk drive
  • data is randomly accessible, by block
• character special file
  • is like a character stream
  • data is only accessible sequentially, on character at a time