Unix/Linux, learning what is where

• Simple unifying concepts back most of the Unix/Linux world.
• Files as bytestreams
• Processes are simple reifications that have well-defined state.
• Linux Layers

What kind of state is available?

• Like Windows, (useful) state and caching of state generally are found in (1) memory and (2) file structured areas. In Unix/Linux, such file-structured areas can include raw disk drives, aggregations of disk space such as RAID (and RAID can either be hardware or software, or even a mix!), CD/DVD/Blue Ray, USB flash drives, and state is held remotely, such as by NFS or sshfs sessions.

Live system tool derived state

• The first category of state that we will look at is that of "Live system tool derived state".
• How can we carry out these queries? Here we start running into forensics issues: just as with Windows, we should be leery of trusting the system to answer these questions. Whatever programs we use to extract such state from the system themselves should be considered carefully. For live forensics, popular approaches include running programs from some sort of removable media, such as a CD or a USB memory key.
• If we are on a network, tools like "netcat" allow us to do simple network transfers with very little overhead.
• We might even be able to use a live mount such as a sshfs mount to do such work, but it's not guaranteed that this can be made available.

"Shopping List" for volatile information

• What kind of information are we going to look at? Let's put at least together a Unix/Linux "shopping list".

Date/time information

• Date/time information. We could use something like

  	    uptime
  	  

  Or to get network time:

              ntpdate -q pool.ntp.org
  	  

  [Hat-tip to ServerFault for the latter idea. Note, however, that we have just added networking state when we make network queries; it would probably be better to hold off on this step until we have profiled the networking state...]

Memory acquisition

• As with Windows, it is a good idea to copy memory very early on — MF recommends on page 9 that you copy it immediately after you do a time check. Fortunately, with Unix/Linux, this is almost always easy to do: you can use the dd program to copy memory. MF also recommends memdump on page 100. Also, as MF notes on the same page, grabbing a copy of the /proc/meminfo to get a layout of memory is a good idea.
• Also, there are many programs that can show you what processes are in the system, such as ps, but the reality is that everything for all processes is now contained in the dynamic filesystem /proc

System identification

In the Unix/Linux environment, you can use hostname, whoami, uname -a, and possibly cat /etc/SOMEFILE (SOMEFILE depending on the operating system — for instance, in Redhat, it would be redhat-release. Another candidate is the previously mentioned uptime. MF suggests both tools from CTC and from Helix. You can also write a simple "profiler" in Python or Perl to collect volatile information from the /proc and /sys pseudo-files. Also, you might want to just copy everything from the /etc/ and /usr/local directories — these are considered to embody the "personality" of a Linux/Unix system. You might also consider copy /root since it is not uncommon to find important configuration and setup information in that directory.

Who is using the system? Login sessions.

• w,who are two of the classics (but note that some of these classic programs do not show full usernames if the username is longer than 8 characters). You can copy off such information from files such as /var/log/
• You can also try to answer the question "who was using the system" with programs such as last.
• Of course, like Windows, these are only going to show "proper" user sessions; if "backdoors" have been set up or logging has been compromised, then such use won't show up in these tools.

Who is using the system? Open files.

• In the Unix/Linux world, we have the excellent lsof program to show exactly what files each process is using.

What else might be scheduled?

• The Unix/Linux world has used cron as its scheduler of preference for a long time (generally it's "Vixie" cron these days, which allows quite a few configuration parameters to be buried in a crontab. Look for a crond in the process table. Generally, cron files are stored in both /etc/ and in /var/spool/cron. Also a program called anacron has become quite popular; it takes care of the problem of possibly missing a scheduled activity due to downtime by using a "fuzzy" (or perhaps more accurately characterized as "periodic") schedule rather than an exact one.

NFS and sshfs

• Remote state can be mounted from other servers using NFS and sshfs. While you can widely access NFS state when doing an investigation as "root", unless you have a password or private ssh key, you won't be able to easily access remote sshfs information. (This is due to the design of sshfs.)

Network connectivity

• Like Windows, one of the most useful tools is netstat -ano which may let you see which PID created a connection.
• Don't forget grab a copy of the ARP table; like Windows, you can use arp -a to do this. Remember, the ARP table provides the mappings from IP numbers to MAC addresses, and malefactors.
• Also, try a portscan with nmap. As with Windows, this can help avoid the problems of using internal structures that might be manipulated or obscured by malware by looking at an "outward" view.

Keyboard history

• In the Unix/Linux world, it is likely that a malefactor will use a command shell. While there are a large number of shells out there (csh, zsh, sh, tcsh...), bash has become prevalent. You can look for files such as .bash_history to find a history of bash commands.

SUMMARY

• The final goal of any forensics work is to understand state.
• State can be found in many places: disk drives, memory, device registers, and a host of other places.
• Understanding state can be contextual: for instance, state associated with networking, state associated with local keyboard and mouse activities, state associated with authentication systems, ...