Incident Response Plan

Rugged TUFF Computer Incident Response & Management Plan
Responding to computer security incidents, generally, is not a simple matter. Incident management and
response activities require technical knowledge, communication, and coordination among personnel who
respond to the incident.

Although incident management may vary in approach, depending on the situation, the goals are constant.

Accordingly, the goals of this plan are:

  • Helping affected entities recover quickly and efficiently from security incidents.
  • Minimizing the impact due to the loss or theft of information or disruption of critical computing services when incidents occur.
  • Responding, systematically, following proven procedures, which will dramatically decrease the likelihood of reoccurrence.
  • Balancing the operational and security requirements within realistic budgetary constraints.

1. Introduction
1.1 Identification of Document
This document is the computer incident response and management plan for the Computing and
Telecommunications Services (CaTS) Department of Rugged TUFF.

1.2 Purpose
The purpose of this document is to detail a process to guide CaTS staff in responding to a computer
incident in a consistent manner.

1.3 Audience
The primary audience for this plan is CaTS staff.

2. Background
Responding to computer security incidents, generally, is not a simple matter. Incident management and response activities require technical knowledge, communication, and coordination among personnel who respond to the incident. Although incident management may vary in approach, depending on the situation, the goals are constant.

Accordingly, the goals of this plan are:

  • Helping affected entities recover quickly and efficiently from security incidents.
  • Minimizing the impact due to the loss or theft of information or disruption of critical computing services when incidents occur.
  • Responding, systematically, following proven procedures, that will dramatically decrease the likelihood of reoccurrence.
  • Balancing the operational and security requirements within realistic budgetary constraints.

2.1 Definitions
The term “incident” refers to an adverse event in an information system and/or network or the threat of the occurrence of such an event. Examples of security incidents include:
  • Unauthorized use of the system as a gateway to other systems.
  • Unauthorized use of another user’s account.
  • Unauthorized use of a system.
  • Execution of malicious code that destroys data.

Other adverse events include floods, fires, electrical outages, and excessive heat that cause system crashes. Adverse events such as natural disasters and power-related disruptions though certainly undesirable incidents are not generally within the scope of this incident response plan and should be addressed in Rugged TUFF’s continuity (contingency) plan. For the purpose of Incident Response, therefore, the term “incident” refers to an adverse event that is related to Information Security.

2.2 Defining High Risk Incidence and Personal Information
High Risk – An incident is high risk if it involves one or more of the following:

  • Criminal activity.
  • Unauthorized external access to, or potential access to, personal information
  • Lost or compromised device containing, or possibly containing, confidential, sensitive, critical data
  • Potential unauthorized access due to discovery of a keystroke logger, rootkit, remote access agent, password cracking agent, or similar exploit
  • Disrupts continuity of critical business processes or communication.

Low Risk – Any incident not meeting the criteria for a high risk incident.

Personal Information – A person’s first name (or initial) and surname, in combination with any of the following:

  • Social Security Number.
  • Student or employee records
  • Driver’s license number or state identification card number.
  • Financial account, debit, or credit number.
  • Other information that creates a material risk of the commission of the offense of identity fraud or other fraud to the individual.

3. Requisite Infrastructure
Planning is paramount to successful incident response and mitigation. In order for a system to be restored to its original state, that state must be known and recorded. Likewise, in order for system logs to be reviewed, logging functions must be enabled. Finally, in order for file comparisons to be made to detect if file alteration has occurred, the most recent copy of all system executables, or binaries, must be
preserved. 

In furtherance of security planning infrastructure CaTS has instituted the following infrastructure to both
prevent an incident and in the event of an incident to facilitate successful incident mitigation:

  • Intrusion detection has been implemented on critical systems.
  • System logging functions are enabled and the actual logs should are placed well inside of the secure perimeter of the system.
  • A password management program has been established that forces periodic change of passwords as well as enforcing rules to make selected passwords are less susceptible to attempts to guess or crack encrypted passwords.
  • Unused recording media is available to make system backups and copies of files that have been altered. These copies are used for evidentiary purposes.
  • System binaries and executables and copies of all configuration files for network devices have been archived.
  • Login banners have been posted identifying ownership of the system whenever a user logs onto the Wright State University system.

In addition to the above, the following are recommended:

  • Assign roles and responsibilities to IT personnel before an incident occurs to insure that they clearly understand what is required of them should an incident happen.
  • Determine, in advance, who will be notified that an incident has occurred, and who is responsible for the overall coordination of the mitigation effort (usually the role of the IT security officer).
  • Establish a mechanism to provide update or progress information that will not interfere with the personnel tasked to deal with the intrusion.
  • Implement a trouble ticketing system to record, track and quantify incidents. The trouble ticketing system should be capable of generating a file or case number, be accessible to all that are assigned to resolve incidents, and be capable of receiving and filing of email messages

4. Incident Handling Procedures
The incident handling process consists of six steps:


  • Initial Response – Identify whether or not an incident has occurred or is occurring. This process begins after someone notices some anomaly in the system or network.
  • Intrusion Analysis – Determine the extent of the incident and contain it to prevent it from getting worse.
  • System Repair – Make sure that the problem is eliminated.
  • Security Improvement – Identify and eliminate the means by which the system was compromised.
  • Network Reconnect – Restore the system to an operational status.
  • Security Policy Update – Based upon any lessons learned from analysis of the incident update the security policy if needed.

The following further details each of the six aforementioned steps. The recommended actions are
predicated upon a worse case scenario – a root compromise. Incidents of lesser severity require a more
limited response.

4.1 Initial Response
Compromises must be resolved as soon as possible, preferably the day of the notification. Compromised hosts must be reformatted, rebuilt and have vulnerabilities resolved before reconnecting them to the network. CaTS may decide, after review, that compromised hosts may be cleaned and patched expeditiously. Incidents must be resolved to the satisfaction of Cats before compromised hosts are reconnected to the network or filters are lifted. In some cases, CaTS may request privileged access to ensure the host is safe to resume network connectivity, or may require that it be evaluated for vulnerabilities before being placed back in service.

4.1.1 Initial Determination.
Determine whether the event is actually indicative of an incident and if so the type and severity of the incident.

4.1.2 Required Response.
Based upon the severity of the incident – attempt to determine the appropriate level of response required to mitigate the incident. Determine if the objective is solely to restore the system and prevent a re-occurrence, or will there be an effort to identify the perpetrator and file either a civil or a criminal action? If the intention is to pursue legal action then anything on the system may potentially be evidence and is subject to special requirements. If the objective is unknown then it must be assumed that a legal action could take place, therefore you must treat all files on the system as if they are evidence. If a loss of confidential information has occurred or is suspected contact the CaTS computer incident response team immediately.

4.1.3 Notification
Open a ticket in the incident reporting system and notify the designated party(s) responsible for initially responding to security incidents.

4.1.4 Identify contiguous systems and share information. Keep key people in the loop, particularly administrators of adjacent or contiguous systems.

4.1.5 Isolate the affected system or systems from the rest of the network. Regain control by disconnecting all compromised machines from the network. If the compromised machine is not disconnected from the network, there is the risk that the intruder may be connected to your machine and may be undoing your steps as you try to recover the machine.

4.1.6 Back-Up
Back up the affected system(s) to new unused media. Before analyzing the intrusion, create a backup of your system. This will provide a “snapshot” of the file system at the time that the root compromise was first discovered. Creating a low-level backup is important in case you ever need to restore the state of the compromised machine when it was first discovered. Label, sign and date the backup and keep the backups in a secure location to maintain integrity of the data.

4.1.7 Avoid the Obvious
Avoid looking for the intruder with obvious methods like finger, ping or traceroute.

4.1.8 Identify all items that could possibly be considered as evidence. Identify all times that have evidentiary value. Include computer printouts and logs that may have evidentiary value or be helpful in identifying the intruder.

4.2 Intrusion Analysis

With your system disconnected from the network, you can now thoroughly review log files and configuration files for signs of intrusion, intruder modifications, and configuration weaknesses.
http://sans.org/resources/winsacheatsheet.pdf
http://www.sans.org/score/checklists/ID_Windows.pdf
http://www.sans.org/score/checklists/ID_Linux.pdf

4.2.1 Look for modifications made to system software and configuration files

4.2.2 Look for modifications to data

4.2.3 Look for tools and data left behind by the intruder
Intruders will commonly install custom-made tools for continued monitoring or access to a root compromised system.

The common classes of files left behind by intruders are as follows:

  • Network Sniffers

A network sniffer is a utility which will monitor and log network activity to a file. Intruders commonly use network sniffers to capture username and password data that is passed in clear-text over the network.

  • Trojan Horse Programs

Trojan horse programs are programs that appear to function properly, but either add or remove features. Intruders use Trojan horse programs to hide their activity, capture username and password data, and create backdoors for future access to a root compromised system.

  • Vulnerability Exploits

The majorities of root compromises are a result of machines running vulnerable versions of software. Intruders often use tools to exploit known vulnerabilities and gain root access. These tools are often left behind on the system in “hidden” directories.

  • Intruder Tool Output

You may find log files from any number of intruder tools. These log files may contain information about other sites involved, vulnerabilities of your compromised machine(s), and vulnerabilities at other sites. Search thoroughly for such tools and output files. Be sure to use a known clean copy of any tool that you use to search for intruder tools.

4.2.4 Review log files

Reviewing your log files will help you get a better idea of how your machine was compromised, what happened during the compromise, and what remote hosts accessed your machine. Keep in mind when reviewing any log files from a root compromised machine that any of the logs could have been modified by the intruder.

4.2.5 Look for signs of a network sniffer
The first step to take in determining if a sniffer is installed on your system is to see if any process currently has any of your network interfaces in promiscuous mode. If any interface is in promiscuous mode, then a sniffer could be installed on your system. Note that detecting promiscuous interfaces will not be possible if you have rebooted your machine or are operating in single user mode since your discovery of this intrusion. Keep in mind that some legitimate network monitors and protocol analyzers will set a network interface in promiscuous mode. Detecting an interface in promiscuous mode does not necessarily mean that an intruder’s sniffer is running on a system. If you find that a packet sniffer has been installed on your systems, we strongly urge you to examine the output file from the sniffer to determine what other machines are at risk. Machines at risk are those that appear in the destination field of a captured packet.

You may need to adjust the command for your particular case. You should be aware that there may be other machines at risk in addition to the ones that appear in the sniffer log. This may be because the intruder has obtained previous sniffer logs from your systems, or through other attack methods.

4.2.6 Check other systems on your network
In examining other systems on your network, use the Intruder Detection Checklist in Appendix A:

4.2.7 Check for systems involved or affected at remote sites
While examining log files, intruder output files, and any files modified or created during and since the time of the intrusion, look for information that leads you to suspect that another site may be linked with the compromise. We often find that other sites linked to a compromise (whether upstream or downstream of the compromise) have often themselves been victims of a compromise. It is therefore important that any other potential victim sites are identified and notified as soon as possible.

4.3 System Repair

4.3.1 Install a clean version of the operating system
Keep in mind that if a machine is compromised, anything on that system could have been modified, including the kernel, binaries, data files, running processes, and memory. In general, the only way to trust that a machine is free from backdoors and intruder modifications is to reinstall the operating system from the distribution media and install all of the security patches before connecting back to the network. Merely determining and fixing the vulnerability that was used to initially compromise the machine may not be enough.

4.3.2 Disable unnecessary services
Ensure that your system is configured to offer only the services that the system is intended to offer, and no others. Check to ensure that there are no weaknesses in the configuration files for those services, and that those services are available only to the intended set of other systems. In general, the most conservative policy is to start by disabling everything and only enabling
services as they are needed.

4.3.3 Install all vendor recommended security patches
Ensure that the full set of security patches for each of your systems is applied. This is a major step in defending your systems from attack, and its importance cannot be overstated. Check with your vendor for any updates or new patches that relate to your systems.

4.3.4 Consult advisories, summaries, and vendor-initiated bulletins
Consult past industry advisories, summaries, and vendor-initiated bulletins, and follow the instructions that are relevant to your particular configuration. Verify that you have installed all applicable patches or workarounds described in the industry publications.

4.3.5 Caution use of data from backups
When restoring data from a backup, ensure that the backup itself is from a non-compromised machine. Keep in mind that you could re-introduce a vulnerability that would allow an intruder to gain unauthorized access. Also, if you are only restoring users’ home directories and data files, keep in mind that any of those files could contain Trojan horse programs. You may want to pay close attention to .rhosts files in users’ home directories.

4.3.6 Change passwords
After all security holes or configuration problems have been patched or corrected, change ALL passwords of ALL accounts on the affected system(s). Strong passwords should be used.

5. Appendices
Appendix A – Intrusion Checklist

6. Examine log files for connections from unusual locations or other unusual activity.

7. Look for scripts everywhere on your system, especially in the /temp directory.

8. Check your system binaries to make sure that they haven’t been altered.

9. Check your systems for unauthorized use of a network monitoring program, commonly called a sniffer or packet sniffer. Intruders may use a sniffer to capture user account and password information.

10. Examine all the files that are run by ‘cron’ and ‘at.’

11. Check for unauthorized services.

12. Examine the password log files on the system and check for modifications or suspicious entries.

13. Check your system and network configuration files for unauthorized entries.

14. Look everywhere on the system for unusual or hidden files (files that start with a period and are normally not shown by ‘ls’ or ‘dir’), as these can be used to hide tools and information (password cracking programs, password files from other systems, etc.).

15. Examine multiple machines on the local network when searching for signs of intrusion. Most of the time, if one host has been compromised, others on the network have been, too.