Intrusion and Intrusion Detection

IJIS (2001) 1: 14–35 / Digital Object Identifier (DOI) 10.1007/s102070100001 Intrusion and intrusion detection John McHugh ∗ CERT Coordination Center , Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA 15213-3890, USA E-mail: [email protected] Published online: 27 July 2001 – Springer-Verlag 2001 Abstract. Assurance technologies for computer security panel that solutions to the problem will not occur have failed to have significant impacts in the marketplace, spontaneously, nor will they come from the various with the result that most of the computers connected to well-intentioned attempts to provide security as an the internet are vulnerable to attack. This paper looks at add-on to existing systems. the problem of malicious users from both a historical and practical standpoint. It traces the history of intrusion and Although these words could have been written today, intrusion detection from the early 1970s to the present they come from one of the seminal documents in com- day, beginning with a historical overview. The paper de- puter security, a report [3, Preface] prepared by James P. scribes the two primary intrusion detection techniques, Anderson & Co. for then Major Roger R. Schell of the anomaly detection and signature-based misuse detection, USAF in 1972. In the nearly 30years since the prepar- in some detail and describes a number of contemporary ation of the Anderson report, little has changed, except research and commercial intrusion detection systems. It that we are, perhaps, less sanguine about the solvabil- ends with a brief discussion of the problems associated ity of the problem. The line of research and development with evaluating intrusion detection systems and a dis- proposed in the report has produced a few fairly secure cussion of the difficulties associated with making further systems, but none that have achieved commercial success progress in the field. With respect to the latter, it notes or wide-scale deployment. The problems posed by mali- that, like many fields, intrusion detection has been based cious users are rampant, and the inability of commodity on a combination of intuition and brute-force techniques. operating systems to provide more than minimal protec- We suspect that these have carried the field as far as they tion has led to a variety of attempts to secure computing can and that further significant progress will depend on systems through add-on or external means. Firewalls and the development of an underlying theoretical basis for the similar mechanisms form the principle line of defense for field. many installations. Intrusion Detection Systems (IDSs), the primary topic of this paper, are an attempt to iden- Keywords: Computer misuse – Intrusion detection – In- tify both successful and unsuccessful attempts to abuse trusive anomalies – Intrusion signatures – Intrusion de- computer systems. tection systems (IDS) – IDS evaluation The paper begins with a discussion of intrusive activities and the kinds of flaws or vulnerabilities in computing systems that enable them. This is followed by a historical 1 Introduction overview of the intrusion detection field, illustrated with descriptions of systems of both historical and current in- The principal unsolved technical problem terest. With the historical view to give perspective, the found by the working group was that of how paper takes an in-depth look at the technologies involved to provide multilevel resource and information in intrusion detection, considering both their strengths sharing systems secure against the threat from and weaknesses. This is illustrated with descriptions of a malicious user. This problem is neither hopeless current commercial, research and public-domain systems. nor solved. It is, however, perfectly clear to the The problems associated with evaluating IDSs are considered briefly. The paper concludes with a discussion of the ∗ current state of the intrusion detection technology and its CERT and CERT Coordination Center are registered service marks of Carnegie Mellon University prospects for future improvement. J. McHugh: Intrusion and intrusion detection 15 2 Intrusions, intrusive activities, penetrations, compiler to capture files from other concurrent users of and exploits the same processor. Experiences such as these led to systematic investigations of flaws and exploitations as well as From the earliest days of computer security, the possibil- techniques to design and build secure systems. ity that malicious users could defeat protection mechan- Although there are earlier discussions of the issues isms was an area of serious concern. Due to the relatively associated with malicious users, James P. Anderson’s limited networking of early systems and the prevalence of 1980report “Computer Security Threat Monitoring and multiuser batch systems, coupled with the fact that pub- Surveillance” [5] set up the first coherent framework for licly accessible services (such as present-day web servers) an investigation of intrusions and intrusion detection. We were almost unknown, most of the early efforts con- will use the following definitions, given by Anderson in centrated on mechanisms that untrusted insiders could this paper, supplementing them later, as necessary. use to access sensitive materials on multi-level secure 1 systems. Under this model, the primary threat is from Threat: The potential possibility of a delib- legitimate users of the system who try to gain access to erate, unauthorized attempt to: material for which they do not have authorization. By (a) Access information the early 1970s, timesharing and other multiuser systems (b) Manipulate information were well established and there were growing pressures to (c) Render a system unreliable or support multi-level operations on these systems, whether unusable they were actually capable of supporting them or not. In Risk: Accidental and unpredictable ex- 1970, the Defense Science Board’s Task Force on Com- posure of information, or violation puter Security supported by Willis H. Ware of the RAND of operations integrity due to mal- Corporation issued a report [81] that laid the groundwork function of hardware or incomplete for the development of adequate computer systems for the or incorrect software design. processing of multi-level data. As noted in the Anderson Vulnerability: A known or suspected flaw in the report [3], the Ware report did not get the attention it hardware or software design or op- 2 deserved, and systematic approaches to building multi- eration of a system that exposes the level secure systems did not emerge until the mid 1970s. system to penetration or its infor- Early multi-user operating systems provided minimal mation to accidental disclosure. security features and those that were present were often Attack: A specific formulation or execution directed towards protecting the data associated with one of a plan to carry out a threat. task from accidental corruption by another. While mod- Penetration: A successful attack; the ability to erately successful in this respect, such mechanisms were obtain (unauthorized) access to often trivial to defeat. For example, in the mid-1960s, files and programs or the control the resident engineer for a site where the author was em- state of a computer system. ployed gave him a list of about a dozen techniques that a job could use to gain supervisor mode with complete ad- Note that threat class (c) includes what are commonly dressability to all physical memory on systems running called “denial of service” attacks today. Attacks that mis- the IBM OS-360MFT operating system. appropriate computing resources also fall into this cate- In the meantime, security flaws that could be ex- gory. ploited by untrusted users were discovered in a number of Anderson classifies threats as shown in Fig. 1. The first systems that were considered to be secure. Paul Karger task faced by an external penetrator is to gain access to and Roger Schell performed an evaluation [44] of the Mul- the system in question. Note that the true external pen- tics operating system and discovered flaws that would etrator may be either an outsider with no connection to allow an untrusted user to access or modify protected in- the organization that owns or controls the system being formation. Work at the Naval Research Laboratory [74] attacked or it may be someone associated with the organi- uncovered flaws in the sharing mechanisms of Univac’s zation who is not authorized to use the system. In today’s Exec-VIII operating system for the 1108 that would al- world of networked systems, it could also be someone who low users of a language processor such as the FORTRAN has legitimate access to systems on the network, but not to the target of the attack. While there are certain classes 1 A multi-level secure computing system is one that is capable of attacks, notably some denial of service attacks, that do of supporting a mandatory access control policy that bases access not require the penetrator to actually become a recognized decisions on the classifications assigned to the information objects that it stores and clearances given to users on whose behalf pro- user of the attacked system, most attacks aimed at either cesses seek access. accessing or manipulating information require the pene- 2 Although the Ware report contained no sensitive information, trator to take on a user role, even though it may be limited it was classified at the CONFIDENTIAL level to control dissemi- to using (and abusing) a public service offered by the sys- nation of its contents. This, combined with its emphasis on stating requirements rather than suggesting solutions probably reduced its tem. Anderson further characterizes internal penetrators impact substantially. as masqueraders, legitimate users, or clandestine users. 16 J. McHugh: Intrusion and intrusion detection Penetrator Penetrator This hardware bug represents a violation of one Not Authorized Authorized of the most fundamental rules of the Multics de- to use to use sign – the checking of every reference to a seg- Data/Program Data/Program ment by the hardware.

Load more