Expertise and Behavior of Unix Command Line Users: an Exploratory Study
Total Page：16
File Type：pdf, Size：1020Kb
ACHI 2016 : The Ninth International Conference on Advances in Computer-Human Interactions Expertise and Behavior of Unix Command Line Users: an Exploratory Study Mohammad Gharehyazie Bo Zhou Iulian Neamtiu Department of Computer Science Department of Computer Science & Engineering Department of Computer Science University of California, Davis University of California, Riverside New Jersey Institute of Technology Davis, CA, USA Riverside, CA, USA Newark, NJ, USA Email: [email protected] Email: [email protected] Email: [email protected] Abstract—Understanding users’ behavioral patterns and quanti- user session characterization [10] in terms of commands per fying users’ expertise have a myriad applications, from predicting session, errors encountered and directories explored; or pre- user actions and tailoring the environment to that specific user, to dicting high-level user actions [11]. In Section II, we provide detecting masquerade attacks and assessing learning outcomes. a detailed comparison with related work. However, none of Toward this end, we have conducted a study on three Unix these previous studies have attempted to quantify user or command datasets, totaling 263 users and more than 1 million command expertise and study temporal patterns associated commands. We first introduce the notions of command expertise, command line expertise, and command category. Next, we use with users/expertise. these metrics, combined with other attributes to define and We start by describing the datasets and the approach we quantify several user expertise metrics, e.g., category breadth, have used to identify commands and their arguments (Sec- command line expertise. Our study has revealed many Unix tion III). Identifying individual commands is nontrivial due commands characteristics, e.g., Unix command can be grouped to several reasons, such as the myriad ways in which Unix into 25 categories; file management is the most common activity; commands can be chained or used as arguments for other the most commonly used commands are two-characters long. Our commands. study has also revealed many insights into user expertise and behavior, such as: command line length is not an indicator of user In Section IV, we discuss the methodology we have used expertise; users activity is highest on Monday and decreases every for quantifying expertise and assigning expertise values to day through Saturday, picking up on Sunday; peak command commands and users. First, we assign an expertise value usage hours are 11 a.m., 1 p.m. and 4 p.m.; development activities to each command — the higher the value, the higher the happen mostly in the afternoon. probability that the command is used by more advanced users, Keywords–User behavior; user expertise; Unix; empirical study. or requires more advanced knowledge. Next, using command expertise, we assign expertise values to entire command lines. We then group commands into categories such as file man- I. INTRODUCTION agement, editor, and compiler. Based on these metrics, we Unix, Unix-like, and Linux operating systems dominate introduce metrics for users expertise: user category breadth the market in segments where human-computer interaction is to quantify user expertise in terms of the number of different centered around command-line; specifically, the market share categories employed by that user, as well separating users into of these operating systems, in January 2016, was 66.9% high- and low-expertise groups. (server), 100% (supercomputer) and 100% (mainframe) [1]. In Section V, we present our findings. We first characterize Understanding Unix command usage and the behavior of Unix commands in each dataset; we found that file management users has many applications: repetitive sequences of commands is the principal activity, with cd and ls accounting for can be transformed into scripts for correctness and ease of substantial percentages of user commands. We found that the use; temporal patterns can expose busy and idle periods hence most common command length is two; that command line allowing capacity to be scaled accordingly; and noticing that a length is not necessarily an indicator of expertise, and that user U’s commands or temporal access patterns are markedly commands/users permit a natural separation into high- and different compared to U’s prior commands and usage patterns low-expertise commands/users. can indicate a masquerade attack [2] (i.e., U’s account has been compromised and is being used by a malicious user M). We also analyzed temporal patterns in one of the datasets Quantitative measures of user expertise can serve as a base (the only dataset to contain command timestamps). We found for comparing users (e.g., who are the most competent or that command activity tends to decrease from Monday until efficient users?), inferring user roles (e.g., student vs. seasoned Saturday, and increase slightly on Sunday; that peak command developer vs. system administrator) or assessing how users usage hours are 11 a.m., 1 p.m. and 4 p.m.; and that devel- learn. opment activities (use of editors and compilers) peaks in the afternoon. Few studies have focused on understanding user behavior based on command line usage. Rather, prior studies’ focus II. RELATED WORK has been on: masquerade detection (malicious users taking Schonlau et al. [3] have used various statistical methods to control of a legitimate user’s account) [3][4][5][6][7][8][9]; detect masquerade attacks by finding “bad data” (attacker’s Copyright (c) IARIA, 2016. ISBN: 978-1-61208-468-8 315 ACHI 2016 : The Ninth International Conference on Advances in Computer-Human Interactions TABLE I. FEATURE AVAILABILITY FOR EACH DATASET. After separating a line into sections, each section contains a command, along with zero or more parameters. For example, datasets Mahajan Greenberg Schonlau Users 45 168 50 the following line will be separated into 3 sections: * * Commands per user 709 1,441 15,000 ls -l | grep key | less Command timestamp X Session start/end X Backquotes or “backticks” are commands that are executed Command arguments XX Command chaining XX before the rest of a line, and their result is “pasted” at their User Group X position in the line. For example: * Median number of commands per user. vim notes.`date +%F` Hence we look for backquotes in each section and treat it commands) inside sequences of “good data” (benign com- as a separate section; we find the command and its parameters mands issued by a legitimate user). Other studies have sim- (as described below) and we treat the whole section as an extra ilarly used machine learning or grammars for masquerade de- parameter for the section which contained the backquotes. tection [4][5][6][7][8][9]. Greenberg [10] has collected traces of 168 users via a modified csh shell; they partitioned the To count the number of parameters for each command, users into four non-overlapping categories (novice program- we separate each section by space and redirection characters mers, experienced programmers, computer scientists and non- (‘>’, ‘>>’, ‘<’, and ‘<<’). As mentioned above, each programmers) and characterized user sessions in terms of com- section is split based on space or redirection characters. mands per session, errors encountered and directories explored. Redirection to/from a file is also considered a parameter. In Chinchani et al. [12] has focused on the reverse problem some rare cases there are two commands per section such sudo apt-get install blah of generating user models so that synthetic command traces as which contains two sudo apt-get can be generated automatically. Fitchett and Cockburn [11] commands (‘ ’ with 0 parameters and ‘ ’ with 2 developed a predictor model for revisitation/reuse based on parameters). Only two commands were found that used such sudo time user actions (commands, window switching, URL accesses). a feature: ‘ ’ and ‘ ’. Note that our focus is different compared with the afore- To conclude, in the aforementioned example, we detect 3 mentioned efforts. Rather than finding suspicious/anomalous commands: ‘ls’ with 1 parameter, ‘grep’ with 1 parameter, commands for purposes of masquerade detection, we aim to and ‘less’ with no parameter. Notice that all commands answer more general questions: how can command expertise, after the first pipe have one more parameter than immediately command line expertise, and user expertise be operationalized? visible, and that is because the other parameter has been piped. How can users and commands be grouped into categories Finally, we proceeded to identifying sessions, i.e., start that generalize and are stable across datasets? What are the and end of time intervals when users started the command temporal patterns associated with commands, command cate- line interaction. While Mahajan’s dataset is very rich in most gories, and users, e.g., when (time-of-day/day-of-week) does aspects, it does not explicitly record session information, so development happen, as opposed to editing, and when are to identify sessions we computed the time intervals between experts active compared to novice users? consecutive commands, plotted their distribution, and visually identified cut-off points hence session begin/end. III. DATASETS 2) Greenberg: This dataset consists of 168 Unix users [10]. Our analysis is based upon three main datasets — collected Like Mahajan’s dataset, it contains command parameters, and by other researchers [9][3][10] — totaling 263 users and complex command lines, but it does not contain timestamps 1,023,993 commands. Table I provides an overview of the like Mahajan’s. It does however contain session delimiters. datasets and the features they include: each dataset consists of Another feature of this dataset is that it categorizes users real commands collected from actual usage — ranging from into 4 categories based on their expertise. In detail, there 709 up to 15,000 commands per user. The dataset attributes are 52 computer scientists, 36 experienced programmers, 55 vary: while Mahajan’s set contains a timestamp for each novice programmers and 25 non-programmers respectively.