CPSC 424/624 Quiz #4 Solutions Closed book/computers Spring 2017 Last update: 3/3/2017 Name: 1 Create a simple Bash command line pipeline involving the find utility to assemble a set of files and then have this result piped to wc to sum the number of lines . You should use xargs in your pipeline. Explain what xargs does, using your example to illustrate. Include in your summary what would happen if your pipeline did not use xargs. For example, what is the problem if you were to run your pipeline but without using xargs. Based on this exercise (along with any additional reading), please explain the benefits of xargs. Solution: xargs [option ...] [command [initial-arguments]] xargs is another way to call commands or programs. The term xargs is synonymous with 'combine arguments'. It builds and executes command lines by gathering together arguments from standard in (which as we will see are many times supplied by find) A simple example….in a directory, issue the following: #ls; ls | wc; ls | xargs; ls | xargs | wc; From the bashScripts/monitors directory, the command line result is: 1 ls; 2 monitorScriptRunzhen.py monitorScriptRunzhen.sh mySysMon.sh paul.py paul.sh tecmint_monitor.sh xin.py xin.sh 3 ls | wc; 4 8 8 109 5 ls | xargs; 6 monitorScriptRunzhen.py monitorScriptRunzhen.sh mySysMon.sh paul.py paul.sh tecmint_monitor.sh xin.py xin.sh 7 ls | xargs | wc 8 1 8 109 9 ls | xargs wc 10 72 208 1885 monitorScriptRunzhen.py 11 57 184 1787 monitorScriptRunzhen.sh 12 63 206 1456 mySysMon.sh 13 82 219 1866 paul.py 14 64 201 1444 paul.sh 15 117 491 4015 tecmint_monitor.sh 16 75 252 2329 xin.py 17 59 124 1167 xin.sh 18 589 1885 15949 total Lines 1 and 2 display the output of ‘ls’- a list of filenames with each separated by a ‘\n’. Lines 3-4 counts the number of results. Line 5-6 shows that xarg’s will gather the results from the previous command and places them in a single line. Note that od is useful….. try ls | od –c and then ls | xargs –c If a program is specified to xargs (wc in our example), xargs calls the program passing the data from the pipe as parameters. If xargs is used without a p, total of program, it defaults to issuing an echo of the parameters. Lines 7 and 8 (ls | xargs | wc) pipes to wc effectively a string (1 line, 8 words…) Line 9- 18 - (ls | xargs wc ) causes wc to be invoked with 8 parameters….wc interprets each as a file name, and provides the wc output for each file on a separate line. Xargs adds flexibility to command lines – it allows pipelines that would require a more complicated script to achieve without xargs. GNU’s parallels can be viewed as a replacement of xargs, adding support for concurrency. 2 On your VM, issue the following commands: sleep 5; alert; Poke around on your VM to learn about ‘alert’. What is the most effective method for learning about something you see used in a command line? Once you have found alert, explain how it works. So…if it is a program, deduce how it works. If it is a script, decompose it to learn and then explain how it works. Solution: The type command is usually the best way as it can give you the information needed to know roughly what to look for. Using type, we find that ‘alert’ is an alias. With a bit of digging you would find the alias in ./bashrc. type alert alert is aliased to `notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e 's/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//')"' `notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e 's/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//')"' Decomposing the alias: Notify-send is a program allowing a script to send desktop notifications. The –i allows the caller to specify a graphics icon that will be displayed with the message. Executing the following: `notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e 's/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//')"` Does display a msg (upper left of desktop) but the txt message is the actual shell code between the backquotes. Why? It is doing the right thing….it is displaying the last command….in this case the notify- send. When called by the alias, the last command is delay 5 3 The following script (paul.sh) is available at https://people.cs.clemson.edu/~jmarty/courses/Spring 2017/CPSC424/code/bashScripts/monitors/. This following is an excerpt from the monitor script. If invoked as ‘./paul.sh 5 enp0s17’ it will send the following information to standard out every 5 seconds. The second param is the network interface name (use ifconfig to get this) we want to monitor. Please answer the following questions: On lines 24-25, explain exactly what a ‘5 minute load CPU utilization ‘ is and second, exactly how this code obtains that information On lines 27-31, explain what exactly the awk code produces and explain exactly how the awk code actually works. Interval (sec): 5 Inteface: enp0s17 Number of Cores: 1 5 Minute Avg CPU Utilization (%): 8.0 Outbound BW (bits): 0 Inbound BW (bits): 0 1 #!/bin/bash 2 # System Health Monitor 3 # monitorScript.sh 4 interval=$1 5 interface=$2 6 if [ "$#" -ne 2 ] 7 then 8 echo 9 echo "Usage: $0 <interval> <interface>" 10 echo 11 exit 1 12 fi 13 declare -i tempOut=0 14 declare -i tempIn=0 15 while true; do 16 clear 17 echo "Interval (sec): "$'\t'"$interval" 18 echo "Inteface: "$'\t'"$interface" 19 echo -n "Number of Cores: "$'\t' 20 cores=$(grep pro /proc/cpuinfo -c) 21 echo $cores 22 # - CPU Utilization 23 echo -n "5 Minute Avg CPU Utilization (%): "$'\t' 24 util=$(cat /proc/loadavg | awk '{print $1}') 25 echo "scale=1; (($util*100)/$cores)/1" | bc # >> cpuUtilBash.txt 26 # - Bandwidth Consumption 27 echo -n "Outbound BW (bits): "$'\t' 28 outbound1=$(awk -v interface="$interface" -F'[: \t]+' \ 29 '{ sub(/^ */,""); 30 if ($1 == interface) print $10; 31 }' /proc/net/dev) Solution: Note that this implementation has problems. First, the original script used the 1 minute avg field in /proc/loadavg … it should use the second field (5 minute avg). Second, we need to be precise in describing what we actually want to monitor. The loadavg assesses the demand for not only the CPU but also for I/O by the count of both types of processes waiting on system resources. In general, this should be divided by the number of total CPU cores to get an average across all CPUs. This is different from a CPU utilization which counts the relative amount of time the CPU is busy (versus idle). This measure is more of a result of the OS scheduling – assessing the % of time the CPUs are busy. The measure is the ratio of the number of scheduling intervals that were allocated to a process to the total number of intervals in the measurement period. If a CPU in an interval is not allocated, it is ‘idle’. The measure can be per CPU/Core or it can be the avg for all cores. The ‘5 minute load CPU utilization ‘ is based a piece of information available in the /proc/load. A ‘cat /proc/load /proc/stat’ on my VM shows: /proc/load 0.08 0.04 0.00 1/413 4071 /proc/stat: cpu 14468 150 17920 5610958 594 0 268 0 0 0 cpu0 5913 135 7698 2806934 183 0 121 0 0 0 cpu1 8555 15 10222 2804024 410 0 146 0 0 0 intr 4782837 36 8466 0 0 0 0 0 0 0 0 0 0 1004 0 0 28013 14238 19425 2 0 46538 25509 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ctxt 14984512 btime 1488540624 processes 4070 procs_running 2 procs_blocked 0 softirq 1113111 3 419335 7264 34027 37660 0 8486 419336 0 187000 A ‘man proc’ explains /proc/loadavg as follows: The first three fields in this file are load average figures giving the number of jobs in the run queue (state R) or waiting for disk I/O (state D) averaged over 1, 5, and 15 minutes.