CPU Resource Control and Accounting in the NOMADS Mobile Agent System1 Paul T. Groth and Niranjan Suri Institute for Human & Machine Cognition University of West Florida {pgroth,nsuri}@ai.uwf.edu
1. Introduction NOMADS [7] is a mobile agent system for Java-based mobile agents. One of the key enhancements provided by NOMADS is the ability to monitor and control resources consumed by agents running within the NOMADS environment. This paper describes the CPU resource control mechanism, which complements the disk and network resource controls already available in the NOMADS environment.
The CPU resource control allows the rate of CPU usage of agents to be limited. Such controls are necessary to protect a host from malicious agents that may launch denial-of- service attacks. NOMADS also allows the limit to be changed dynamically. The ability to apply and change resource limits can be used not only to protect the host, but also to control poorly written or buggy agents from running amok and to change the priorities of executing agents.
The rest of this paper is organized as follows. Section 2 presents a brief overview of the NOMADS mobile agent system. This is followed by a short discussion of our requirements and motivations for CPU resource control in section 3. Then section 4 discusses the current implementation of the resource control mechanism. Section 5 describes the CPU accounting mechanisms. Section 6 presents some initial performance results. Finally, we conclude with a discussion of related and future work in sections 7 and 8.
2. Overview of NOMADS The NOMADS mobile agent system offers two key advantages over other Java-based mobile agent systems: strong mobility and fine-grained resource control. Strong mobility allows the execution state of an agent to be captured and moved with the agent from one host to another. NOMADS uses the state-capture mechanism to go beyond strong mobility and also support forced mobility, which allows the system to move agents from one host to another by force (potentially in a completely transparent manner to the agent). Such forced mobility is extremely useful for situations that involve load balancing, process migration, systems shutting down, etc.
The resource control mechanism in NOMADS allows control over the rate and quantity of resources used by agents. Dynamically adjustable limits can be placed on several parameters including disk read rate, disk write rate, network read rate, and network write
1 This research is supported in part by DARPA’s Control of Agent Based Systems (CoABS) program and the National Technology Alliance (NTA)/NIMA. rate. Such limits are expressed in bytes per millisecond. Quantity limits include disk space, total number of bytes read from the disk, total number of bytes written to the disk, total number of bytes read from the network, and total number of bytes written to the network. These resource control mechanisms complement Java’s access control mechanisms and help in making the NOMADS system much more secure against malicious agents.
The NOMADS system consists of two major components: the Aroma Virtual Machine (VM) and the Oasis Agent Execution Environment. The Aroma VM is a clean-room implementation of a Java-compatible VM and offers several capabilities over those provided by the standard Java VM including the ability to capture execution state and to limit resource access. Oasis builds on top of the capabilities of the Aroma VM to provide strong mobility and resource control. The Oasis Console program allows users to administer the NOMADS environment and interact with running agents. Figure 1 shows the structure of NOMADS.
Oasis Execution Environment Oasis Console (Administration and Interaction Program) Oasis Process AromaVM AromaVM Policy Manager (Running Agent A) (Running Agent B)
Dispatcher Protocol Handler Messaging Handler
Figure 1: NOMADS Components
The resource limits enforced by the Aroma VM may be specified in a number of different ways. NOMADS includes a policy manager component that allows administrators to specify security policies for different agents. Policies may specify limits for any or all the different resource controls supported by NOMADS. When an agent is authenticated, the policy manager locates the policy that needs to be applied to the agent and notifies the Aroma VM running that agent to enforce the resource limits. Oasis also supports a network-based administration protocol that allows other components to dynamically query the resource utilization and to change the resource limits of any executing agent.
3. Requirements for CPU Resource Control Unlike the disk and network resource control mechanisms, the widely differing speed of processors poses a unique challenge for CPU resource control. In particular, we wished to support two alternative means of expressing the resource limit. The first alternative is to express the limit in terms of some quantity of processing done per unit time by the agent. Since NOMADS executes Java bytecodes, the approach we have adopted is to express the limit in terms of bytecodes per millisecond. The advantage the administrator expressing a limit in terms of bytecodes per unit time is that given the processing requirements of an agent, the agent’s execution time (or time to complete a task) may be predicted.
Another advantage of expressing limits in terms of bytecodes per unit time is that the limit is system and architecture independent. We plan to extend NOMADS to allow agents to specify requirements for minimum resource limits. Using bytecodes per unit time allows agents to request quality of service in a platform independent manner. This is particularly important if a resource conscious mobile agent travels to systems with different CPU characteristics.
The second alternative is to express the limit in terms of some percentage of CPU time that may be consumed by the agent. This limit is expressed as a number between 0 and 100. The disadvantage of the previous mechanism of expressing limits in terms of bytecodes per unit time is that the administrator of the host may not know the demands that the agent may place on that particular host. This variability may result in a hosts resources being either over committed or underutilized. This problem is addressed by allowing administrators to express limits in terms of percentage of CPU time.
We expect that in the general case, agent developers (or agent owners) will specify requirements for the agents in terms of bytecodes per unit time whereas system administrators will specify constraints in terms of percentage of CPU time. Although the current thread scheduler in NOMADS does not support agents specifying requirements, we believe that when requirements are supported, that the scheduler will give priority to constraints specified by administrators.
4. CPU Resource Control Implementation CPU resource control in NOMADS is implemented at two levels. At the lower level, the Aroma VM enforces a limit on the number of bytecodes that a thread should be allowed to execute per millisecond. At a higher level, a thread scheduler takes limits specified in terms of CPU percentage and maps those limits to underlying per-thread limits expressed in bytecodes per millisecond. The thread scheduler runs as a separate native thread in the Aroma VM and periodically monitors the CPU utilization of each Java thread. When the scheduler observes that the CPU utilization of a Java thread has increased, the scheduler lowers the Java thread’s bytecodes per millisecond limit and vice-versa. The following two subsections describe the implementation in more detail.
1. Limit Expressed as Bytecodes Per Unit Time Each Aroma VM executes a single agent but may contain several concurrently running Java threads. The Aroma VM allows a bytecodes per millisecond limit to be placed on each individual thread running. After every Java bytecode is executed, the VM invokes a function named enforceCPURateLimit, which performs the necessary accounting and enforcement.
Figure 2 shows the basic algorithm implemented by the enforceCPURateLimit function. The algorithm first computes the number of bytecodes that should be allowed in a 100-millisecond interval. Then, each time the number of bytecodes allowed per interval has been executed, the algorithm checks to see how much of the 100 milliseconds have elapsed. If there is any remaining time, the algorithm puts the thread to sleep so that no more bytecodes are executed until the start of the next interval.
Note that for efficiency the algorithm uses intervals of 100 milliseconds in enforcing the CPU rate limit. This implies that within 100 milliseconds, the algorithm does not care about the distribution of the load on the CPU. However, the algorithm guarantees that the average number of bytecodes per 100 milliseconds will be limited correctly. This interval may be decreased if a more even distribution of the CPU load is desired although decreasing the interval will result in an increase in overhead.
// State Variables intervalStartTime // Start time of the current 100 MS interval bytecodeCount // Number of bytecodes executed in the interval bytecodesPerInterval // Number of bytecodes allowed in one interval
bytecodeCount++
if (bytecodeCount == bytecodesPerInterval) currentTime =
Figure 2: Algorithm for enforceCPURateLimit
2. Limit Expressed as Percentage of CPU Usage Another way to specify a CPU limit inside NOMADS is in terms of percentage of CPU usage. Enforcement of this limit is the responsibility of the thread scheduler, a component inside the Aroma VM. A user, monitor application, or the agent itself may cause a CPU percentage limit to be set. The thread scheduler is responsible for translating the percentage rate limit into a bytecode limit that can be enforced by each thread. Figure 3 shows the role of the thread scheduler.
Virtual Machine Thread Scheduler User / setCPUPercentageLimit Monitor / Agent getCPUUtilization
setBytecodeLimit
Thread Thread Thread
Figure 3: Role of the Thread Scheduler The thread scheduler requires that the performance of CPU in terms of bytecodes per millisecond (BPMS) be known. When the Aroma VM initializes itself, it computes the system’s performance in terms of BPMS by running an instruction mix and measuring the elapsed time.
Once the BPMS for a system has been determined, the thread scheduler uses the algorithm shown in Figure 4 to enforce CPU percentage rate limits. When a limit is set or changed, the algorithm uses the BPMS factor to compute the bytecode per millisecond limit that needs to be placed on the thread. This limit is then enforced using the previously described mechanism that enforces the bytecodes per unit time limit.
Notice that the two else if statements execute even when a new percentage has not been set. The sampling mechanism used to determine the BPMS is not an exact value for every thread since it is based on an instruction mix. Therefore a method to tune the bytecode limit is necessary. The algorithm accomplishes this tuning by allowing the bytecode limit to be incrementally adjusted in order to exactly reach the percentage limit specified.
Also notice that a tolerance of 3 percent is instituted when the utilization is below the rate limit. The tolerance allows for minor variations in the CPU utilization without the thread scheduler constantly adjusting the bytecode limit.
// State variables BPMS // Bytecodes per millisecond newLimitIsSet // Whether or not a new limit has been set percentageLimit // Limit in percentage of CPU
currentCPUUtilization =
if (newLimitIsSet equals true) bytecodeLimit = (percentageLimit / 100) * BPMS setBytecodeLimit (bytecodeLimit) else if (currentCPUUtilization > percentageLimit) setBytecodeLimit (currentBytecodeLimit – BPMS) else if (currentUtilization < percentageLimit + 3) setBytecodeLimit (currentBytecodeLimit + BPMS)
Figure 4: Thread Scheduler’s Algorithm
5. CPU Accounting Interface Another feature of the NOMADS agent system is the ability to record and query the agent’s use of system resources. This usage can be queried, as stated above, via a network-based protocol. There are two types of CPU resources that may be queried: the current utilization of the CPU for each Java thread in the Aroma VM and the total number of bytecodes executed by an agent. This accounting information can be used by an administrator, by other agents, or by the agent itself.
The NOMADS Guard is an example of a high-level monitoring agent that relies on the accounting interface. The Guard periodically queries each monitored agent’s resource consumption and looks for potential patterns of resource abuse. If an abuse (such as a denial-of-service attack) is detected, the Guard lowers the resource limits of the misbehaving agent and notifies an administrator [1]. Figure 5 shows the visualization of the NOMADS Guard Agent.
Figure 5: NOMADS Guard Agent Visualization
A foreseeable use of CPU accounting in NOMADS is the ability to charge a fee for the CPU resources used by an agent. These cycles could be measured in the total number of bytecodes executed by the agent.
6. Performance Results We used three different benchmarks to measure the overhead of the CPU resource control: Linpack [4], Tak [5], and the UCSD Java benchmark [3]. The performance times were collected without the resource control code and with the resource control code in place (but without any resource limit). Table 1 below summarizes the results of the performance measurements. All times are reported in seconds. The results were colleted on a Dell Pentium II 450 MHz system with 256 MB of RAM running Windows 2000. The results show that the CPU resource control code introduces an overhead of 6.6% to 6.9%.
The current implementation does not support systems with multiple processors. In the future, we would like to allow CPU resource limits to be expressed as a percentage of the overall processing power of system, rather than on a single processor basis.
References
1. CoAX - Coalition Agents eXperiment. http://www.aiai.ed.ac.uk/project/coax/
2. Czajkowki, G., & von Eicken, T. JRes: A resource accounting interface for Java. Proceedings of the 1998 ACM OOPSLA Conference. Vancouver, B.C., Canada.
3. Griswold Bill, Phillips Paul. UCSD Benchmarks for Java. http://www- cse.uscd.edu/users/wgg/JavaProf/javaprof.html
4. Hardwick, Jonathan. Optimized Java Linpack Benchmark. http://www.cs.cmu.edu/~jch/java/linpack.html.
5. Harold, Elliotte. The Tak Benchmark Applet. http://www.ibiblio.org/javagaq/newsletter/TakApplet.html.
6. Lal, M. & Pandey, R. CPU Resource Control for Mobile Programs. Proceedings of the First International Symposium on Agent Systems and Applications and the Third International Symposium on Mobile Agents (ASA/MA’99). IEEE Computer Society Press.
7. N. Suri, J. M. Bradshaw, M. R. Breedy, P. T. Groth, G. A. Hill, and R. Jeffers. Strong Mobility and Fine-Grained Resource Control in NOMADS. Proceedings of the Second International Symposium on Agent Systems and Applications and the Fourth International Symposium on Mobile Agents (ASA/MA'2000). Springer-Verlag Press.