CSM27 Security

University of Surrey

Autumn 2008 – Week 5 Exercise

Georgios V. Michalakidis [email protected]

Weekly Exercises 1.1 Gollmann 5.2 Can you have security without security kernels? Discuss the advantages and disadvantages of having a security kernel built into the Kernel (as opposed to the Application Layer) to form the base (TCB). 1.2 Gollmann 5.5 Some buffer overrun attacks put the code they want to be executed on the call stack. How can the ability to distinguish between programs and data help to construct a defense against this particular type of buffer overrun attacks? Briefly describe a protection mechanism based on this distinction. 1.3 P&P 5.9 Consider time-sharing on the CPU. Explain what is necessary to provide temporal separation (with proper security). Your answer can take one of two approaches • Describe the (formal) conditions which must be met in order for two processes to be adequately separated. • Describe each action which must be taken by the CPU and OS during a context switch (i.. when one is swapped out and a new one in)? Give rationale for each condition/action.

Response to Exercise 1.1:

Security kernels are not only needed but are obligatory in order to have good protection over code or data or any other type of insecure state. In all modern Operating Systems there exist different and separated security levels inside the kernels, coping with intrusion matters or execution of malicious code.

If we were only using security on the application layer, chances are we would be facing problems caused by bugs or any other types of threats. Viruses can at certain times either bypass security (that would usually, but sometimes would not be) handled by the Operating System kernel. The most innocent application could have a serious bug that either causes a security thread by itself, or by being combined with some other piece of intruder . In that case, we would result in either compromised data (Confidentiality) or Integrity faults.

A secured Operating System with different levels of protection (e.g. separating the User data from the Program data as some Central Processing Units would do) can

2 therefore have many advantages over the possibility of letting an application take control of security matters, and result in more trust than one would or should offer to lines of code on applications rather than a solid Operating System.

On the opposite side, however, we can never trust the Operating Systems either. It is possible that some kind of software specialized in a certain field can be of good quality and thus not contain any bugs or security risks (or the least possible, anyway). This might not be the case most times, still however, we could face Operating System threats and faults as they usually contain tons of programming code, and as one would guess, more bugs. This is why patches are released for operating systems all the time.

Another important factor against having a security kernel based on the operating system is how these bug threats are widely known when found in OS’s. Different pieces (or versions) of software could be facing the same problems and have the same security risks although updated, because of the nature of the Operating System (being used by millions of users in most cases). Also, viruses and other types of malware can use these holes until a patch is released, or even worse, until the average user has finally managed to install this certain patch/update.

If we summarize the above, we should be thinking of 2 different levels of security, built in on both the application layer and Operating System kernel. You can’t have too much security… and in case of sensitive data, using different layers for the same (or enhanced as we get lower on the core) features could be not only valuable, but compulsory too.

Response to Exercise 1.2:

The buffer overrun attack itself, uses the calls inside functions to go over the size of the set variables and using an overrun, execute code which can be part of functions stored in the memory (or in certain cases pass their own code through the

3 arguments, for example). That way, malicious code can execute code as if it were data. In the case where a CPU has the ability to distinguish between programs and data, no functions would be executed while the CPU knew it was at that time handling data.

Based on the above, a mechanism using this type of protection would have 2 states, thus switching between Data mode and Program mode. No code would be allowed to be executed when the assembly directions were manipulating data. That way, the CPU itself would prevent moving to other memory addresses as a result of a buffer overrun and executing code put on the call stack. No such code would execute (blocked from the CPU or other lower security layers) when being on Data state with specific instructions on what can be handled by the CPU at that point.

Response to Exercise 1.3:

Taken from the IEEE publication “VLSI Register, Instruction and Data Caches Suited to on Chip CPU Multi-Threading Support for Real-Time Multi-Media Applications” by Graham R. Hellestran, http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=00569278

“Further analysis of data hazards [able to be caused with multi-threading] indicates that there are two types [of those hazards] : • the first due to resource contention, arises when distinct contemporaneous accesses are attempted to: the same data register file, the same data or instruction cache, the virtual to physical memory translator, and the same physical (primary) storage; and • the second due to the temporal separation of data, being accessed from cache or primary storage in a particular pipeline stage , from that which is intended to be its alias in the register file.”

If we take consideration of the above abstract, we understand that there are problems that need to be considered so that we end up having totally separated processes so that there is no interference (or security threat) on any data transferred between them (confidentiality) or changed by one another (integrity). We need to have separated Registers (which include but are not limited to: General purpose

4 registers, program counter, stack pointer and status register ), so that there is no way one process can change or ‘see’ the data of the other. Such protection, could limit possible Confidentiality or Integrity problems. We need to limit the communication between processes as much as possible. In case of Unix, we have a protection over the files open and used, since any process needs to explicitly open any file. Another way, is to allocate a different memory address space for each process, that Cannot be overcome (preventing overruns?). Virtualization software (sandboxes) might be simple frameworks offering easy ways to implement those security related features easily, unless of course they are either provided by the OS kernel in conjunction with the CPU security layers and states. Another parameter to be mentioned, is whether we are talking about processes or thread, as the second ones need to, in principle, share the same resources. This is not necessarily a price in need to be paid, it’s more of what they are about. Sharing resources over threads that could run at the same time, resulting in better speed and efficiency, although there should be some considerations during coding as the CPU or OS won’t be able to tell when we have a Confidentiality or Integrity breach based on some bug in our code: • Either because of faulty software design • Or because of not following a hundred percent of the design itself

Getting back to the CPU sharing time over processes, we could have the following example as a way to overcome collisions: 1. Halt execution of Process N 2. Save all Registers of Process N to RegMemN 3. Chose a process to continue, e.g. Process N-1 4. (Clear Registers and) Load all Registers of Process N-1 from RegMem(N-1) 5. Continue Execution of Process N-1 Step 1 ensures we will have the most up-to-date Registers to same on our Temporary Memory (INTEGRITY). From Steps 2 and 4 we can have the result of N temporary registers memory spaces, needed in total, where N is the number or processes between which we would want to switch at any time. Step 4 should make sure that either all Registers are emptied first, or that data of all existing Registers in CPU is transferred from the RegMemx (CONFIDENTIALITY).

5