International Journal of Advances in Electronics and Computer Science, ISSN(p): 2394-2835 Volume-6, Issue-11, Nov.-2019 http://iraj.in OPERATING SYSTEM DEVELOPMENT FROM SCRATCH

SHIVAM BASIA

Student, KIIT Deemed To Be University, Bhubaneswar, India E-mail: [email protected]

Abstract - This research project is based on developing an operating system from scratch to demonstrate and understand how a entire computer system actually gets built and how it works internally. Our Operating System has been built by using Assembly Language only. Our Operating System in general consists of a Bootloader, a Kernel,FAT32 partition file system and a simple shell all of which has been built from scratch. Various command line features has been added in our operating system like change directory, listing and even the cat command for displaying contents of any text file. The size of our Operating System is 8 MB. The memory disk of our Operating System is divided into 16384 blocks with each block’s size being 512 bytes. The testing of our Operating System was performed in two emulators namely qemu and virtualbox.

Keywords - Operating System, Bootloader, Assembly Language,Kernel,FAT32 partition file system, shell.

I. INTRODUCTION Thirdly, the entire boot sector gets load at address 0000:7c00 and with that the CPU starts executing the Before the development of an operating system from very first instruction it founds at that address. So, for scratch it is important to know what is an operating building our bootloader and for ensuring that the system and what components and resources it consists kernel gets load from the FAT32 partition file system of. An operating system can be said to be a software by the boot sector we defined the first 90 bytes of the which controls the entire operation of a computer boot sector as the FAT32 BPB (BIOS Parameter Block) system as well as handles all its resources. Some and external BPB and a Disk Address Packet for functions of operating systems are to manage memory memory address handling. The details contain in BPB, and other system resources, keep the security and external BPB and the disk address packet are given in access policies in check as well as to schedule Table 1,Table 2,Table 3 respectively. After that the processes. An operating system in general consists of a bootloader loading code was written, the steps of bootloader, a kernel and a command line interface which are mentioned in Table 4. BPB, externalBPB, which we can be called as a shell. A boot loader can be disk address packet along with bootloader loading said to be a software which brings the kernel into code was enough to load the kernel at 17c0:0000 and memory and provides kernel with all the required transfer control to the kernel by using long jump. information it needs to work correctly. The kernel of an operating system is basically a software which i. Jump bit specifying the address of kernel to jump enables any other programs to execute in the computer ii. Original equipment manager id for security and system. The kernel also handles various events access policies generated by hardware and software which are also iii. Bytes per sector called as interrupts and system calls respectively. It iv. Sectors per cluster also performs the task of management of access to v. Reserved Sectors resources. A command line interface or a shell is vi. Number of File Allocation Tables typically a program that is integrated into the vii. Root Entries distribution of an operating system which basically viii. Total Sectors helps human beings in getting an interface with the ix. Media Descriptor computer. x. Sectors per File Allocation Table

xi. Sectors per Track II. DEVELOPMENT OF BOOTLOADER xii. Number of Heads

Firstly, a Bootloader was to be created for our xiii. Hidden Sectors Operating System. So, to create a Bootloader it was xiv. Total Sectors important to know the Booting Process. The Booting Table 1:Details contain in FAT32 BPB

Process works in three ways in a computer system. i. Sectors per File Allocation Table Firstly, when a computer is switched on, the computer ii. Flags performs a Power-On Self Test (POST) which refers to if the computer system is getting enough power iii. FAT Version supply to run. Secondly, it detects if a bootable disk iv. Root Cluster has been inserted in the disk drive or not and if inserted v. File System Information Sector it loads the boot sector which is generally the first 512 vi. Backup Boot Sector bytes on the disk and will be executable if the last two vii. 12 reserved bits bytes of the boot sector are the magic value 0xaa55. viii. Drive Number

Operating System Development from Scratch

59 International Journal of Advances in Electronics and Computer Science, ISSN(p): 2394-2835 Volume-6, Issue-11, Nov.-2019 http://iraj.in ix. Drive Signature vii. Restore the drive number. x. System Identifier viii. Mount the FAT32 partition. Table 2: Details contain in external BPB ix. Print the mount message. x. Since the DL register already contains the drive i. Unused sectors number, clear DH. ii. Number of sectors xi. Read the partition entry DH on drive DL. iii. Offset xii. Preserve EDX for copying the LBA number into iv. Segment EDX for printf. v. Starting Logical Block Address xiii. Restore EDX. Table 3:Details contain in disk address packet xiv. Final mount the partition. xv. Initialize the shell. i. Clear the direction flag. xvi. Jump into the shell. ii. Set Data Segment to Code Segment. Table 5:Initializing the init code of the kernel iii. Place the stack at the top of the current segment. iv. Initialize FS. Next, we had to find a way to halt if an interrupt is v. Store the current drive number. encountered. For the interrupt to work we performed vi. Find the FAT offset. the steps given in table 6. vii. Add the number of reserved sectors. viii. Store the sectors. i. Register an interrupt handler. ix. Preserve EAX. ii. Initialize BL as the interrupt number. x. Find the data cluster offset. iii. DX will be the CS offset. xi. Set CX to the number of FATs. iv. Disable interrupts by calling clear interrupt flag. xii. Perform the operation of AX*CX -> EDX:EAX. v. Create the Interrupt Vector Table(IVT) segment. xiii. Add the previous offset. vi. Initialize the BH register as 0 and multiply BX xiv. Store the data offset. register by 4. xv. Find the number of directory entries per cluster. vii. Add ES:BXoffset to DX register. xvi. Multiply CX by 16 as its the number of 32-byte viii. Move the CS register to DX register. directory entries per a 512-byte sector. ix. Enable interrupts by calling set interrupt flag. xvii. Store the value. x. Finally Return from an interrupt without xviii. Read the root clusters. restoring the original CF. xix. The buffer will be at 0050:0000. Table 6:Interrupt handling xx. Read the first cluster of the root directory. xxi. Clear DI. Next, kernel system call was to be designed. Forit,we xxii. Iterate CX times. performed the steps given in table 7. xxiii. For finding the next entries compare the file names. i. Assign BP register to specify the function to call. xxiv. Restore the registers when the entry is found. ii. Assign 0 for kalloc function. xxv. Load the number of the next cluster. iii. Assign 1 for kfree function. xxvi. Finally, load the kernel at 17c0:0000 and transfer iv. Preserve DS and set DS to the kernel segment. control to the kernel by long jump. v. Compare BP to 0 to jump to kalloc or kfree Table 4: Steps of bootloader loading function. Table 7:Kernel System Call Finally, the development of bootloader was completed. Next, The console I/O system call was to be designed. For it, we performed the steps in table 8. III. DEVELOPMENT OF KERNEL i. Assign BP register to specify the function to call. Since the bootloader was initially developed, it had ii. Assign 0 for print. already placed the kernel at 17c0:0000.Now,we had to iii. Assign 1 for printf. jump to the initcode. The steps of init code initializing iv. Assign 2 for newline. are given in table 5. v. Assign 3 for printChar. vi. Assign 4 for readLine. i. Initialize the segments. vii. Compare BP to 0,1,2,3,4 for jumping to the ii. Preserve the drive number. required I/O system call. iii. Call the Initkalloc subroutine. Table 8: Console I/O System Call iv. Clear the screen by calling console.clearscreen. v. Print the boot message. Also, interrupt service routines for console, string and shell was designed and integrated with the kernel for vi. Register the system calls by calling interrupt screen handling, keyboard handling and command line subroutines.

Operating System Development from Scratch

60 International Journal of Advances in Electronics and Computer Science, ISSN(p): 2394-2835 Volume-6, Issue-11, Nov.-2019 http://iraj.in interface respectively. The steps for building console recursively. subroutine, string subroutine and console subroutine xvii. Lists the files in the current directory along with are given in table 9, table 10,table 11 respectively. their attributes and initialize the directory enumeration where DX counts the entries i. Initialize our screen width and height as 80 and printed. 25 respectively. xviii. Parse the directory entry at FS for the next entry. ii. Clear the screen and go to 0,0. xix. Set the temporary file name type of the file as '-' iii. Set cursor position as 10h at 0x02. if its a file and as '+' if its a directory. iv. Set 10h at 0x0e for printing characters and ASCII xx. Print the filename and increment the DX register string at DS:SI. until all files has been listed. v. Continue loading characters until its zero and xxi. Find the directory entry in the current directory finally print the 10h. whose filename matches that at ES:DI which vi. Set ax at 0x0e0d and AL at 0x0a for new line. will set EAX, ECX,BL appropriately. vii. Shift EAX until AL contains the upper nibble of xxii. Read the clusters by restoring the segments and EDX for printing a hexadecimal number in free the cluster chain for the old directory. EAX. xxiii. Read the cluster chain for the current directory viii. Finally print character by setting 10h at 0x0e. and write the new segment address and print the ix. To print a formatted string, DS:SI is used. message if error occurs. x. For reading a line,ES:DI is used. xxiv. Change the current directory according to the Table 9: Console Subroutine path specified in ES:DI by letting BX points to the beginning of the current subdirectory name. i. Compare two strings at DS:SI and ES:DI and set xxv. Create the main loop of the shell. the ZF pointer. xxvi. The command will be read into the command ii. Find the length of a string at ES:DI and return it buffer. in CX. xxvii. Write the codes for all the commands to be iii. Increment CX till the length of the String and executed. copy DS:SI into ES:DI. xxviii. Define the system call for shell functions Table 10:String Subroutine where BP is the function number.The function i. Include the string routine. number for different shell functions are given in ii. Initialize the shell command buffer size to 256 so Table 13. that the commands can be only upto 256 xxix. Preserve DS and set it to the kernel data segment characters long. for the output. iii. Initialize the LS lines per page to 20 so that the xxx. Finally,The output for shell initializing will be LS listing will pause after every 20 entries. displayed. iv. Initialize the current status structure of the Table 11:Shell Subroutine

shell.The current status structure of the shell is i. FAT Context Segment given in Table 12. ii. Current Directory Segment v. Initialize the shell where GS will point to the iii. Current Directory Length FAT Context structure of the root partition. iv. Command Buffer Segment vi. Register all the interrupts. v. Enumerated Current Directory Position vii. Preserve the FAT context segment so that the vi. Enumerated Current Directory Segment root directory can be read. vii. Previous Directory Segment viii. Make the FAT context segment the current one. viii. Previous Directory Length ix. Update the current directory status. ix. Previous Directory Valid x. Free the current directory so that the buffers and x. Free Current Directory command buffer can be allocated. xi. Signature xi. Check the shell status signature to detect Table 12:Current Status Structure of the Shell corruption. xii. Prepare to enumerate the current directory by i. 0 for init process of Enumerated Directory reseting the enumeration position of the current ii. 1 for next entry of Enumerated Directory directory. iii. 2 for Find Entry xiii. Parse the next directory entry if it exists. iv. 3 for Read File xiv. Set the return registers according to FAT32 v. 4 for Change Directory parsed Directory Entry so that the end of the vi. 5 for Reading File Path current directory can be known. vii. 6 for Changing Directory Path xv. Set FS to the correct segment and parse the Table 13: Function number for various shell functions directory entry. xvi. Increment the segment and the position

Operating System Development from Scratch

61 International Journal of Advances in Electronics and Computer Science, ISSN(p): 2394-2835 Volume-6, Issue-11, Nov.-2019 http://iraj.in Finally, the kernel boot Message for successful kernel vi. Store the value. booting, finding partition, reading partition, and vii. Add the FAT lengths (CX * sectors Per FAT). mounting partition was printed. viii. Perform the operation of EAX*CX -> EDX:EAX. IV. DEVELOPMENT OF FAT 32 PARTITION ix. Add the previous offset. FILE SYSTEM x. Store the value in EAX. xi. Copy the root cluster number. Since its impossible to generally restrict a file’s size xii. Find the number of directory entries per cluster. less than 512 bytes i.e. size of a sector, clusters was xiii. Multiply by 16 (directory entries per sector). formed which can be said as the combination of xiv. Store the value in GS. sectors. For accessing clusters, a file system was built xv. Find the amount of bytes per cluster. so that is why a FAT32 partition file system was xvi. Multiply CX (sectors per cluster) by 512 (the created in our operating system. For the creation of sector size). FAT32 driver, the Partition Structure, Bios Parameter xvii. Store the value in GS. Block(BPB), FAT32 context structure,FAT32 directory entry and FAT32 Disk Address Packet in the xviii. Jump to the next sector and perform same operations. storage space were initialized which are given in table Table 17: Allocation of memory for the sectors 14,table 1,table 15,table 16,table 3 respectively. Next, a FAT entry for a given cluster was read. The i. Start Cylinder, Head, Sector bits steps for reading a FAT entry for a given cluster is ii. System Id given in Table 18. iii. End Cylinder,Head,Sector bits iv. Starting of Logical Block Address i. Allocate a sector for reading the FAT into (FS v. Size of the partition will point there). Table 14: Partition Structure ii. Find the correct sector of the FAT and the offset into that sector. i. Drive Number iii. Multiply EAX by 4 (the size of a FAT32 entry) ii. Sectors Per Cluster and divide it by 512 (the size of a sector). iii. Bytes Per Cluster iv. Add the FAT offset. iv. FAT Offset v. Read the sector. v. Data Offset vi. Finally,restore the registers. vi. Directories Per Cluster Table 18: Reading a FAT entry for a given cluster vii. Root Cluster Next, a given cluster (numbered EAX) was read into viii. Signature the buffer at FS:0.The steps for reading a given cluster ix. Size into the buffer at FS:0 is given in table 19. Table 15:FAT32 Context Structure i. Subtract the cluster offset. i. Filename ii. Find the count of sectors past to be read. ii. Attributes iii. Perform the operation of EAX*ECX -> iii. Reserved EDX:EAX. iv. Created Time iv. Add the offset of the first data sector. v. Created Date v. Read the sector. vi. Last Accessed Date Table 19: Reading a cluster into the buffer at FS:0 vii. First Cluster Next, allocation of memory need to be done along with viii. Last Modified Time reading a cluster chain (starting from EAX) into it. The ix. Last Modified Date steps for reading a cluster chain into memory are given x. File Size in Table 20. xi. Size Table 16: FAT32 Directory Entry i. Find out the length of the cluster chain first.

Next, mounting of FAT32 partition file system was ii. Read the index of the next cluster. done and memory for the sectors was allocated iii. Allocate the buffer for the cluster chain. recursively. Steps for allocation of memory for the iv. Read the cluster EAX. sectors recursively are given in table 17. v. Read the index of the next cluster. vi. Read another cluster until an EOF is i. Read the first sector of the partition. encountered. ii. Store the drive number. Table 20: Reading a cluster chain into memory iii. Store the number of sectors per cluster. Next, a directory entry was read at FS:0.The steps for iv. Find the FAT offset. reading a directory entry are given in table 21. v. Add the number of reserved sectors.

Operating System Development from Scratch

62 International Journal of Advances in Electronics and Computer Science, ISSN(p): 2394-2835 Volume-6, Issue-11, Nov.-2019 http://iraj.in i. Load the cluster number. VIII. CONCLUSION ii. Load the attributes. iii. Copy the filename from FS:DI into DS:SI. Our Operating System successfully got developed iv. Load a character of the filename. from scratch just by using the Assembly Language. v. Store the character. Thus, we were successful in demonstrating how an vi. Check to see if we reached the 8-character limit.. operating system is built from scratch and how a entire vii. Copy the filename. computer system actually gets built and how it works viii. Store it in DL register. internally. Table 21: Reading a directory entry at FS:0 REFERENCE Finally, the FAT32 partition file system was [1] Broken Thorn OS Development Series developed. (https://www.brokenthorn.com). [2] James M’s OS Development Tutorials V. DEVELOPMENT OF DISK IMAGE AND (https://www.jamesmolloy.co.uk). MEMORY ALLOCATION [3] Brans OS Development Tutorials (https://www.osdever.net/tutorials/view/brans-kernel-develop ment-tutorial). For building the disk image we used Device Driver, [4] Mike OS Development MKFS and parted to partition a disk space into (https://mikeos.sourceforge.net/write-your-own-os.html). 16384(2^14) block where each block size was 512 [5] OS Development by Dr. Nicholas Blundell (https://www.cs.bham.ac.uk/~exr/lectures/opsys/10_11/lectur bytes. Thus, the size of the disk was ((2^14)*(2^9)) es/os-dev.pdf). which was (2^23) bytes or 8 MB. Thus, our disk [6] Lightning OS Development Tutorials memory size was of 8 MB. (https://os-development.000webhostapp.com). [7] How to Make a Computer Operating System by Samy Pesse VI. ENTIRE FLOW OF OUR OPERATING (https://github.com/SamyPesse/How-to-Make-a-Computer-O SYSTEM perating-System) . [8] Cfenollosaostutorial (https://github.com/cfenollosa/os-tutorial) . The entire flow of our operating system is given in [9] Creating Your Own Operating System by Pritam Zope figure 1. (https://www.createyourownos.blogspot.com) . [10] FlingOS Getting Started Tutorials (https://www.github.com/FlingOS/Getting-Started-Tutorials). [11] Iknowbrain OS Development videos (https://www.youtube.com/iknowbrain). [12] EduOS (https://github.com/RWTH-OS/eduOS). [13] BeeOS (https://github.com/davxy/beeos). [14] Write your own Operating System videos (https://www.youtube.com/writeyourownoperatingsystem). [15] Yaxos (https://gitlab.com/SopaXorzTaker/yaxos). [16] BareMetalOS (https://www.returninfinity.com). [17] Bare Bones Tutorial(https://wiki.osdev.org/Bare_Bones). [18] Littleosbook (https://github.com/littleosbook/littleosbook). [19] OSDev Wiki (https://wiki.osdev.org). [20] Baby Step Tutorials (https://wiki.osdev.org/Babystep1). Figure 1: Entire flow of Our Operating System [21] Independent Software Operating System Development Guide (https://www.independent-software.com/operating-system-de VII. TESTING AND RESULTS velopment.html).

Our operating system was tested in qemu emulator as well as in virtual box and in both the emulators our operating system worked perfectly. The demonstration of our operating system has been shown in figure 2.



Operating System Development from Scratch

63